How Apache NiFi schema registry integration works for record flows
A team whose Kafka topics carry Avro or Protobuf messages needs a schema registry in its NiFi record flows when the flow must parse those messages without hard-coding field definitions. NiFi provides three registry controller services for this: ConfluentSchemaRegistry, which looks up schemas by Confluent subject name, ApicurioSchemaRegistry, which looks up schemas by artifact identifier, and AmazonGlueSchemaRegistry, which looks up schemas stored in the AWS Glue Schema Registry. The registry you choose should match the one your producers already write to, because the flow can only read what the registry holds under the name or identifier the record reader asks for. The two lookup styles differ in where the schema identity comes from. With a registry lookup, the record reader or writer is configured with a subject or artifact identifier, typically set by the flow designer or taken from an attribute, and the registry service returns the schema for it. The message bytes carry no schema information. With an encoded schema reference, the schema identity travels inside each message, and a reader such as ConfluentEncodedSchemaReferenceReader or AmazonGlueEncodedSchemaReferenceReader reads that reference, fetches the matching schema, and parses the record. The key decision is whether your producers embed the reference in each message, because that determines whether the flow can handle mixed schema versions on one topic without a per-topic configuration change. Before building, confirm with the producer team which serializer format and reference scheme they use, and which registry and version of that registry they run. Also confirm that the NiFi version you target includes the reader you need, since the encoded schema reference components are documented for NiFi 2.12.0 in the NiFi components index. Ask the registry owner who may read which subjects or artifacts, because the NiFi controller service will need credentials with read access that match your access model.
Which NiFi registry service fits Confluent, Apicurio or AWS Glue
The ConfluentSchemaRegistry controller service looks up schemas by Confluent subject name, which is the identifier the Confluent registry uses to group schema versions for a topic or record type. NiFi resolves the subject to a schema version, so the flow needs the correct subject name for each topic it reads or writes. When messages carry a schema ID in their payload, NiFi 2.12.0 also lists ConfluentEncodedSchemaReferenceReader and ConfluentEncodedSchemaReferenceWriter, which read and write that reference inside the message content.
- Lookup key: the subject name, often derived from the topic name with a suffix such as
-value, so confirm the naming strategy in use. - Encoded reader and writer: ConfluentEncodedSchemaReferenceReader and ConfluentEncodedSchemaReferenceWriter in NiFi 2.12.0 handle schema IDs embedded in messages.
- Version selection: confirm whether consumers should use the latest version or a pinned version, because subject evolution rules decide which changes a flow can absorb.
- Authentication: confirm the access method, credentials and TLS trust chain the registry requires, and where those secrets are stored.
- Compatibility mode: confirm the registry compatibility setting for each subject, since it determines whether schema changes are accepted.
Ask the registry owner for the subject naming convention, the compatibility mode for each subject in scope, and whether messages carry schema IDs in the Confluent wire format.
The ApicurioSchemaRegistry controller service looks up schemas by artifact identifier. An artifact identifier points to a schema definition in Apicurio, and the flow must know which identifier, and optionally which version, applies to each stream of records. Apicurio is often chosen where teams want a registry that stores Avro, JSON Schema and Protobuf artifacts side by side. NiFi's documentation lists the ApicurioSchemaRegistry component in the components index, but this page does not confirm a matching encoded schema reference reader or writer for Apicurio, so check the components index for the version you run before designing around one.
- Lookup key: the artifact identifier, which must be stable across environments if flows are promoted with parameter contexts.
- Encoded references: confirm whether producers embed a global ID or content ID in messages, and whether a matching reader exists in your NiFi version.
- Version handling: confirm whether consumers resolve the latest artifact version or a fixed one.
- Schema type coverage: confirm which formats the record readers in your flow need, and that Apicurio stores them with the expected type.
- Network path: confirm the registry endpoint is reachable from every NiFi node, including nodes in a cluster.
Ask the registry owner for the artifact identifiers in scope, the version policy, and how producers reference schemas inside messages.
The AmazonGlueSchemaRegistry controller service looks up schemas stored in the AWS Glue Schema Registry. The flow references the schema by its registry and schema name, and NiFi retrieves the definition through the AWS credentials and region configured for the service. For messages that carry the schema reference inside the payload, NiFi 2.12.0 lists AmazonGlueEncodedSchemaReferenceReader, which reads that reference. Writers for the Glue encoded format are not listed in the same component set, so confirm the writing path before committing to it.
- Lookup key: the schema stored in the Glue registry, identified by registry and schema name within a given AWS account and region.
- Encoded reader: AmazonGlueEncodedSchemaReferenceReader in NiFi 2.12.0 reads schema references embedded in messages.
- IAM permissions: confirm the role or credentials NiFi uses have read access to the registry and schema, and that the policy is scoped to the required resources.
- Region and account: confirm the region and account that hold the schemas, and whether cross-account access is needed.
- Versioning: confirm which schema version producers write and whether consumers should follow new versions automatically.
Ask the registry owner for the registry and schema names, the IAM policy that will apply to NiFi, and the region and account in each environment.
Configuration options and when each one fits
This table compares the three registry controller services and the encoded schema reference components by registry, lookup key and where the schema identifier lives. Read each row as a pairing: the registry service resolves a schema from outside the message, while the encoded reference components read the identifier from the message content itself. The main takeaway is that the registry you run and the way your producers write schema identifiers decide which components you configure.
| Option | Lookup key or identifier location | Record readers and writers it feeds | When it fits |
|---|---|---|---|
| ConfluentSchemaRegistry | Confluent subject name, resolved against the registry outside the message | Avro, JSON or Protobuf record readers that take a schema from a registry service | Kafka topics whose producers register schemas under Confluent subject names |
| ApicurioSchemaRegistry | Artifact identifier, resolved against the Apicurio registry outside the message | Record readers that take a schema from a registry service | Teams standardized on Apicurio artifacts rather than Confluent subjects |
| AmazonGlueSchemaRegistry | Schema stored in the AWS Glue Schema Registry, resolved outside the message | Record readers that take a schema from a registry service | Flows on AWS where schemas are governed in Glue and access runs through AWS identity |
| ConfluentEncodedSchemaReferenceReader | Schema identifier encoded in the message content | Record reader that fetches the schema named in each message | Producers that embed Confluent schema IDs in every payload |
| ConfluentEncodedSchemaReferenceWriter | Schema identifier written into the message content | Record writer that embeds the Confluent reference on output | Consumers downstream expect Confluent-encoded messages |
| AmazonGlueEncodedSchemaReferenceReader | Schema identifier encoded in the message content for Glue | Record reader that fetches the schema named in each message | Producers that encode Glue schema references in payloads |
| Connection settings | Registry URL, credentials and TLS trust material, to be confirmed against each component's property list | Applies to every registry service above | Confirm property names in the NiFi component documentation for your NiFi 2.x release before building |
| Authentication | Registry credentials or AWS identity, depending on the registry | Applies to the registry service controller, not the reader | Confirm the supported auth mode for your registry version and keep secrets in sensitive parameters |
| Hortonworks Schema Registry | Deprecated in NiFi 1.20.0, so plan a replacement rather than a new build | Migrate existing readers to one of the supported registry services | Existing flows only, with a documented cutover to a supported registry |
Setting up a registry lookup in a NiFi record flow
This sequence takes a NiFi 2 record flow from a confirmed runtime to a tested reader and writer that resolve Avro, JSON or Protobuf schemas from an external registry. Each step ends with a check so that schema resolution and malformed-message handling are proven before the flow runs against live data.
Confirm the NiFi 2 version and Java runtime
Start by confirming that the target cluster runs a NiFi 2 release and a Java runtime that the NiFi administration guide lists as the minimum, which is Java 21. If the flow will use Python-based processors, confirm a supported Python version as well, noting that the guide describes those processors as a beta feature. The check is a recorded version output from every node, with the same Java and NiFi version on each one. If any node differs, fix that before continuing, because registry services behave differently across mixed nodes.
Enable and configure the registry controller service
In the NiFi controller settings, add the controller service that matches your registry: ConfluentSchemaRegistry, ApicurioSchemaRegistry or AmazonGlueSchemaRegistry. Set the registry URL or AWS region, and the authentication values your registry requires, as sensitive parameters where possible. Enable the service and confirm it reaches the enabled state without validation errors. If it stays invalid, check the endpoint, the trust store used for TLS and the credentials with your registry administrator, since these are the usual causes of failure at this step. Document which registry and which identifier scheme (subject name, artifact identifier or Glue schema name) the flow will use.
Attach the registry to a record reader
Add a record reader for your format, such as an Avro or JSON reader, and set its schema access strategy to use the registry service. Choose the lookup key that fits your registry: a schema name for ConfluentSchemaRegistry subjects, an artifact identifier for ApicurioSchemaRegistry, or a schema name for the AWS Glue registry. Where messages carry an encoded schema reference in their content, use the matching encoded schema reference reader instead, such as ConfluentEncodedSchemaReferenceReader or AmazonGlueEncodedSchemaReferenceReader, which ship with NiFi 2.12.0. The check is that the reader validates as enabled, with no unresolved schema reference warnings in the processor configuration.
Test the lookup with sample messages
Place a known-good sample Avro, JSON or Protobuf message in a test input, and run a single processor that uses the reader to confirm the schema resolves. Use a message produced against the same registry version your production producers use, and repeat the test for each format and schema version the flow will receive. Inspect the output FlowFile attributes and content to confirm that fields parse as expected. The check is a successful parse with the registry schema, and a recorded version or identifier for each schema tested. Do not start the flow until every sample resolves.
Route malformed messages to a failure relationship
Send a deliberately malformed message, such as one with a schema identifier the registry does not hold or a payload that does not match its declared schema, through the same reader. Confirm that the processor routes it to the failure relationship rather than stopping the flow or passing it downstream. Connect the failure relationship to a retained queue or a dead-letter path that your team monitors. The check is that the malformed FlowFile arrives in that path with its original content and error attributes intact, and that valid messages continue to flow to success.
Wire the record writer and confirm schema output
Add a record writer for the output format and configure its schema access strategy. Decide whether the output schema comes from the same registry, from an explicit schema, or from a schema the writer inherits from the reader, and document that choice for downstream consumers. If the output goes back to Kafka with an encoded schema reference, pair the writer with the matching encoded schema reference writer. Run the test FlowFile through the reader and writer together, then check that the output content and any schema reference match what the consumer expects. The check is a round-trip output that a downstream consumer can decode.
Start the flow and verify on the live path
Start the processors in the order of input, reader, writer and output, and send a small batch of real messages from a test producer. Watch the failure and success counts, and confirm in the data provenance view that each FlowFile shows a successful schema lookup. Record the registry version used for each message so that schema changes can be traced later. The check is a clean run with no unexpected failures and provenance entries for every test message. Only then should the flow move to the production input.
Replacing the Hortonworks Schema Registry service in existing flows
The HortonworksSchemaRegistry controller service was deprecated in NiFi 1.20.0, so any flow that still references it needs a planned replacement with one of the supported registry services: ConfluentSchemaRegistry, ApicurioSchemaRegistry or AmazonGlueSchemaRegistry. Start with an inventory. List every controller service that points at a Hortonworks registry, then list every record reader and record writer that uses one of those services as its schema access strategy. Readers and writers are easy to miss because they often sit inside processors such as ConvertRecord, QueryRecord and PublishKafka, and a single controller service can feed many of them across several process groups. Record the process group, the schema identifier each reader expects and the Kafka topics or files involved.
Next, map each schema source to a target registry. Decide which registry holds the authoritative copy of each schema, since the Confluent subject name, the Apicurio artifact identifier and the AWS Glue schema name are not interchangeable keys and must be derived from the existing identifiers in a deliberate way. Where a message carries an encoded schema reference, check whether the ConfluentEncodedSchemaReferenceReader, ConfluentEncodedSchemaReferenceWriter or AmazonGlueEncodedSchemaReferenceReader can read it without changing producers. Then test compatibility before cutover: run sample Avro, JSON and Protobuf records through a replacement reader against the target registry, and confirm that every schema version still in use resolves and parses. Confirm the target registry's schema format support in your own environment, because format coverage differs by registry and by NiFi release.
Flow migration from NiFi 1.x to 2.x is covered by the upgrade pages, so this section assumes the flow is already moving to a 2.x release. Keep the registry swap as a separate change so that a failed cutover can be rolled back without also reverting the runtime upgrade. Plan the cutover with the owners of each producing and consuming system, and keep the old service enabled but unused until downstream readers have run cleanly on the new one. The NiFi component documentation for each replacement service is the reference for its properties, and the Apache NiFi components index lists the controller services available in the release you run.
Security and credentials for registry access
A schema registry decides how NiFi reads every record in a flow, so access to it needs the same care as access to the data itself. The cards below cover where registry credentials live, how registry endpoints are trusted, how access is scoped, and how schema changes and version drift are controlled.
Keep registry credentials as sensitive parameters
Registry connections often need credentials or tokens that must not appear in flow exports or the UI. Store them as sensitive parameters in a parameter context and reference them from the controller service properties, so they are encrypted at rest and masked for users. Check that the flow definition committed to Git contains parameter references only, not secret values, and confirm which authentication options your registry version supports before you design the parameter set.
Trust the registry endpoint explicitly
Each registry service calls an HTTPS endpoint, and the NiFi JVM must trust the certificate chain that the registry presents. Decide whether the registry uses a private certificate authority, add that CA to the truststore used by the controller service, and test the connection from every node, not only the one you configured. Verify that certificate rotation on the registry side does not silently break record parsing on the NiFi side.
Scope access to subjects and artifacts
Confluent subjects, Apicurio artifacts and Glue schemas are separate namespaces that can be granted or denied independently in their own platforms. Give the NiFi service identity read access only to the subjects, artifact groups or schema names its flows actually parse, and keep write or delete rights with the team that owns the schema lifecycle. Document which NiFi process groups depend on which subjects so a permission change can be assessed before it reaches production.
Limit AWS IAM scope for Glue
The AmazonGlueSchemaRegistry service calls AWS on behalf of the NiFi host, so its IAM policy defines what the flow can read. Grant read actions on the specific registry and schemas used, restrict the region and account, and prefer an instance or workload role over long-lived access keys. Confirm the exact action set against current AWS Glue documentation, because the required permissions depend on whether the flow only reads schemas or also registers them.
Audit schema changes that alter parsing
A new schema version can change field names, types or defaults, and a record reader that resolves that version may route records differently or fail them without any NiFi configuration change. Log which schema identifier and version each flow resolved, alert on new versions for critical subjects, and review compatibility rules in the registry before a change is promoted. Provenance and flow logs help show which records were read under which schema.
Align registry and NiFi versions in upgrades
Registry clients and encoded schema reference readers are shipped with NiFi, so upgrading NiFi can change how these components behave even when the registry itself stays the same. Plan NiFi and registry upgrades in one change window, test record parsing against a copy of production schemas, and check the NiFi migration guidance for any component that changed in the target release. Keep the target NiFi version and the registry API versions in your change record.
Questions teams ask before integrating a schema registry with NiFi
How much effort does a schema registry integration in NiFi take?
Effort depends mostly on how many schemas, formats and topics the flows touch, and on how many registries are involved. The main work items are configuring the registry controller service, choosing how each record reader and writer finds its schema, testing compatibility rules against real messages, and setting up credentials and TLS for each registry. Each registry (Confluent, Apicurio or AWS Glue) has its own lookup model, so a flow that mixes them takes more design time. Ask any partner for a quote built from a written inventory of flows, schema formats (Avro, JSON or Protobuf), registry types and the number of encoded schema references in messages. A quote should separate one-time build work from any recurring support.
Can our own team build the registry integration without outside help?
Yes, if your team already runs NiFi in production and has someone comfortable with record readers, writers and controller service configuration. The NiFi components documentation covers ConfluentSchemaRegistry, ApicurioSchemaRegistry and AmazonGlueSchemaRegistry, so the configuration itself is well documented. Teams most often need outside help with the parts that span systems: agreeing subject or artifact naming, handling schema evolution safely, and testing with production-like messages. Assess your team against those skills, then decide whether a short review or a full build fits your situation.
What do our registry and Kafka owners need to provide?
The registry and Kafka owners control the facts the NiFi flows depend on, so they should be involved before design starts. Typically you need the registry endpoint and its authentication method, the naming convention for subjects or artifacts, the compatibility rules applied to each subject, and the topics and access rights NiFi will use. For Kafka, the team should confirm how clients authenticate and which ACLs apply. Kafka 4.x supports SSL and several SASL mechanisms, and in KRaft clusters resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, so access must be granted explicitly. Confirm these details against the Kafka authorization documentation for your Kafka version.
How long does a pilot take to validate the integration?
The pilot length depends on how many schemas it must cover and how varied the format mix is. A pilot that covers one registry, one format and a few subjects can be validated quickly. A pilot that proves Avro, JSON and Protobuf across more than one registry, including schema evolution and encoded references in message content, takes longer because each combination needs its own test messages. Agree on a pass condition before starting: which topics, which schema versions, and which failure cases (unknown schema, incompatible version, unreachable registry) must behave as specified. A fixed-scope pilot with a written test list gives you a clear end point and a clear cost.
Can we replace Hortonworks registry flows without rewriting every processor?
In many cases, yes, but it depends on how the flows reference the schema service. The HortonworksSchemaRegistry controller service was deprecated in NiFi 1.20.0, so it should be replaced rather than kept indefinitely. If record readers and writers point to a single controller service, you may be able to swap in a supported registry service and keep most processors in place. Flows that rely on Hortonworks-specific schema identifiers or encoding will need changes to how schemas are resolved and referenced. Inventory every flow that uses the service and test each one against the new registry before cutover. Your existing NiFi version also matters, since NiFi 1.x is out of maintenance and planning a move to 2.x may affect the same flows.
Does adding registry services change how NiFi is licensed?
No. Apache NiFi is an Apache Software Foundation project released under the Apache License 2.0 (SPDX: Apache-2.0), and the project itself has no paid edition. The Confluent, Apicurio and AWS Glue registry services are bundled components of NiFi, so adding them does not introduce a NiFi license fee. However, the registries themselves may carry their own terms. Confluent Schema Registry, Apicurio Registry and AWS Glue are separate products or services with their own licensing and hosting costs, and those must be checked with each provider. Your cost quote should list those third-party costs separately from the NiFi integration work.
Kafka settings that affect schema-encoded messages
The Kafka release you run sets the client, broker and security behavior that your NiFi consumer and producer flows depend on. The Kafka downloads page lists 4.3.1, 4.2.2 and 4.1.2 as the supported releases, and earlier versions are archived. Pick one of these lines before you build the flows, then confirm the Java runtime and client library versions that the NiFi Kafka processors will load against it. Kafka 4.0 and later run only in KRaft mode, so a cluster still on ZooKeeper must migrate first. Schema-encoded messages carry their schema reference inside the payload, so a mismatch between the producer and consumer settings in the Kafka layer shows up as failed record reads in NiFi, not as a clear connection error.
Authentication and encryption are set on the Kafka cluster and must match what NiFi presents. The Kafka security overview documents SSL for encryption and SASL for client authentication, with GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER as the SASL mechanisms. Confirm which listener the NiFi processors will use, whether the broker certificates chain to a trust store NiFi can read, and which SASL mechanism the NiFi principal is provisioned with. Keep the TLS and SASL settings in parameter contexts so that the same flow can move between test and production clusters without edits.
Authorization is the check most often missed. In KRaft clusters, the StandardAuthorizer stores ACLs in the metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true. A NiFi consumer that reads the schema-encoded topic therefore needs explicit read access on the topic and its consumer group, and the producer flow needs write access. Confirm these ACLs for the NiFi principal in each environment before the first run. Also confirm that the registry lookups in NiFi can reach the schema registry over the same network path, since a permission gap on either side produces errors that look similar.
Official sources for registry services and Kafka
The three component pages describe how NiFi resolves a schema from each external registry. Start with the page for the registry you already run, because the lookup key differs in each case: the Confluent service looks up schemas by subject name, the Apicurio service looks up schemas by artifact identifier, and the AWS Glue service reads schemas stored in the AWS Glue Schema Registry. Your record reader or writer depends on these properties, so the page is the authority for what each controller service expects.
- Confluent lookup: The ConfluentSchemaRegistry page documents subject-based lookup and the properties that point NiFi at your registry endpoint.
- Apicurio lookup: The ApicurioSchemaRegistry page explains artifact-identifier lookup, which differs from subject naming and affects how you map topics to schemas.
- Glue lookup: The AmazonGlueSchemaRegistry page covers schemas held in AWS Glue and the AWS credentials that the service uses.
- Per-service settings: Record each property name, its default and whether it is sensitive, since these values move into parameter contexts later.
- Schema format support: Confirm on each page which schema formats the service resolves before you choose Avro, JSON or Protobuf.
Before you rely on any lookup behavior, confirm the exact property names and supported formats on the page for the NiFi version you deploy, because the component documentation is versioned with the release.
The NiFi components index is the place to find the encoded schema reference readers and writers, which are different from the registry services. A registry service resolves a schema by name or identifier, while an encoded schema reference reader reads a reference to the schema that is embedded in the message bytes. Confirm which of these you need before you add components, because a flow that expects embedded references will not work with a registry lookup alone.
- Confluent references: The index lists ConfluentEncodedSchemaReferenceReader and ConfluentEncodedSchemaReferenceWriter for schema IDs carried in message content.
- Glue references: AmazonGlueEncodedSchemaReferenceReader covers schema references encoded for AWS Glue.
- Version scope: These readers appear in the NiFi 2.12.0 component set, so check the index against the version you actually run.
- Bundle presence: Confirm that the bundles containing these components are installed in your NiFi distribution.
- Lookup pairing: Each reference reader normally pairs with a matching registry service, so note both in your design.
Before you choose a reader or writer, open the components index for your NiFi version and confirm that the component names and bundles match your deployment.
The deprecation record explains why a flow built on the Hortonworks registry needs replacing rather than simply keeping its existing configuration. The HortonworksSchemaRegistry controller service was deprecated in NiFi 1.20.0, and this page is the reference for that status and for the other components the project has retired or marked for change. Teams moving from 1.x to 2.x should read it alongside the migration guidance.
- Deprecation point: The HortonworksSchemaRegistry service was deprecated in NiFi 1.20.0, so it is a candidate for replacement in any flow still using it.
- Replacement mapping: Map each Hortonworks schema lookup to a Confluent, Apicurio or Glue service, based on where your schemas are actually stored.
- Inventory: Search your flow definitions for every controller service that references the Hortonworks registry, including those inside process groups and parameter contexts.
- Other deprecations: Check the same page for any other components in your flows, since a replacement project often reveals more than one.
- Upgrade order: Plan replacements before a version move, so the flow is tested on its new service and not on a removed one.
Before you start the replacement, confirm the current deprecation list for your NiFi version on the wiki page, because the status of individual components can change between releases.
The Kafka security overview covers the transport and authentication that NiFi must match when it reads from or writes to a topic. A schema lookup can succeed while the Kafka connection fails, so the security model of the cluster should be settled before the registry integration is tested. This page describes which channels can be encrypted, which authentication mechanisms exist, and where the trust boundary sits.
- Encryption: Kafka uses SSL/TLS between clients, brokers and tools, and security is optional, so confirm that your listeners are all secured before you connect NiFi.
- Authentication options: The documented SASL mechanisms include GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER, plus SSL client authentication.
- Mixed listeners: Clusters can run secured and unsecured listeners side by side, so confirm that the NiFi connection targets a secured listener.
- Client configuration: Match the NiFi Kafka controller service settings to the mechanism and certificates the broker expects.
- Registry access: Secure the registry endpoint separately, since the Kafka security model does not cover it.
Before you connect NiFi, confirm the listener, authentication mechanism and certificate chain that your Kafka cluster actually enforces, because the overview describes options and your cluster configuration decides which one applies.
How DeployOpen scopes and delivers registry integration
Registry integration fails most often at the edges: which schema source is authoritative, which flows touch it, and what the Hortonworks replacement must preserve. The four steps below connect inventory, a written scope, tested record flows and a handover so that your team can run the result after delivery.
Inventory schema sources and flows
The work starts with a list of every schema source, every flow that resolves schemas, and every topic or file those flows read and write. For each flow, we record which registry service it uses today, whether messages carry an encoded schema reference, and whether any flow still points at the Hortonworks Schema Registry service. The inventory decides the scope, because a flow that reads Avro with an embedded schema needs different work from one that resolves a Confluent subject name at runtime.
Agree a fixed written scope
The scope names the registry services in use (Confluent, Apicurio, AWS Glue or a combination), the encoded schema reference readers and writers, the flows to change and the Hortonworks replacement path. It also states what is excluded, such as schema governance decisions owned by your data team. Most engagements are fixed-scope, so the scope, deliverables and timeline are agreed in writing before work begins and a quote follows from that document.
Build and test against sample messages
Record flows are built and tested against sample messages captured from your own topics or files, including the schema reference bytes that Kafka producers actually write. Tests cover successful resolution, unknown subject or artifact identifiers, schema evolution and malformed payloads routed to failure relationships. Credentials for each registry are handled through parameter contexts so that test and production use the same flow definitions.
Hand over documentation and runbooks
Handover includes a flow map showing which registry service each record reader and writer uses, a runbook for changing a schema in the registry, and steps for diagnosing a failed lookup. Your operators receive the procedures for registry credential rotation and for checking flows after a NiFi upgrade. Ongoing support, upgrades and continuous testing are available as retainers, so you can choose between internal ownership and a support agreement.
