When you build or change a pipeline
After a mapping, join, transformation, or sink change, compare the completed new output against the reference output. Start withCOUNT_ROWS, then add SUM and COUNT_DISTINCT on fields affected
by the change. Use VALUE on stable join keys to compare individual fields,
and OUTER_VALUE when rows missing from either side must be included. Apply
Pair filters if the check covers a deliberate subset.
A passing result supports the release decision for those configured checks.
Widen coverage when a change can affect more datasets or fields than the
initial scope.
When pipelines run in production
Reconcile each completed load after both source and target outputs are ready. UseCOUNT_ROWS and meaningful aggregates as recurring checks, then add keyed
value comparisons for the fields where a discrepancy matters.
Enable bounded failed-row capture for value and set Tests when investigators
need examples of mismatches or missing records. The evidence stays in your
Azure Storage; an authorized browser retrieves it directly. See
Runs, results & evidence.
Use your existing orchestrator and the API to trigger checks after loads. The
current hosted application does not provide a built-in schedule editor. See
Validating on an ongoing basis.
When the platform or runtime changes
Before an integration runtime, Spark, or Fabric change, retain a representative reference output and a baseline of Tests. Run the same checks against the completed output after the change. Include counts, aggregates, keyed values, and the field types your workloads rely on. Review the selected scope and any tolerated differences. Coverage is limited to the Pairs and rules in the baseline. A hosted application update and an update to your customer execution runtime are separate events. Check Environment readiness and address any required runtime repair before submitting validation.When you migrate or cut over
Pair the old pipeline’s output with the migrated output. Start with row counts and aggregate reconciliation, then addVALUE and OUTER_VALUE on keyed
Pairs for the fields that must match.
Review the delivered outcomes and retained customer evidence before making a
cutover decision. Record the Run ID, scope, and results in your organization’s
approval process. The full workflow is in
Validating your ADF to Fabric migration.