The Re-Derivation Test: If You Can't Reproduce It, It Wasn't Evidence

Your auditor looks up from the evidence package and asks a simple question: "Can you show me this control is still operating? Right now?"

Your compliance lead freezes. The evidence was collected in February by an analyst who's since left the team. It came from a screen in the Azure portal that nobody remembers navigating to. The screenshot shows a configuration, but nobody documented which blade, which subscription, or which policy produced it. Re-creating that evidence would require reverse-engineering the dead analyst's process.

The auditor notes the gap. The control, which looked "complete" in the GRC platform 30 seconds ago, is actually unverifiable. It passed the existence test (evidence was collected at some point) but failed the re-derivation test (the evidence can't be reproduced on demand from the same source).

That distinction separates operating controls from documented ones.

What the re-derivation test is

The re-derivation test asks one question: can you produce the same evidence again, right now, from the same source system, through the same integration path?

If yes: the control is operating. The source system still exists, the integration still works, the configuration is still present, and the collection path is still valid. You've demonstrated not just that the control existed at some point, but that it continues to function.

If no: you collected a point-in-time artifact that proves nothing about today. Maybe the source system changed. Maybe the integration broke. Maybe the configuration was modified. Maybe nobody knows where the evidence came from in the first place. In any case, the "evidence" is a historical artifact, not proof of current state.

This test seems obvious when stated plainly. But the majority of evidence in traditional GRC platforms would fail it. Not because the underlying controls have failed, but because the collection process was manual, undocumented, and non-repeatable.

Why most evidence fails re-derivation

Evidence fails the re-derivation test for predictable reasons:

Each of these failure modes is independent. A single piece of evidence might fail for multiple reasons simultaneously. The analyst who collected it left, the portal UI changed, and the process was never documented. Re-derivation is impossible three times over.

The maturity distinction: Implemented vs. Operating

In CMCAE (Continuous Maturity and Compliance Assessment Engine), controls progress through maturity levels. Two levels are relevant here:

Level 2, Implemented: The control exists. A policy is written. A configuration is deployed. Evidence shows the control was put in place at some point.

Level 3, Operating: The control is functioning. It can be verified now. Evidence is reproducible. The control isn't just deployed, it's actively enforced and demonstrably current.

The re-derivation test is the gate between Level 2 and Level 3. A control with evidence that can't be reproduced is implemented (maybe) but not demonstrably operating. It sits at Level 2 regardless of how confident your team feels about it, because confidence without verification is assumption.

Most organizations claim their controls are "operating" based on the fact that nobody has reported a failure. But absence of reported failure is not evidence of operation. The re-derivation test forces you to prove it positively: produce the evidence again, right now, from source.

The jump from Level 2 to Level 3 is where most organizations get stuck. They've deployed controls but can't prove, on demand, that those controls are still doing what they were designed to do.

How Kyudo implements re-derivation

The Evidence Hub solves re-derivation by design. Every evidence collection is defined as a pipeline, not a one-time action.

The derivation path is the evidence. When Kyudo collects evidence, it doesn't just store the artifact. It stores the complete derivation path: which integration, which API endpoint, which query parameters, which tenant, which resource scope. The path is as much a part of the evidence record as the artifact itself.

Re-derivation is a button press. Any evidence artifact can be re-derived at any time by re-running its pipeline. Same integration, same endpoint, same scope. The new artifact is compared against the previous one. If they match, the control is operating unchanged. If they differ, drift is flagged.

Automated re-derivation on schedule. The system doesn't wait for someone to press a button. Evidence pipelines run on defined schedules. High-risk controls re-derive daily. Medium-risk controls re-derive weekly. The re-derivation test runs continuously, not just when an auditor asks.

Here's the pipeline architecture for a typical evidence collection:

Source systems: Microsoft Defender XDR (security posture and threat signals), Sentinel (event telemetry and incident data), Purview (data classification and governance), Entra ID (identity configuration, Conditional Access, authentication methods), Azure Policy (resource compliance state).

Collection path: Each source system connects via a native integration. The Evidence Hub authenticates with least-privilege credentials, calls the relevant API, and retrieves the full configuration or telemetry object. The path is deterministic: same input parameters always query the same resource.

Artifact storage: The raw response is stored in the Compliance Graph with its SHA-256 hash, collection timestamp, and full pipeline metadata. The artifact is immutable after storage.

Lineage preservation: The Compliance Graph maintains typed relationships from the artifact to its derivation pipeline, its source system, the control(s) it satisfies, and the framework requirement(s) those controls map to.

Diff on re-derivation: When the pipeline runs again, the new artifact is compared to the previous one. Changes are flagged with field-level specificity. Not just "the Conditional Access policy changed," but "the grantControls.builtInControls array no longer includes mfa for the All Users assignment."

This architecture means every piece of evidence in the system is inherently re-derivable. The pipeline definition is the documentation. The scheduled execution is the verification. The diff history is the operating proof.

The counter-argument: "We can re-collect manually if asked"

Some teams argue that re-derivation doesn't need to be automated. "If the auditor asks, we can go back to the portal and pull it again." That's technically true, but it fails on three counts:

Time cost. Manual re-collection during an audit is expensive. The auditor is waiting. The analyst is scrambling. Multiply by the number of controls sampled and you're talking about days of unplanned work.

Knowledge dependency. Can every analyst on your team re-derive every piece of evidence? Or do specific artifacts require specific people who know where to look? If the answer is the latter, you have a bus-factor problem disguised as a compliance process.

Scope risk. When you manually re-collect, how do you ensure you're collecting from the same scope as the original? Same tenant, same subscription, same resource group, same policy? Manual re-collection introduces scope drift that automated pipelines eliminate by definition.

If re-derivation is a fire drill, your evidence model is fragile. If re-derivation is the normal operating mode, your evidence model is resilient.

Building a re-derivation culture

The re-derivation test isn't just a technical capability. It's a cultural standard for how compliance teams think about evidence.

The old mental model: evidence is something you collect and file. Once it's in the folder, the work is done.

The new mental model: evidence is something you produce continuously from source. The pipeline is the work. The artifact is a byproduct.

This shift changes how teams scope evidence requirements. Instead of asking "what do we need to show the auditor?" they ask "what signal from which source system, collected at what frequency, through which pipeline, proves this control is operating?" The answer to the second question is inherently re-derivable. The answer to the first is often a one-time screenshot.

The Controls Hub surfaces re-derivation status explicitly. Every control shows when evidence was last re-derived, whether derivation succeeded, and whether results matched the previous collection. Controls outside their freshness window are flagged for staleness. Controls whose re-derivation produced different results are flagged for drift.

This turns re-derivation from an audit event into an operational metric. Proven-operating (re-derived recently, result consistent) vs. assumed-operating (not re-derived within window, status unknown).

Why this matters for audit day

An auditor performing a SOC 2 Type II examination needs to test that controls operated effectively over the examination period. With a re-derivation-capable system, the auditor can ask "show me this control is operating right now" and receive current evidence within minutes. They can see re-derivation history across the entire period, not just sampled points. They can verify that the collection pipeline hasn't changed.

Fewer sampling requests. Fewer evidence gaps. Fewer "can you re-collect this?" cycles. The audit becomes a review of a continuous evidence stream rather than an evaluation of a manually assembled package.

Your Monday morning checklist

1. Run the re-derivation test on 5 controls. Pick five controls from your framework. For each one, try to reproduce the evidence right now, from the same source system. How many can you actually re-derive without tribal knowledge or manual portal navigation?

2. Document your derivation paths. For every piece of evidence, ask: which system, which API/blade, which scope, which credentials, which steps? If the answer is "someone opened the portal and found it," the derivation path is undocumented and non-repeatable.

3. Identify your Level 2 controls. Which controls have evidence that was collected once and never refreshed? Those are Level 2 (Implemented) at best, regardless of what your GRC platform says. They haven't passed the re-derivation test.

4. Calculate your re-derivation coverage. What percentage of your controls could be re-derived automatically, without human intervention, within the next hour? That's your actual operating confidence level. Everything else is assumption.

5. Set a re-derivation cadence. Define how frequently each control tier should be re-derived. Daily for high-risk, weekly for medium, biweekly for low. Build toward a state where re-derivation is the default, not the exception.

---

Kyudo's Evidence Hub stores the derivation path alongside every artifact. Re-derivation runs on schedule, compares results to previous collections, and flags drift automatically. Every control either passes the re-derivation test or drops from "operating" to "unverified."

Try the AI risk assessment to see how many of your controls would pass the re-derivation test today.