Deployment drift as a security finding
When production provably differs from the reviewed source, what can an assessment still claim, and how should the uncertainty itself be treated?
Source review has a seductive built-in assumption: that the code on your screen is the code running in production. In a mature pipeline, that assumption is earned: known commit in, immutable artifact out, digest recorded, provenance attached, the same artifact promoted through every environment without being touched again.
In smaller shops the relationship is a lot flimsier. An image gets built by hand, pushed under a mutable tag, pulled onto a box, restarted through Compose. Someone edits infra config directly on the host at 11pm. A production fix exists live but was never committed. A control sits proudly in the repo and has never once reached production. At that point deployment drift stops being a footnote about assessment scope. It becomes a security condition in its own right, and I think it’s chronically under-rated as one.
The real issue isn’t that two systems differ. It’s that nobody can say anymore which system is authoritative, reproducible, or actually covered by whatever review happened.
What drift actually means
Deployment drift exists whenever the effective production system can’t be reliably reconstructed from the reviewed source, build config, and deployment records. It can show up anywhere: application code, container images, dependencies, env vars, migrations, proxy config, published ports, host services, security headers, credentials, logging, hotfixes.
Drift by itself doesn’t mean production is worse off. Maybe production has an uncommitted hotfix. Maybe the repo has a security improvement that hasn’t shipped yet. Maybe an operator quietly added a firewall rule that infra-as-code knows nothing about. The actual problem is that neither direction can be trusted without more evidence. A reviewer can’t safely say a source-level control protects production. An operator can’t safely say the repo is sufficient to rebuild the running system after a host dies. That uncertainty needs to be named out loud, the same way a vulnerability finding gets marked “reachability unknown” rather than silently assumed safe.
The attribution chain, and where it breaks
Every source-based security claim leans on a chain: reviewed commit, build process, generated artifact, registry digest, deployment config, running process. Confidence falls off a cliff wherever one link is missing.
Say a review confirms solid server-side authorization on every project-scoped endpoint. Genuinely meaningful, about the reviewed commit. It is not proof that the same logic is running in production when the deployed commit is unknown, the image tag is mutable, the runtime exposes no build identifier, the image was built on someone’s laptop, or nothing maps the image to an approved change. The correct conclusion isn’t “authorization is broken.” It’s: verified in source, unverified in the effective production deployment. Different claim entirely, and it’s the same distinction a dependency audit draws between “this CVE exists in a transitive package” and “this CVE is reachable from code that actually runs,” except here the gap is in the deployment chain instead of the call graph.
I find it useful to keep three categories of statement strictly separate: source-verified fact (“the reviewed commit checks project scope before loading the resource”), runtime-verified fact (“the deployed system rejected a cross-project request during testing”), and inference (“production is expected to contain this control, but the running artifact can’t be mapped back to the commit”). Collapsing these three into one sentence is where most overclaiming in security reports actually happens.
When drift earns a finding
Not every discrepancy deserves a formal entry. I write drift up as a real finding when it creates a concrete failure somewhere specific: attribution, reproducibility, incident response, recovery, or change control.
Production can’t be mapped to a reviewed commit at all, no SHA, no digest, nothing. Deployment uses mutable references (latest pointing at a different image tomorrow, with zero change to the deployment definition itself, which is a genuinely sneaky failure mode, and functionally identical to the supply-chain risk of a compromised registry account swapping the image behind a familiar tag). Runtime observations contradict source: a header exists in code but not live, validation behaves differently, extra ports show up in recon, migration level doesn’t match the repo. None of this proves which side is correct, only that the chain connecting them is broken. The real process lives in shell history and undocumented one-off commands instead of anywhere reviewable. Or rollback isn’t deterministic because nobody ever recorded the previous known-good digest, in which case “roll back to the previous version” is a nice sentence with no procedure behind it.
Why this is bigger than audit confidence
It’s tempting to file provenance under documentation hygiene. It isn’t. It’s operational.
During an incident the team needs fast answers: what changed, which artifact is actually running, who deployed it, is the same digest live anywhere else, which revision is known clean, which one can replace the suspect one. This is exactly the order-of-operations problem a PICERL-style response runs into during identification: you cannot scope an incident against a system you can’t fingerprint. Without provenance, containment becomes guesswork, and guessing during an incident tends to mean redeploying the same compromised image, destroying evidence, or reverting to something equally unverified.
On the supply-chain side, a compromised registry account is dramatically more dangerous against mutable tags, because an attacker can swap the image behind a familiar tag without touching source at all, and your deployment system will happily go fetch the malicious code on schedule, exactly as designed. This is the same failure mode an SBOM and artifact-signing practice exists to close: know what’s actually running, not what you assume is running.
On recovery, a replacement host can’t be rebuilt with any confidence when production depends on unknown images, untracked host config, undocumented env vars, and operator-specific muscle memory. Backups protect data. They do not preserve the application state required to actually use that data.
And on finding closure specifically: a developer can fix a vulnerability in source, but the finding isn’t closed until the fixed artifact is tied to the running revision. Without that chain, the report can say a fix exists. It cannot say production is fixed. Those are not the same sentence, and conflating them is how “fixed” findings quietly reopen, the same way a vulnerability-management program tracks a CVE as “patched in source” versus “confirmed patched in the running environment” as two different states with two different pieces of evidence.
Assessing drift in practice
Pull evidence from both ends. From source and build: commit SHA, working-tree state, Dockerfiles, lockfiles, CI/CD config, registry destination, tags versus digests, signing and attestation, migration definitions. From production: running image digests, container labels, effective orchestration config, published ports, host listeners, env-var names, migration level, installed host services, relevant firewall rules, deployment timestamps.
The goal was never diffing two directories. It’s determining whether a trustworthy chain exists between what got reviewed and what’s actually running, which is the same posture a log-source health check takes toward a SIEM: don’t assume the feed is complete, verify that it is.
Manual deployment isn’t automatically the villain here
A small team doesn’t need an enterprise CI/CD product to have strong attribution. A disciplined manual process gets you there too: clean checkout, record the full commit SHA, run tests, build, push to the approved registry, capture the immutable digest, sign it, deploy by digest rather than tag, record who and when, verify the running digest, keep the previous known-good digest on hand.
Automation reduces human error, sure, but the actual control here is artifact identity, not the presence of a fashionable pipeline tool. A manually deployed system can be fully attributable. Meanwhile an automated system can still be completely opaque if it rebuilds mutable inputs, deploys tags instead of digests, or never records what actually landed in production. The tooling was never the point.
One invariant worth enforcing everywhere
Every running production artifact maps to an approved repository commit, a recorded build, an immutable digest, and a named deployment event.
That single sentence gives you objective checks: the commit is recorded, the build starts clean, the digest is recorded, the deployment references that exact digest, the runtime exposes the expected build identity, signatures verify, the previous known-good digest is retained, rollback has a defined target. It’s a stable object for both the assessor and the operator to reason about, which is more than most drift discussions usually offer, and it maps almost one-to-one onto the “every finding needs an owner, an evidence reference, and a closure criterion” discipline a compliance gap analysis runs against any control.
Scoring it honestly
Drift is an odd finding because it doesn’t necessarily create a direct exploit path on its own. Severity really comes from what decisions are leaning on the affected evidence, the same combined-signal thinking a vulnerability program applies when it weighs CVSS severity against exploit-probability data and known-exploited status rather than trusting the raw score alone. A low-stakes internal tool with no sensitive data: probably a minor rating, mostly operational annoyance. A multi-tenant platform about to onboard customer data, where a source review is being used to justify launch readiness but source can’t be tied to production: now the uncertainty is contaminating every source-derived conclusion in the report, and that’s a very different severity conversation.
Weigh data sensitivity, the degree of source-runtime contradiction, mutable-artifact usage, reproducibility, availability of a known-good rollback target, and how much weight the assessment is carrying for a launch decision. And be precise in the writeup: the finding isn’t “every source-level control is absent.” It’s “the organization cannot currently prove which controls are present in the deployed system.” Those two sentences read similarly and mean completely different things.
TL;DR
Security review evaluates evidence, not repo aesthetics. A clean codebase and a well-designed authorization model only matter once you can tie the running system back to them, and when that link is missing, the honest move is to document it, assign it an owner, give it closure criteria, and remediate it like any other finding, not bury it as a caveat nobody reads.