Blog

Secret rotation after a supply-chain incident needs an order

Rotating every credential at once creates confusion; rotate from a clean control plane according to privilege, persistence, and evidence.

Armor12 min read

“Rotate all secrets” describes an objective. It leaves out the most consequential question: where will the replacement secrets be issued and used?

If the workstation or runner is still compromised, an attacker can collect each replacement as it arrives.

The Shai-Hulud, elementary-data, and postmark-mcp incidents involve different kinds of exposure. Use the actual accessible data and identities to define scope, then order the response around credential dependencies.

1. Stop reuse of the compromised environment

Quarantine affected hosts and pause jobs that can reintroduce the malicious artifact. Block known malicious destinations where appropriate. Preserve the artifact, configuration, relevant logs, and volatile evidence that your response process can capture.

Establish a known-clean administrative path with independent authentication. Do not perform recovery through a suspected browser session, terminal environment, or CI runner.

2. Secure identities that can issue other access

Prioritize identities that can create credentials, reset users, alter trust policies, or change where secrets are stored. These may include identity-provider administrators, cloud account administrators, source-control owners, secrets managers, and CI administrators.

Containment is not a rigid queue. Revoke an actively abused credential immediately while the broader recovery proceeds. The ordering principle is to avoid replacing a leaf secret while an attacker still controls the authority that can recreate it.

3. Work outward through dependent credentials

Credential group Related state to review
Package publishing and signing New releases, maintainers, signing material
Deployment and workload identity Role trust, federation, issued sessions
Database and SaaS access Service accounts, exports, recipient changes
Model-provider and MCP access Keys, connected tools, unexpected use

Track refresh tokens and active sessions separately from passwords. Changing a password may leave existing access valid.

4. Prove revocation

For each affected identity, record an owner, scope, revocation action, replacement method, and validation result. Confirm the old credential fails where a safe test is available.

Review audit logs before and after the response. Search for added users, keys, workflow files, package ownership changes, and new federation relationships. These may preserve access without reusing the original secret.

5. Reconnect rebuilt workloads

Restore from trusted artifacts and approved configuration. Ensure caches or persistent runners cannot reinstall the compromised version. Attach replacement credentials only after the workload is ready.

The response is complete when the new credentials operate through a restored chain of trust, with evidence that the exposed access and persistence have been addressed.

incident-responsecredentialssupply-chain

See what your own fleet is running

Armor1 discovers and risk-scores every MCP server, skill and tool your developers have connected. Free for personal use, no credit card.

Start Free