Most engineering organizations now have a secrets manager. It was provisioned during the cloud migration, or required by a compliance framework, or set up after the incident where a developer accidentally committed an API key to a public repository. The secrets manager exists. The checkbox is checked.
The secrets manager is also being used incorrectly in ways that undermine most of the security value it was purchased or provisioned to provide.
This is not a judgment about intent. Secrets managers are a category of tooling where "using it" and "using it correctly" look identical from the outside both result in credentials not appearing in source code, which is the primary visible indicator of secrets hygiene. The difference is in the failure modes: a correctly used secrets manager fails safely when credentials are rotated, when access is revoked, or when an application instance is compromised. An incorrectly used secrets manager fails in ways that are invisible until they are not.
The gaps are consistent across organizations, across cloud providers, and across technology stacks. They appear in AWS Secrets Manager deployments and in HashiCorp Vault installations and in GCP Secret Manager configurations. They appear regardless of team size and regardless of how long the secrets manager has been in use. They persist because the documentation for secrets managers describes how to store a secret which is easy but rarely describes the complete set of practices that make storing a secret meaningful.
This post is about those practices.
The Inventory Problem: What You Do Not Know Is Stored Somewhere Else
The first and most common gap in secrets manager usage is incomplete inventory. The organization has a secrets manager, and it contains some secrets. It does not contain all secrets, because nobody has ever produced a comprehensive list of all secrets the organization uses and verified that each one is managed by the secrets manager and not by something else.
The something elses are numerous. Environment variables set directly on deployment targets EC2 instances, ECS task definitions, Lambda function configurations that were set during initial provisioning and never moved into the secrets manager because they were working and nobody had a reason to change them. Secrets in CI/CD pipeline configuration: GitHub Actions secrets, CircleCI environment variables, Jenkins credentials stores, each operating as an independent secrets management system with its own access controls and rotation practices. Secrets in application configuration files checked into private repositories where the assumption was that private means safe. Secrets in third-party integration dashboards where they were copy-pasted from the secrets manager during initial setup and then left as a backup in case the integration with the secrets manager stopped working.
The inventory problem compounds over time. Every new integration, every new service, every new developer who joins and needs to get something working quickly creates opportunities for secrets to land somewhere outside the managed store. The secrets manager accumulates the secrets that were intentionally put there. Everything else accumulates everywhere else.
Running a secrets inventory means not just asking "what is in the secrets manager?" but actively discovering where credentials live across the full infrastructure. That means scanning application configuration files, environment variable configurations on every compute resource, CI/CD pipeline configurations, third-party integration settings, and developer local environments. The result is usually surprising, even in organizations that believe their secrets hygiene is in good order.
Tooling exists to assist with this: GitLeaks and TruffleHog scan repositories for credential patterns. Cloud provider APIs can enumerate environment variables set on compute resources. CI/CD platforms expose their secret stores through APIs that can be queried for inventory purposes. None of these tools catch everything, and all of them require someone to act on what they find. But the starting point is knowing what exists, and most organizations are operating without that knowledge.
The Access Control Problem: The Secret Everyone Can Read
A secrets manager that stores all secrets but grants every service and every developer read access to every secret is providing one security property secrets are not in source code while providing almost none of the others that secrets management is supposed to deliver.
The correct access model for secrets management is least privilege applied at the secret level: each service, application, and individual should be able to read only the secrets it specifically needs to operate. The database credential used by the payments service should not be readable by the analytics pipeline. The third-party API key used by the email delivery service should not be readable by the mobile application backend. The administrative credential used by the infrastructure automation should not be readable by the application runtime.
In practice, most secrets manager deployments use a small number of IAM policies often just one or two that grant broad read access to large categories of secrets. The policies were written when the secrets manager was first set up, when the inventory was small and the policy complexity seemed unnecessary. They were never revisited as the inventory grew and the access model became progressively less appropriate.
Implementing least privilege in an existing secrets manager deployment requires knowing which secrets each service actually needs which requires the inventory work described in the previous section and then writing IAM policies, Vault policies, or their equivalents that reflect those actual requirements. It is detailed work. It is not technically complex work, but it requires time and attention that most teams have not allocated to it.
The consequence of not doing it: when an application instance is compromised which is a when, not an if, for applications running at production scale over long time horizons the attacker gains access to whatever secrets that instance can read. If the compromised instance can read every secret in the store, the breach radius is the entire secrets inventory. If the compromised instance can read only the three secrets it actually needs, the breach radius is three secrets, and credential rotation for those three secrets contains the damage.
Least privilege in secrets access is not about being bureaucratic. It is about the difference between a manageable incident and a full credential rotation across the entire organization under time pressure.
The Rotation Problem: Secrets That Have Not Changed Since 2021
Secrets rotation is the practice of periodically replacing credentials with new ones and invalidating the old values. It is universally acknowledged as a best practice. It is almost universally not done on any consistent schedule in practice.
The reasons are structural. Rotating a secret requires coordinating the update across every place the secret is used every application instance, every configuration that references it, every dependent service. In a deployment that pulls secrets from the secrets manager at startup, rotation requires a restart of every affected service. In a deployment where secrets are baked into container images, rotation requires a new image build and a new deployment. In a deployment where secrets are distributed via environment variables set manually on instances, rotation requires finding every instance and updating it manually.
Most secrets managers provide automation for rotation. AWS Secrets Manager has built-in rotation for RDS credentials, Redshift credentials, and custom rotation Lambda functions. HashiCorp Vault has dynamic secrets credentials that are generated on demand with a configurable lease duration, invalidated automatically when the lease expires. GCP Secret Manager supports versioning that integrates with rotation workflows. The automation exists. The adoption rate is low because setting it up requires more configuration work than initial secret storage, and the urgency is low because the existing credentials are working.
There is a subtler problem with non-rotated secrets beyond the obvious one. Every secret that has not been rotated carries the full history of who has had access to it. A developer who left the organization eighteen months ago, who had access to the database credential, retains effective access to the database if they retained the credential value and the credential has not been changed. The offboarding process removed their IAM access to the secrets manager. It did not change the credential itself. The secret manager access was revoked. The secret was not.
Rotation policy for secrets should distinguish between different credential types and set rotation schedules based on sensitivity and breach impact. Long-lived service account keys for cloud provider APIs should rotate frequently monthly or quarterly. Database credentials for production data stores should rotate at least as frequently as the sensitivity of the data warrants. Third-party API keys should rotate on any event that changes who has had access: a developer offboarding, a credential leak incident, a suspected compromise.
Dynamic secrets, where the secrets manager generates a time-limited credential on demand rather than storing a static value, eliminate the rotation problem for supported credential types by making rotation continuous. Every time the application fetches the credential, it gets a new one with a limited validity window. The old one expires automatically. This is the correct architecture for credential access where the underlying service supports it, and it deserves more adoption than it currently receives.
The Injection Problem: Secrets That Become Environment Variables Anyway
Here is an implementation pattern that appears in a surprising number of deployments that have adopted secrets managers: the application starts, fetches all its secrets from the secrets manager, and immediately sets them as environment variables in the process environment. The secrets manager is used correctly at the retrieval step. The benefit is immediately undone at the injection step.
Environment variables are not a secure medium for secrets. They are accessible to any code running in the process, including third-party libraries and dependencies. They are frequently captured in diagnostic outputs crash reports, debug logs, error tracking tools because the tooling that captures those outputs captures the full process environment. They are visible through the /proc filesystem on Linux systems to other processes with appropriate permissions. They persist for the lifetime of the process and are accessible to any child processes spawned during that lifetime.
The pattern emerges because it is convenient. Application code that reads secrets from environment variables does not need to be modified to support the secrets manager the secrets manager client is added to the startup code, the secrets become environment variables, and the existing application code works unchanged. The migration path requires zero changes to application logic.
The correct pattern is direct injection at the point of use: the application code retrieves a specific secret from the secrets manager at the moment it is needed, uses it for the operation, and does not store it in a location accessible to other parts of the system. This requires the application code to reference the secrets manager client directly rather than reading from environment variables, which means the migration from environment variables to secrets manager is not zero-effort it requires changes to the application code at each point of secret use.
That effort is the cost of actually using the secrets manager correctly. Skipping it in favor of the environment variable bridge preserves some benefi secrets are not in source code while undermining others secrets are still accessible to anything running in the process context.
The Audit Logging Problem: The Access You Cannot Account For
Every major secrets manager provides audit logging: a record of every read, write, create, and delete operation performed against the secrets store. This logging is the mechanism that answers "who accessed this credential and when?" after an incident. It is also the mechanism that can answer "is this credential being accessed in ways that are inconsistent with normal application behavior?" as a proactive detection signal.
Most deployments have audit logging enabled it is on by default in many secrets managers and do not use it for either purpose.
The audit logs accumulate in whatever log aggregation system the organization uses. Nobody has set up alerting on access patterns that would indicate a problem: a credential being accessed from an IP address outside the expected application infrastructure, a credential being accessed at a time of day when no deployments are running, a credential being accessed far more frequently than the application's request volume would require, or a credential that was just rotated being accessed using the old version (which should be impossible if rotation was implemented correctly, but which reveals configuration errors when it happens).
Implementing meaningful audit log monitoring for secrets access does not require sophisticated tooling. The patterns worth alerting on are simple: access from unexpected sources, access at unexpected times, access rates inconsistent with application behavior, and access to secrets by principals that should not have that access. Most log aggregation platforms support the alerting rules needed for these patterns with basic configuration.
The audit log is only useful as a detection mechanism if someone defines what they are looking for and sets up the query to find it. The log existing is the prerequisite. The monitoring is the work that has not been done.
Running the Secrets Audit
An audit of secrets management posture across an organization has five components, each addressing one of the gaps described in this post.
Inventory audit: Enumerate every credential type the organization uses database passwords, API keys, certificates, service account keys, OAuth client secrets and for each one, determine where it is stored. Is it in the secrets manager? Is it set as an environment variable on a compute resource? Is it in a CI/CD platform's secret configuration? Is it in a configuration file in a repository? The goal is a complete map, not a partial one.
Access control audit: For each secret in the secrets manager, enumerate which IAM roles, service accounts, or users have read access. Compare that list against which services actually need that secret to function. Flag every principal with access that is not required for normal operation.
Rotation audit: For each secret in the secrets manager, determine when it was last rotated. Flag any secret that has not been rotated within the rotation window appropriate for its type. For credentials with no rotation history, establish when they were first created and note that every person who has had access since creation retains effective access to the credential value.
Injection pattern audit: Review the application startup code for each service. Identify whether secrets are retrieved and immediately set as environment variables, or whether they are retrieved at the point of use. Flag all services using the environment variable bridge pattern.
Audit log audit: Verify that audit logging is enabled for the secrets manager. Check whether any alerting is configured against those logs. If no alerting exists, note the absence of detection capability for abnormal access patterns.
Closing: The Manager That Manages Nothing
A secrets manager that holds some secrets, grants everyone access to all of them, has not rotated a credential in three years, injects its secrets into environment variables on startup, and generates audit logs that nobody reads is performing one function: keeping credentials out of source code.
That function is valuable. It is not sufficient.
The gap between "we have a secrets manager" and "our secrets are managed" is the gap between having the tool and using it in ways that deliver the properties the tool exists to provide. Those properties inventory completeness, least-privilege access, rotation that actually limits breach impact, safe injection that does not undo the benefits of retrieval, audit logging that enables detection are what make a secrets manager meaningful as a security control rather than as a compliance checkbox.
Every organization that has a secrets manager already has the infrastructure needed to address all five gaps. The secrets manager itself, the IAM system it integrates with, the CI/CD platform it could be wired into, the log aggregation system where its audit logs land all of this exists. The work remaining is configuration, policy, and the audit that reveals the starting point.
That audit is not complicated. It is, however, specific it requires looking at the actual configuration of the actual deployment, not at the documentation of what the secrets manager is capable of in theory. What the secrets manager can do in theory and what it is doing for your organization right now are different statements. The gap between them is worth knowing.
Axeploit surfaces secrets hygiene failures through the application layer finding hardcoded credentials in API responses, identifying endpoints that leak configuration data in error messages, and testing whether credentials that should have been rotated or restricted are accessible in ways the deployment's access model was not supposed to permit. It validates what the secrets manager configuration assumes about the application's behavior against what the running application actually does.
