5 Ways to Prevent Secrets Sprawl

The number of secrets exposed in public repositories is staggering. With reports in 2021 reaching up to 6 million secrets detected, an increase of 50% from 2020. Secret sprawl is part of every organization, but it is a plague on the open-source world of software development.

Even if your organization has a top-notch security-aware culture, human error will inevitably cause secrets to leak and data to be lost or compromised. In 2022, data leaks are estimated to cost organizations an average of $4.35 million. 

Cybercrime is here to stay, but that doesn’t mean you can’t make it much harder for cybercriminals to steal your data and ransom your infrastructure. Learn how to prevent secret sprawl and protect yourself from being the next victim.

What is secret sprawl?

Passwords are the most common secret used by organizations worldwide, but there are many other secrets, especially in software development. Encryption keys, SSH keys, API keys, and Certificates are just a few examples. Where do you keep those secrets?

Software engineers are often more security-aware than employees who place sticky notes with a password on their monitor, but they are not impervious to human error. Passwords get stored in files, hard-coded into source code, and sometimes accidentally pushed into public repositories. Some end up in configuration files that are easily accessible from the internet.

Risks of secret sprawl

Secrets end up everywhere. This secret sprawl is the easiest trap to fall into and can cause significant problems, the most obvious of which is security breaches. When secrets get stored in precarious places, or even securely, but without being trackable, they are easily lost, compromised, and much harder to terminate when they fall into the wrong hands.

But security isn’t the only concern. Finding a secret when needed can be challenging without knowing where each secret is stored. Some secret locations may be only known by a limited number of people. What happens when someone is in sole control of a secret and goes away for a few days? And the issues of secret sprawl only grow over time: The more secrets your organization has, the wider the sprawl, and the harder it will be to recover.

Secret sprawl can affect your organization in several ways:

1. Security: With multiple storage locations, it becomes more difficult to ensure those locations are encrypted and monitor their access. 
2. Access: As the area where secrets spread increases, it becomes more challenging to know where to look.
3. Employer Dependency: When certain employees are accountable for knowing the locations of the secrets stored and left, this can become problematic for the organization.
4. Long-term scalability: Wihtougb a comprehensive secret management system in place as your organization scales, the sprawl will impact your organization’s ability to scale in size and infrastructure.

How to tell whether you have a secret sprawl problem

When an organization starts out, secret sprawl seems like an issue for the ‘future-you’ to worry about. Your developers start by hard-coding secrets when testing frameworks and setting up cloud resources. You’re only building a prototype. You take the quick “done and dusted” approach because you need that prototype ASAP.

Soon enough, you hire a few more teams and still use your prototype code and throwaway resources. You’re now working on live code and already have a secret sprawl problem. No one even knows which resources you have access to and which keys are used to access them, and there is no central record of those secrets. Some secrets may be falsely recorded as assets while they’re no longer in use. The usernames and passwords of a former developer are still valid because nobody knew to terminate them.

You may need to change credentials to comply with a new standard as you picked up a more sensitive client. You don’t know who manages which secrets, who has access to what, and you have no easy way to share secrets with new developers. These are the symptoms of secret sprawl. But the good news is that you can prevent it.

Ways to prevent secrets sprawl

As part of your efforts to shift left security, developers must remain in control of their secrets. But you cannot eliminate human error even with superb security awareness culture, and it doesn’t take much for a cybercriminal to gain access to sensitive data using privilege escalation. Stackoverflow was compromised when the attacker first gained access to private chat rooms. Every secret is crucial as they are all keys to doors that may be breached.

Cloud computing has made development cheaper and faster but brings new security risks. That’s because accessing cloud resources requires secrets, and each secret is a security risk. Moving to the cloud requires a better understanding of cybersecurity, and preventing secret sprawls is one of the main reasons why.

1. Don’t store secrets inside Git repositories

While public repositories are exposed to the public, storing secrets in private repositories is still ill-advised. Having your private repositories breached can be a disaster, but if you add secrets stolen along with the source code, it could be catastrophic.

Since Git stores the entire history of your repositories, any copies made include every revision; this means that even if you place a secret in a repository and then remove it later, there is still a record of it in the repository.

Storing secrets in Git is an unnecessary risk that you can avoid. One such method is employing a secret scanner like Spectral that will passively scan any code pushed to Git and alert you of any secrets that may be hiding inside.

2. Use a trusted Key Management System (KMS)

Key Management Systems ensure a healthy encryption and decryption lifecycle. Keys generally go through several stages, and a KMS will ensure you complete all the steps securely.

Every key needs to be generated with an adequately secure algorithm. Those keys must be securely delivered to where they need to go via secure channels such as TLS or SSH. Once applied to data, the key needs to be stored for decryption purposes; Otherwise, you may lose the encrypted data with the key. Once the key is no longer needed, it must be destroyed and disposed of without compromising the encrypted data. Encryption keys should naturally follow this lifecycle periodically to maintain security. A KMS makes this process easy, which ensures it happens frequently.

3. Rotate login handles frequently

Changing passwords and other credentials is an excellent way to prevent them from being used against you. After all, if you retract authorization, those login handles are useless. Security policies that ensure constant rotation of secrets reduce the risk of privilege escalation, as anyone who gains access with compromised credentials has a limited time to use them to escalate access. How often you rotate login handles depends on how much risk you’re willing to take and how much pain it is to rotate passwords.

4. Consider restricting API access to a minimal scope

It is tempting to grant developers access to everything. They are often security aware, and free reign allows them to get creative with resources. But developers aren’t impervious to human error. Restricting API access to the minimal necessary scope of daily operations is an easy way to reduce the risk of complete exposure in case of secrets leaking.

5. Automate secrets detection

Git repositories aren’t the only place where secrets may leak. Configuration files often hold secrets and may find themselves in public spaces. Chat rooms such as Slack may be a channel where secrets are passed insecurely. Given you cannot prevent human error, it’d be best to automate secret detection.

Automatic secret detection allows you to monitor against secret sprawl. With a good tool, human participation should be minimal and not delay any part of your SDLC. Seamless integration into a CI/CD pipeline is crucial for ease of use, effective secret detection, and leak prevention.

Stop the secret sprawl before it starts

While secret sprawl may be the easiest trap to fall into, it is also reasonably easy to avoid. Using a key management tool and built-in secret management systems such as Kubernetes Secrets or Azure Vault is a significant first step. However, tools such as these do not prevent human error and don’t stop keys from ending up in Git repositories or other public spaces.

Secret scanning is a critical component in your security stack and the only way to stop human error from causing secrets to leak. Spectral will alert you of any vulnerabilities and potential secrets leaking in any part of your SDLC and your cloud infrastructure. Monitor, catalog, and protect your assets from exposed secrets. 

Spectral offers an automated and AI-driven solution for secret scanning that is part of your CI/CD pipeline and doesn’t burden your SDLC’s resources. There are no downsides to the high accuracy of secret recognition and a low number of false positives. See it for yourself by scheduling a demo.