An Introduction to Secrets
“And above all, watch with glittering eyes the whole world around you because the greatest secrets are always hidden in [your logs].” — Roald Dahl
What’s a secret?
A secret is anything that authenticates or authorizes you to a system, and so you want to keep it, well, secret. If you have an application, the chances are you have some secrets.
For example, you may have a connection string that you pass to a database so you can authenticate a session and request data from it. Or you may have an API token that you supply when you make a call to your cloud provider so you can read and write from its storage. And you may have a TLS certificate that you present to your clients so you can authorize yourself into an elevated position of trust. Each of these things is a secret.
What isn’t a secret?
You probably have other sensitive data you’d like to keep secret too, like credit card numbers or PII. But to be clear, they’re not a secret if they don’t directly grant you access to a system. In contrast, something like a database password does. It is in and of itself a mechanism of access. If it falls into the wrong person’s hands, it can be used to authenticate or authorize that person to perform actions they shouldn’t. So, at least in the context of your application, that’s a secret. Securely storing and managing secrets like it will help your application protect the not-secret-but-still-sensitive-data in turn.
Likewise, configuration isn’t a secret either. It influences how your application operates and often holds enough sensitive data that it’s separated from code, but not all of it is really private. Let’s say your configuration defines parameters for a database connection like an adapter and a password. We just talked about why we absolutely have to keep the password secret, but it isn’t so critical to keep the adapter secret too. While it connects your application to the database, it doesn’t authenticate or authorize you in any way and your data won’t be compromised if it were to leak. Some of your configuration may be a secret then, but the configuration itself isn’t.
How do you store a secret?
While a secret is extremely sensitive, it must be accessible to you and your application. For convenience’s sake, that’s often treated as a more pressing concern than security — even when 60% of small businesses that have a data breach fold within half a year or an honest developer mistake can be unreasonably costly.
When secrets aren’t overlooked, they’re easily misunderstood and mishandled. (What came first: the chicken or the egg?) If I give you a secret, are you able to answer questions like where it is, how it’s accessed, how it’s updated, how it’s revoked, who can use it, when they use it, and what happens in a compromise? The root of it all is how you store the secret, so for now, let’s break down the common ways you shouldn’t.
Maybe you hardcode it.
Uh-oh. If you hardcode a secret in plaintext, anyone who has access to your application can read it. And if you check it into a version control system like Github, anyone anywhere can read it.
This is regardless of the secret being in your application code or a distinct file. A configuration file that has secrets violates a separation of concerns, and you still may check it in by accident. A secrets file dedicated to holding your secrets is no more foolproof.
Maybe you place it in an environment variable.
Heroku’s 12-factor methodology for building SaaS applications made popular the practice of storing configuration in environment variables. And since secrets are often spoken about in the context of configuration, it may follow to store your secrets in environment variables too.
It’s not illogical to do so. Environment variables are separate from code, so they’re easy to change between deploys without touching code and less likely to be checked into a version control system by accident. They’re also universal to every developer and agnostic to your OS or stack. If you have a Node.js application, you probably store your secrets in environment variables via an .env file. If you deploy your application on a PaaS like Heroku, you probably manually configure your secrets as environment variables in a control panel.
But before environment variables sound so good, let’s look at why you shouldn’t store your secrets in them:
- Anyone who has access to your application can read (and probably edit) your secrets like in any other code or file. And while environment variables are separate from code, it’s still possible to accidentally check them into a version control system for anyone anywhere to see.
- The environment is implicitly available to your entire process, so it’s nearly impossible to track how your environment variables may be accessed or exposed. For example, environment variables are commonly logged or printed in plaintext for debugging and error reporting. If they’re stored in a log file when your application crashes, you may also have them in plaintext on disk.
- Since environment variables are passed to children processes, they are leaky and susceptible to misuse. Your secrets are made available to anything your application calls, including third-party tools! It’s hard to say what they’ll do with this unintended access. This means you break both the principle of least privilege and the principle of least astonishment.
- Environment variables become legacy knowledge. When you bring in a new developer to work on your application, it may be hard for them to grasp the sensitive nature of the environment variables that hold your secrets and to handle them appropriately. The implicit expectation that there’s nothing special in the environment doesn’t align with the requirements you may have for tightly controlled access there.
Maybe you encrypt it.
While encrypting a secret protects it from immediate threat, it isn’t a complete solution. For example, in a Rails application, the convention is to store your secrets in an encrypted secrets file. You can store the encryption key to unlock it in another file or in an environment variable, but that’s a secret too! And a particularly sensitive one: anyone who has access to it (and your application) can read and edit any of your secrets. So, you don’t want it to leak. But if you plan to secure it via encryption first, you’ll just kick the can down the road. This is an important problem we’ll revisit later in the series.
In any case, secrets rotation is critical for your application’s security. However, whether you store your encrypted secrets in a dedicated file or in environment variables, they’re awkward to change at regular intervals and functionally impossible to change between deployments. Worse yet, they’re not trivial to revoke in a compromise, especially since you still lack visibility and control over where they end up, who can access them, and how they’re used.
So then what?
None of this may click for you in a small application, let alone feel worth the trouble to solve. But if you want growth and longevity, it’s imperative to securely handle your secrets from the get-go — unless you don’t want to have any fun later. As your application grows and you have more moving pieces, you’ll thank yourself if you do.
This is the first article in a series on secrets. In the next article, we’ll talk about what poor practices in handling secrets can mean for your application and how you should store your secrets instead. Stay tuned, and thanks for reading!