Encrypting data in transit using HTTPS is now nearly universal, but have you considered the importance of encrypting data at rest? File system permissions only protect your data from prying eyes so long as your servers’ disks are properly mounted within your system. Furthermore, anyone who manages to gain root access can read any file or folder on the system, regardless of its permissions.
Encrypting your disks prevents an attacker from trivially rebooting your server into a live environment (where they have root) and having unfettered access to your data. It also prevents an adversary from physically removing the disks, then extracting the data (as root) using a separate machine.
For these reasons, full disk encryption is becoming a more common requirement of many information systems, particularly ones that handle secrets such as passwords or personal identifiable information. A properly encrypted disk, using a robust specification such as Linux Unified Key Setup (LUKS), is only decrypted block-by-block into memory while the data is accessed.
The main potential weakness, and an important consideration of any full disk encryption scheme is where to safeguard the encryption key. Having a key management strategy that is aligned to the sensitivity of the protected data is crucial — in order to make the right trade-offs between security and convenience.
DM-Crypt + LUKS with a Keyfile
- Pro: Supports unattended reboots
- Con: Less secure, if machine is physically compromised
The least secure, but most convenient option is to store the encryption key in a keyfile on an unencrypted volume on the server. On boot, the LUKS encrypted volume can be opened (decrypted) automatically by referring to the keyfile path in /etc/crypttab and mounted using /etc/fstab.
For additional security, the keyfile can also be protected using a passphrase as a form of two factor authentication, but it would prevent the server from coming back up in an unattended reboot.
DM-Crypt + LUKS with Manual Passphrase Entry
- Pro: Key is kept separate from the system
- Con: Does not support unattended reboots
A more secure option is to store the encryption key outside of the machine where the encrypted data resides. The downside is, a trusted, human operator has to provide the LUKS passphrase from the console when the server is booted, in order to open and mount the encrypted volume.
If the boot volume is to be encrypted using this option, then the passphrase needs to entered from the serial console (from the cloud provider dashboard or vSphere Web Client). If a serial console is not available, there are also implementations where a lightweight SSH server like Dropbear is slipstreamed into the initramfs boot process — so that the passphrase can be entered remotely.
If only non-boot volumes are encrypted, then the operator can login with SSH and use cryptsetup and mount commands from the terminal.
The server cannot boot up, or the data on the encrypted volumes cannot be accessed until the operator intervenes, which can result in downtime in the event of an unplanned reboot (e.g. due to provider maintenance or a power outage).
DM-Crypt + LUKS with Consul, Vault
- Pro: Secure, and supports unattended reboots
- Con: Initial set up and maintenance of the key management server
Using an isolated, dedicated key management server such as HashiCorp Consul and Vault to store your encryption keys (i.e. secrets) is a more sophisticated approach where you can have unattended reboots and not have to use a keyfile.
With this set up, the encryption key is not stored on the same server where the encrypted data resides, providing an immeasurable step-up in security compared to using a keyfile.
When the server with the encrypted volume reboots, it makes a TLS-encrypted request to Vault for the LUKS encryption key needed to open and mount the volume. This request can, of course, be logged for auditing and forensics purposes.
The main downside of this approach is the relative complexity of setting up Consul and Vault to begin with, and the ongoing burden of maintaining and securing the key management server. We can however, assist with these aspects, as it’s a great setup particularly for highly security sensitive data, or self-hosted, on-premise (non-public cloud) environments.
Block Volume Encryption with Key Management Service
- Pro: Simple to deploy and secure, supports unattended reboots
- Con: Requires using a public cloud service
Major cloud providers such as Amazon Web Services or Oracle Cloud Infrastructure provide a fully managed, key management service (KMS) to their customers — AWS KMS or OCI Vault. Using such services, you can generate an AES-256 key, store it within a hardware security module (HSM), and encrypt your block volumes (including boot volumes) using envelope encryption with those keys.
Since these are fully managed services, the cloud provider takes the responsibility of maintaining the high availability of the key management service, so that your virtual servers can call upon the KMS at any time to decrypt your data. We believe that outsourcing key management to a specialized service is a highly compelling option, providing an excellent balance between security and convenience — for the majority of threat models.
The underlying technical architecture is very similar to what you would host yourself using Consul and Vault in an on-premise environment, but you can securely encrypt entire block volumes with a few clicks. The KMS also provides a point-and-click interface where you can easily rotate the keys, which is always a good practice to do periodically.
Notably, the cloud provider’s employees do not have access to your key material, so using this solution gives you an additional layer of privacy — compared to using provider-managed keys. Getting started with a provider’s key management service comes at very minimal additional cost if you already use the public cloud. For AWS, each Customer Managed Key is billed at $1/month and the first 20,000 requests/month covered by the Free Tier. With Vault, the first 20 Key Versions are free for HSM protected Keys — unless you require a Virtual Private Vault.
If you are confused by the options you have to encrypt your servers’ data at rest, please contact our hybrid cloud & infrastructure architects for your particular use cases. We can design a custom solution, such as a HashiCorp Consul and Vault deployment, for your on-premise environment, or advise you on using a managed key management service in the cloud.
