How to keep your data confidential with Active Record Encryption

One of the most relevant new features shipped with Rails 7 is Active Record Encryption.

Active Record Encryption helps to preserve your application’s data confidentiality by adding reversible application-level encryption in a way that you can encrypt, decrypt and do searches on that data.

Advantages of Active Record Encryption

Active Record Encryption makes encryption transparent by encrypting attributes before persisting them, and decrypting them upon retrieval. With Active Record Encryption your application will deal with data as if it was not encrypted, while storing it encrypted in database.

It also automatically filters out encrypted fields in the application logs (unless you configure it not to do so). This makes it a great fit when dealing with PII, and helps your application complying with CCPA and GDPR regulations.

Deterministic vs non-deterministic encryption

Active Record Encryption supports both of them. Before you setup and use Active Record Encryption, it is important that you understand their differences.

Non-deterministic mode

When using non-deterministic encryption, encrypting the same content with the same key twice will result in different cipher-texts.

The advantage of this approach is that it improves security by making crypto-analysis of cipher-texts harder for the attacker.

It also makes impossible to query the database, which is an advantage from the security point of view… but could be a disadvantage from the usability side in case you need to query that data.

For security reasons the non-deterministic approach is Active Record Encryption’s default approach. It is always recommended unless you need to query the data.

Deterministic mode

When using deterministic encryption, encrypting the same content with the same key twice will result in the same cipher-text. As mentioned above, the Rails guides recommend you to use this approach only when you need to query the data.

Encryption algorithm

Both deterministic and non-deterministic approaches use the same AES-GCM encryption algorithm. The difference is that the non-deterministic mode uses a random initialization vector, whereas the deterministic mode initialization vector is generated as an HMAC-SHA-256 digest of the key and contents to encrypt.

Setup Active Record Encryption

Enough talk, let’s get our “hands dirty” and see how you can use Active Record Encryption in your Rails application.

The first thing you need to do is to setup the encryption keys. To do so, first run the following rails command in a terminal to generate the encryption keys:

Now copy those keys and add them to your Rails credentials file (either to the global config/credentials.yml file or to a specific environment’s credentials file).

Let’s explain each key in detail:

primary_key

The primary key is used for non-deterministic encryption.

To support key-rotation, primary_key can also be a list of keys. In that case, Active Record uses the last key to encrypt new values, while keeps trying all the keys from the list when decrypting the content until it finds the one that works.

Since trying all the keys to decrypt previous values hinders performance, you can configure active_record.encryption.store_key_references so that Active Record Encryption stores a reference to the encryption key within the encrypted payload.

If you want to use this feature, remember provisioning some extra storage space on each field for the key reference.

Talking about configuration. The ideal place to put Active Record Encryption configuration options would be config/application.rb since you typically want to have the same encryption configuration across all environments, If instead you want to set specific configurations on a per-environment basis you can do so within each specific environment config file under config/environments/.

deterministic_key

As you can guess, this key is used for deterministic encryption. You can omit it if you do not plan to use deterministic encryption at all.

At the time of this writing Active Record Encryption does not support deterministic key-rotation, so no lists here.

key_derivation_salt

Active Record encryption uses this value to derive generated keys.

Declare encrypted attributes

Ok, now that you have a basic configuration, let’s see Active Record Encryption in action.

As an example, I will create a SecretAgent model. This SecretAgent will have two string fields: code_name, that can be known by everyone, and real_name, that must be kept secret so that the agent’s real identity does not get compromised. It will also have a description rich text field, which will contain information about the agent. Again, we want to keep this information secret so that the agent’s details, strengths and weaknesses do not fall into the wrong hands. To protect real_name and description we will use Active Record Encryption.

To create the SecretAgent I will execute the following model generator command:

And modify the migration file adding a limit to real_name, like this:

It is worth mentioning that since the encryption payload takes some space. The Rails guides recommends you to increase the number of bytes of string columns from 255 bytes to 510 bytes. Although the above migration file sets the limit in characters and not in bytes for string columns (this is how add_column works), since in this example we will store short names it might even not be necessary to increase the real_name field’s length at all. However, what I want to point out is that you need to take this encryption payload’s overhead into account when planning your database schema for encrypted data when using string columns.

The description field does not appear in this migration file since it will be handled by Action Text. Nonetheless, Active Record Encryption’s documentation states that the additional overhead added for text fields by Active Record Encryption is negligible, so following its advice I let it as it is.

As a side note, Active Record Encryption serializes values using the underlying type before encrypting them. The only prerequisite is that they must be serializable as strings.

After running the migration with bin/rails db:migrate let’s create the following model in app/models/secret_agent.rb

As you can see, using Active Record encryption is pretty straight forward. For string fields you just need to add encrypts to your model, followed by the name of the field, and for rich text fields adding the encrypted: true option will do the job.

Let’s now create a secret agent and see encryption in action.

As you can see the encryption process is completely transparent to you. If you check the transaction logs, Active Record persisted real_name and description fields encrypted to the database, But if you later retrieve one of these values, they appear in their original form, as if no encryption ever happened!

Let’s now look at the persisted value for real_name from the transaction log in more detail. Notice that the persisted value is a hash serialized as a string that contains some interesting keys…

In this hash, p represents the encrypted value of real_name, while h is another hash containing some headers related to the encryption process. The first one iv, stands for initialization vector, while at represents the auth_tag that will be used during the decryption process to check that the encrypted string has not been modified, There could be other different headers within this hash depending on the configuration options you have set for Active Record Encryption. If you are curious about what they mean you can check them here.

How to query data

So far we have seen non-deterministic encryption, but what if you wanted to query some encrypted field? You would like the encryption process to always generate the same payload so that you can query it. For this case you must use deterministic encryption.

Using deterministic encryption in your model is as easy as adding the deterministic: true option to your fields encrypt’s declaration in the model. Let’s see it in action.

Imagine you had a User model with unique email addresses and you wanted to have the user emails encrypted, but yet be able to query by email. First of all the migration would look like this:

And then in the model you would indicate that email is an encrypted field, but this time you would also have to add the deterministic: true option so that you can later query the data.

Let’s now add some data:

Now you should be able to query by email as you usually do. Note that the encryption process remains transparent as before:

As you can see, Active Record now encrypts the search value first and then returns the record that matches it.

Migrating from unencrypted to encrypted data

Nobody likes dealing with legacy data.

However, there will be times in which you will decide to start encrypting a field that already had some unencrypted records. While your goal should be to have everything encrypted in the long run, meanwhile you can set config.active_record.encryption.support_unencrypted_data to true so that Active Record Encryption starts encrypting new values but does not raise with legacy ones. Once you have finished encrypting legacy records, you should remove this configuration option.

Differences between Active Record Encryption and has_secure_password

You might have already used Rails has_secure_password in your application to persist encrypted passwords. Let’s first clearly say that this is a completely different feature that solves a completely different problem.

The main difference with Active Record Encryption is that has_secure_password is focused only on securing passwords and not other types of data. Given this fact, has_secure_password‘s encryption is not reversible. Once you have encrypted a password with the BCrypt algorithm, you can not get its original plain text version again. This is great since you do not want anyone to see them in their original form for security reasons.

If you want to learn more about has_secure_password please take a look to my previous post.

Advanced settings

There are many other options you could add to Active Record Encryption. Since this post would be too long, please check Rails guides to learn about of them. If you have any question or you want me to talk about any of them in detail please leave me a comment below.

And that is basically it!

If you enjoyed this post, do not forget to subscribe so that you do not miss any future updates. If you want me to write about something in particular, please let me know in the comment section below.

See you in my next post! Adiós!