Rails is a web framework, but it does much more than only handling user facing web requests. There is also a background side of the framework: background jobs. Don’t believe me? Stay with me!
In this post you will learn:
- What are Background jobs.
- When should you use Background jobs in a Rails project
- Best practices for writing good Background jobs.
Let’s get started!
What are background jobs in Ruby on Rails?
As mentioned in the introduction, not everything in Rails is handling web requests. That is the user facing synchronous side of the framework, where there is a user waiting for the system to come back with a result. There is also an asynchronous background side. As its name suggests, background jobs are pieces of code that run asynchronously as background processes.
When should you use Background jobs in a Rails project?
In my experience there are three main groups of scenarios in which you should consider using background jobs;
- Scheduled tasks.
- Time consuming tasks within request flows.
- Tasks that require third party systems.
Let’s see each of them in detail.
Scheduled tasks
Within this group you find tasks that are executed periodically, like cron jobs would do. Examples of these are: clean-up tasks, mailing, report generation, billing processing, tasks that deal with big amounts of data or updating search services, among others.
Two of the most common implementations that scheduled tasks in Rails are the whenever and the sidekiq.-scheduler. gems. For now I will just mention them, but if you want we can discuss them in detail in an upcoming post.
Time consuming tasks within request flows
If your application has a use case that includes the execution of a task that takes a long time to complete, you probably do not want to have the user waiting in front of the computer until that long task finishes to serve a response. In this scenario it is better to return a response immediately and let the background job backend take care of time consuming tasks.
A good example of this is the creation of reports. The user would request the application to create a report and the application will immediately reply that the report is being created and that it will notify the user once it is ready.
Tasks that require third party systems.
Since you cannot control how long it will take a third party system to return, or even knowing if it will be available at all, whenever your application has to send a request to a third party system not controlled by you, you should consider handling it asynchronously in a background job.
An example of this could be an API that communicates with IoT devices. Let’s say the user wants to change the status of an IoT device. The IoT device can be unavailable or the communication network could be slow. In this scenario a good practice would be to let the API tell the users that their request has been accepted, while performing the request to the third party system under the hood. The user should be able to poll the status of the IoT device later.
Best practices when writing Background jobs
We have already discussed some good practices within the previous use case examples, but I also want to mention two more when it comes to coding background jobs: Idempotency and queue priorities.
Idempotency
There will be situations in which your background jobs will fail and you might want to retry them. In fact, many of the available background job backends offer built-in retry mechanisms when exceptions occur.
Since some background job backends do not ensure job execution, another possible scenario is when you do not know for sure whether a job has been executed, or whether only part of it has been executed, and you need to retry it.
When retrying a background job, it is a good practice to write your code in a way that there are no side-effects in case it runs several times.
Queue priorities
Another good practice you should consider is defining carefully the queues in which you will schedule your application’s background jobs. You should also pay the same level of attention when assigning the resources that each queue will have available
For example, you could define two queues: low_priority and high_priority, You could then determine that the high_priority queue will have enough resources so that jobs within that queue always have preference over the jobs scheduled in the low_priority one. Once that is set, you will then make sure you schedule the jobs that are critical to your application within high_priority, whereas non critical ones, namely those that can wait to be executed, would be scheduled within the low_priority queue.
What comes next?
In my next post I could share with you a quick overview of the most famous background job backends that you can use in your project (Yes there are a few), or I could talk about the scheduled tasks available implementation. Let me know your preference in the comment section down below.
If you enjoyed this post, do not forget to subscribe so that you do not miss any future updates. If you want me to write about something in particular, please let me know in the comment section below.
See you in my next post! Adiós!