How to renovate? Why and how you should use automated dependency updates in your software projects

von Dr. Claas Wilke | 25. September 2024 | Allgemein, English, Security, Software Engineering, Tools & Frameworks

Dr. Claas Wilke

Senior Developer

Nowadays, every software development team should consider the automation of dependency updates. As already shown in literature [1-3] and previous posts, automation can reduce both maintenance efforts and security risks. However, some development teams struggle with dependency automation, as they receive too many pull requests they must review manually. Ultimately, this often leads to inefficient or even unused automation processes [1].

To further investigate the struggles teams were having with such tools, we explored which dependency update automation tools existed and were used by development teams. We conducted a small survey and interviewed ten development teams at Senacor. Based on these findings as well as our own experience, we derived some lessons learned on which dependency update automation tools should be used for an optimal update process.

Why to update software dependencies automatically?

Today, every software relies on third-party dependencies. Popular libraries and frameworks are heavily used, leading to many direct and transitive dependencies. Unsurprisingly, if you are maintaining a software with a few dozen dependencies, almost every day one of these libraries will release a new version, adding new features and bug fixes or fixing vulnerabilities and security issues.

Consequently, if you do not update your dependencies on a regular basis, you will soon end up with an unmaintainable software, depending on outdated and no longer supported libraries, possibly including highly vulnerable security issues.

Unfortunately, updating dependencies regularly is a both time-consuming and boring task, which is why many teams postpone updates until they are forced to do them. This strategy comes with several disadvantages:

Updating many software dependencies in parallel can be a lot of pain (also known as dependency hell), as every update might introduce some breaking changes leading to adaptations of your own software.
Changing several dependencies at the same time will increase risks for undetected issues and if a problem occurs, you might not know, which of the updates introduced the problem.
If a software vulnerability is detected in a library used by your software and you are forced to update this library as soon as possible, other dependencies might rely on the same affected version and thus, must be updated as well, increasing your effort for a time critical security patch.

The good news: most of these software dependencies can be updated regularly and automatically with manageable risks, as well as in a complete automated manner.

How software dependency bots work

In the following, we will shortly explain how dependency update automation tools (dependency bots) work. If you are familiar with these tools, you can skip this section.

Setup
First, you must configure a dependency bot by telling it which of your software repositories and projects should be scanned for dependency updates. Depending on the bot, this can be done by altering a simple switch (as for Dependabot) or by the setup of an external toolchain (e.g., when using Renovate Bot in a non-GitHub setup, for example by configuring a cron job running on a Jenkins server).
Scan for Updates
The configured bot will scan your repositories and projects periodically (e.g., every night or week), will compute the dependencies from dependency declarations (e.g., NPM’s package.json or Maven’s pom.xml files) and will consult configured software repositories (e.g., Maven Central) for existing newer versions of the respective libraries.
Pull Requests
If a newer version for a library is found, the bot will create a pull request for the update of your dependency declarations, typically including helpful information such as the release log or changelog of the updated library, allowing you getting an overview on the changes without looking them up on your own.
Review and Merge
Pull requests created by dependency bots are typically reviewed and merged by one of your developers manually. However, some of the bots allow for so-called “auto merge” features that will build and integrate updates automatically. If everything works fine, they will even merge (and possibly) deploy the updates automatically – which reduces the manual effort but increases the risk for undetected braking changes, especially if your test automation does not cover the important parts of your application.
Update other Pull Requests
If pull requests are not reviewed on a short-time basis, the dependency bot might find even newer versions of the same dependencies in one of its next iterations. Most bots will then update existing pull requests, typically rebasing and rebuilding existing branches. Besides, some bots close pull requests automatically if they became obsolete – for example, if the dependency has been updated or removed from the software project manually in parallel.

Which dependency bot should I use?

When looking for a dependency bot for your software development project, the following requirements should be on your mind:

The tool should be easily configurable
It should support all programming languages used in your project, as well as all the package managers being used.
It should support the CI/CD platform of your choice
Ideally, it should be free of charge.

Looking at the market, only two bots fulfil these requirements. First, there is Renovate Bot which supports a variety of programming languages and package managers. It is highly applicable to different CI/CD platforms, as it is published as a docker image that can be integrated almost everywhere. Second, there is Dependabot, also supporting a variety of programming languages and package managers. However, Dependabot is part of the GitHub environment, and thus, only applicable if you are using GitHub as source code repository and integration platform.

Some other bots exist (e.g., Depfu, Snyk, and Frogbot), however, all these bots are commercial and some of them do not focus on software dependency updates but on security and vulnerability issues, providing dependency updates with respect to security as one of their many features.

To sum up, if you are planning to use a dependency bot, you should either use Renovate Bot or Dependabot, as both are free of charge and highly configurable. If you are using GitHub as the source code platform of your choice, Dependabot might be easier to set up, otherwise you should use Renovate Bot, as it is even more configurable than Dependabot and is not limited to the GitHub platform.

Why and how teams use dependency bots?

To get an idea on how and why software development teams use dependency bots in their projects, we conducted a small survey at current Senacor projects. The personal interviews consisted of twelve questions lasting for about 20 minutes, regarding the tech stack, the review strategy as well as the motivation for the use of dependency bots, and success and failure stories. All interviews were done in March 2024.

Overall, we found ten teams at Senacor that were using dependency bots. Nine of them were using Renovate Bot and two were using Dependabot (one team was using both, Renovate Bot and Dependabot). Whereas the motivation for the use of dependency bots was almost everywhere developer-driven, the application time for the use varied from just a few months to three and a half years.

Most teams let their dependency bots do their work every night and most often, the reviews for updates were done manually. Only two teams were using the auto merge feature, and two other teams were considering it. For most of the teams, the information used in the code reviews was limited to the release logs and config diffs provided with the pull requests. Unsurprisingly, almost every team was relying on high quality integration tests when merging their dependency updates on a regular manner.

The overall satisfaction rate for the use of dependency bots was high. Nine out of ten teams said they were satisfied with their bots. However, two teams were struggling with too much noise caused by too many pull requests. One team was highly struggling with unmaintained, open pull requests, wasting a lot of their CI resources, as all these pull requests were rebuilt and reintegrated after every change to their software repositories.

How you should use dependency bots

Based on our survey results and our own experiences, we derived some tips and tricks on how you should use and configure your dependency bots.

Define a review process
Before you start using dependency bots, you should think about your review process. In the end, somebody must do the code reviews and if pull requests are not reviewed, you will waste your CI resources and you will not benefit from your dependency bot. Let your team discuss who will review the pull requests, and when. Some teams have rules such as the first developer starts working in the morning will check all the open pull requests, or a different developer will be responsible for all the updates every week. Decide for a solution you and your developers are happy with and keep in mind that the sooner the updates are merged, the more you will benefit from the automation of dependency updates.
Use Renovate Bot or Dependabot
As both our scan for existing tools and the survey revealed, there are only two bots applicable for the job. If you are at GitHub, you may use either Dependabot or Renovate Bot. Otherwise, you must choose for Renovate Bot. Both tools are free, highly configurable and their application effort is low.
Automated tests are mandatory
If your software projects do not come with good, automated tests, you will not benefit from dependency bots. Without tests you cannot be sure that everything is working fine after the updates. In such a case, a green integration build does not mean that your software is still working. Thus, if your project is lacking for test automation, set up automated tests having a good coverage first.
Schedule updates right
Select an appropriate schedule for the dependency updates to avoid too much noise. Let the bots do their job when the resources are available (for example, at night when build servers are typically less exhaustively used than at daylight). Adapt the schedule to your review strategy. If you plan to do your reviews only once per week, why should the scan be done every night? For most of the teams we interviewed, a scan every night seemed to work fine.
Reduce noise and manual effort
The less pull requests your bot will create the less effort you will have. Thus, use grouping of similar dependencies to combine pull requests. In Dependabot and Renovate Bot, groups can be defined with Regex-like naming patterns or semantic versioning (e.g., major vs. minor/patch). Thus, you can define groups such as “core framework” and “other dependencies” or “production”, “development”, and “testing” dependencies. Play around with these groups and find out which grouping fits best for your projects.
Use extensible configuration files
Reuse is always a good idea. Thus, if you have several projects or repositories, consider reusable dependency bot configuration files. You can define a root configuration file that is extended by every software artifact in your repositories. That way, you can easily adapt the configuration of all your projects. For example, you can temporarily exclude specific dependency updates, if you know that you do not want to update the major framework you are relying on just now. Whereas Renovate Bot supports configuration extensions out-of-the-box, for Dependabot you must set up your own approach (for example by writing a script that automatically updates the configuration file in all your source code repositories).
Try to use auto merge
If your reviews consist of just checking the build and test results, you could (or should?) have used the auto merge feature. Check, which of the dependency groups you defined could be applied for auto merge. Development and testing dependencies, Typescript type annotations, or patch version updates can be good candidates to explore the auto merge feature. Of course, you will always have some dependencies you will not update automatically (e.g., minor or major updates of your core framework such as Spring Boot, Micronaut, Angular, or React). However, for other dependencies the auto merge feature can reduce your manual effort dramatically, leading to even higher benefits when working with dependency bots.
Define a maximum review capacity per iteration
Especially when setting up dependency bots for your projects, you might be overwhelmed by the many pull requests you will receive in the first iterations of the update process. Thus, it might be a good idea to limit the number of pull requests you will receive per iteration and repository, which is supported by both, Renovate Bot and Dependabot.
Consider automatically assigned reviewers
An assigned pull request is more likely to be reviewed. Thus, you should consider defining a pool of developers that are assigned to pull requests created by your dependency bot automatically. This way, you can reduce unmaintained updates as well as existing updates nobody is aware of. Automatically assigning reviewers to pull requests is supported by both, Renovate Bot and Dependabot.
Prioritize security updates
Although every dependency should be kept up to date, it may be sensible to focus on security first. Thus, both dependency bots allow you for prioritizing on vulnerability-related updates, creating you pull requests for security updates, even if the configured pull-request capacity is exceeded.
Consider dependency updates being part of your reporting
If you are having automated release reports, consider the integration of dependency updates into your reporting. You might want to label and count dependency-update related releases, showing your management, how much time you are saving by using dependency bots. Depending on your reporting, configured commit message prefixes can be a possibility to mark the commits included in your releases as dependency-bot related.

Summary

Summing up, everybody should use dependency bots everywhere all the time. Manual effort is dramatically reduced as most of the dependency updates can be reviewed and merged within minutes. Furthermore, you save a lot of time and pain when new vulnerabilities are detected, and dependencies must be updated as soon as possible. However, before you set up a dependency bot, think about your review process first. Use grouping of dependencies to reduce the number of pull requests and noise being created. Configure the tool as it fits best to your development process and consider of using the auto merge feature. The less manual effort you are investing, the better. Working with dependency bots is like working with continuous integration: if you have used it once, you will not work without it anymore.