Over the course of Drupal’s lengthy history, one of the most common feature requests has been automatic updates. A common complaint of Drupal site administrators, especially those who have smaller sites updated less frequently, is the frequently complex and drawn-out process required to update a Drupal site from one minor version to another. Updates can involve a difficult set of highly specific steps that challenge even the most patient among us. Indeed, many in the Drupal community simply choose to ignore the automatic e-mails generated by Drupal.org indicating the availability of a new version, and waiting can lead to compounding security vulnerabilities.
Fortunately, the era of frustration when it comes to automatic updates in Drupal is now over. As one of the roughly dozen Drupal Core Strategic Initiatives, Drupal automatic updates are a key feature that will offer Drupal users better peace of mind when minor releases occur. Over the last several years, Tag1 Consulting, well-known as leading performance and scalability experts in the Drupal community, has worked closely with the Drupal Association, MTech, and the Free and Open Source Software Auditing (FOSSA) program at the European Commission to make automatic updates in Drupal a reality.
Recently, I (Preston So, Editor in Chief at Tag1 and author of Decoupled Drupal in Practice) sat down with Lucas Hedding (Senior Architect and Data and Application Migration Expert at Tag1), Fabian Franz (Senior Technical Architect and Performance Lead at Tag1), Tim Lehnen (CTO at the Drupal Association), and Michael Meyers (Managing Director at Tag1) to host a Tag1 Team Talks episode about the story of Tag1’s involvement in the automatic updates strategic initiative. In this blog post, we dive into some of the fascinating background and compelling features in Drupal’s new automatic updates, as well as how this revolutionary feature will evolve in the future.
What are automatic updates in Drupal?
Listed as one of the Drupal Core Strategic Initiatives for Drupal 9, Drupal’s automatic updates are intended to resolve some of the most intractable usability issues in maintaining Drupal sites. Updating Drupal sites can be a challenging, tedious, and costly process. Building an automatic updater for Drupal is a similarly difficult problem, with a variety of potential security risks, but it’s a problem that other ecosystems have solved successfully. Following Dries’ announcement of automatic updates as a strategic priority, several early architectural discussions took place, especially at Midwest Drupal Summit 2018 in Ann Arbor.
Automatic updates in Drupal provide certain key benefits for users of all shapes and sizes who leverage Drupal today, including individual end users, small- to medium-sized agencies, and large enterprises. Advantages that apply to all users across the spectrum include a reduction in the total cost of ownership (TCO) for Drupal sites and especially a decrease in maintenance costs.
As for small- and medium-sized agencies and individual site owners, it can be difficult—and deeply disruptive and anxiety-inducing—to mobilize sufficient resources in a brisk timeframe to prepare for Drupal security releases that typically occur on Wednesdays. For many end users and small consultancies who lack experience with keeping their Drupal sites up to date, high-alert periods on Wednesday can be deeply stressful. And for enterprise users, how to incorporate security updates becomes a more complex discussion: Should we integrate manual updates into our security reviews or keep adhering to continuous integration and continuous deployment (CI/CD) processes already in place?
Where are Drupal’s automatic updates today?
The full roadmap for Drupal’s automatic updates is available on Drupal.org for anyone to weigh in, but in this blog post we focus on its current state and long-term future. Automatic updates in Drupal include updates on production sites as well as on development and staging environments, although some integration with existing CI/CD processes may be required. In addition, automatic updates support both Drupal 7 and Drupal 8 sites.
Because of the ambitious nature of the automatic updates initiative, as well as the desire by the module’s maintainers to undertake a progressive approach from an initial module in the contributed ecosystem to a full experimental module in Drupal core, the development process has been phased from initial architecture to present. Currently, a stable release is available that includes features like public safety alerts and readiness checks.
As for other developments within the scope of available funding from the European Commission, in-place automatic updates have also arrived. If a critical security release is launched, and your site has the automatic updates module installed, you’ll receive an e-mail notification stating that an update is forthcoming in the next several days. Once the update is available, the module will then automatically execute the in-place automatic update if all readiness checks show as green on the Drupal user interface, meaning that no additional action is required on the user’s part.
Key features of Drupal automatic updates
Together with MTech, the Drupal Association, and the European Commission, Tag1 has been heavily involved in architecting the best and most graceful approach, particularly in such a way that it can be generalized for and leveraged by other open-source software projects in the PHP landscape. This includes approaches seen in other ecosystems such as readiness checking, file downloading, and signature verification that generates “quasi-patches” as well as inspiration from the WordPress community. One of the team’s major concerns in particular is ensuring the continuous integrity of update packages such that users can be confident that such packages are installed from a trusted source.
There are three key features available as part of automatic updates in Drupal that will be part of the initial release of the module, and we discuss each of these in turn here.
Public safety messaging
After the noted security vulnerability in 2014 commonly known as “Drupalgeddon,” a notice was posted indicating that a critical release was forthcoming. When it comes to automatic updates, a similar process would occur: Several days before a critical security release for Drupal core or for a contributed project in Drupal, a notice would be posted and available on every Drupal site.
This sort of public safety messaging allows for an additional communication mechanism before a key update so that site owners can ensure they are ready for an update to land. In Drupal sites, the feed of alerts originate from the same security advisories (SAs) that the Drupal Security Team and Drupal’s release managers issue.
Readiness or “preflight” checks
Every Drupal site with automatic updates installed will also have readiness checks, also known as “preflight” checks, that run regularly every six hours through Drupal’s cron and will inform site owners if their site is prepared for an automatic update. Readiness checks are essential to ensure that sites are not functionally compromised after an automatic update.
For instance, if Drupal core has been hacked by a developer, if a site is running on a read-only filesystem, or if there are foregoing database updates that need to be run, readiness checks will indicate that these issues need resolutions before a site can automatically update. There are eight or nine readiness checks available currently, and some are simple warnings to aid the user (e.g. in situations where cron is running too infrequently to update the site automatically in a timely fashion), while others are errors (e.g. the filesystem is read-only and cannot be written to). Whereas warnings will not impede the commencement of an automatic update, errors will.
In-place updates
The final crucial component for automatic updates is in-place updates, the centerpiece of this new functionality. The in-place updates feature in Drupal’s automatic updates downloads a signed ZIP archive from Drupal.org. Using the libsodium library, the feature then compares the signature of the ZIP file to verify that the resulting archive matches Drupal.org’s official archive.
Thereafter, in-place updates will back up all files that are slated for update and update the indicated files. If the process concludes successfully, the site issues a notification to the user that the site has been upgraded. If something fails during the procedure, in-place updates will restore the available backup.
Common questions about automatic updates
On the recent Tag1 Team Talks episode about automatic updates in Drupal, contributors from Tag1 and the European Commission answered some of the most important questions on every Drupal user’s mind as the initiative continues to roll out automatic updates features.
What about using Composer versus tarballs?
One of the key differences between Drupal implementations today is the use of the Composer command-line interface to handle Drupal’s installed modules in lieu of managing module installations through tarballs. Due to the widening use of Composer in the Drupal community, if a site has updated to Drupal 8.8.0 or later, the site will be using Composer. And if the two key Composer-related files in Drupal codebases (namely composer.json and composer.lock) are not modified, automatic updates will continue to function properly. However, for sites leveraging Composer and subsequently modifying the /vendor directory in Drupal codebases, this question becomes more complicated.
At present, the automatic updates team will release early versions supporting all scenarios for Drupal sites, short of those sites that have modified composer.json and composer.lock directly. By observing users as they gradually adopt automatic updates, the team plans to learn much about how users juggle Drupal dependencies in order to release improved update methods that accommodate Composer much more gracefully.
Are automatic updates part of Drupal core?
As of now, automatic updates are not part of a vanilla Drupal installation, but all major components of the module will be incorporated into Drupal core in due course. The in-place updates feature presents the most overt difficulties.
Before in-place updates can land in core, the automatic update team plans to implement an A/B front-end controller that is capable of swapping between two full codebases and toggle back to the backed-up, out-of-date codebase if the update exposes certain issues mid-flight.
What is the future of automatic updates?
While the European Commission has funded the first twelve months of work over the course of 2019, there is much more work to do. The initial European Commission funding accounts for the three aforementioned key features, namely readiness checking, the delivery of update “quasi-patches,” and a robust package signing system, all focused on security updates, which are the most pressing. However, the current year of development excludes better support for Composer and contributed projects.
The longer-term roadmap for automatic updates includes the A/B front-end controller mentioned in the previous section, more robust support for Composer-powered sites, and other types of updates. These include updates for contributed modules and themes as well as batched successful updates for sites that have fallen particularly behind.
Conclusion
Automatic updates will reinvent how we maintain and upgrade Drupal sites, particularly in the realm of security. Because they allow novice and experienced Drupal users alike to save time without the need to worry about how they will implement updates, this strategic initiative improves the total cost of ownership for Drupal users of all sizes and backgrounds.
No account of the extraordinary initiative that is Drupal’s automatic updates would be complete without appreciation for the sponsors of the developers involved, especially from the Drupal Association, MTech, Tag1 Consulting, and the European Commission’s FOSSA program. Organizations and individuals alike have sponsored automatic updates in Drupal to widen awareness of their brands, to showcase their skills as developers, and to attract other Drupal contributors and resource Drupal teams.
To sponsor the continued success of Drupal’s automatic updates, please consider sponsoring development by contacting the Drupal Association. And for more insight into automatic updates directly from the module’s creators, check out our recent Tag1 Team Talks episode on the topic for information we were unable to fit in this blog post.
Special thanks to Fabian Franz, Lucas Hedding, Tim Lehnen, and Michael Meyers for their feedback during the writing process.
Click the following link for our Tag1 Team Talk on Drupal Automatic Updates!