Drupal is one of the largest and most active open-source software projects in the world. Behind the scenes is the Drupal Association, the non-profit organization responsible for enabling it to thrive by architecting and introducing new tooling and infrastructure to support the needs of the community and ecosystem. Many of us know the Drupal Association as the primary organizer of the global DrupalCon conference twice a year. But it's less common knowledge that the Drupal Association is actively engaged in Drupal development and maintains some of the most important elements of the Drupal project. This runs across the spectrum of software localizations, version updates, security advisories, dependency metadata, and other "cloud services" like the Drupal CI system that empower developers to keep building on Drupal.
With the ongoing coronavirus pandemic, the Drupal Association is in dire financial straits due to losses sustained from DrupalCon North America (one of the largest sources of funding) having to be held as a virtual event this year. As part of the #DrupalCares campaign, we at Tag1 Consulting implore organizations that use Drupal, companies that provide Drupal services, and even individuals who make their living off Drupal development to contribute in some shape or form to the Drupal Association in this time of need.
We are putting our money where our mouth is. For years we have donated at least eighty hours a month to support the DA and Drupal.org infrastructure and tooling. I’m proud to announce that we are expanding this commitment by 50% to 120 hours a month of pro-bono work, from our most senior resources, to help the DA offset some of its operating expenses. Furthermore, we contributed to help #DrupalCares reach its $100,000 goal and so that any donation you make is doubled in value.
To gain insights into building software communities at scale in open source, Michael Meyers (Managing Director at Tag1) and I (Preston So, Editor in Chief at Tag1 and author of Decoupled Drupal in Practice) recently kicked off a Tag1 Team Talks miniseries with the Drupal Association's engineering team, represented by Tim Lehnen (Chief Technology Officer at the Drupal Association) and Narayan Newton (Chief Technology Officer at Tag1), to examine all the ways in which the DA keeps the Drupal community ticking.
Why Tag1 supports the Drupal Association
Here at Tag1, we work with a diverse range of technologies, but Drupal has been our passion for many years. It's been a critical part of our business since Tag1's inception, and we're grateful to the Drupal Association for sustaining such an essential part of our work today. By no means is it an understatement to characterize the Drupal Association as the lifeblood of the Drupal ecosystem. Because of our appreciation for what Drupal has given us, we're committed to doing our part to giving back to Drupal, not only over the course of our many years working in concert with the Drupal Association but also right now during the #DrupalCares campaign.
How we contribute to Drupal
Though Tag1 is well-known for being the all-time number-two contributor to the Drupal project, with the largest concentration of core committers, branch managers, release managers, and core maintainers of any organization in the community, we're much less known for how we support the underlying foundations of the ecosystem. Beyond the more visible contributions of staff members like Moshe Weitzman, Nathaniel Catchpole (catch), Francesco Placella (plach), and Fabian Franz (fabianx), we also do much more than add our support to Drupal core development. After all, supporting Drupal requires more than just code; it also requires the tooling and infrastructure that keep the project's blood flowing.
During our Tag1 Team Talks episode with the Drupal Association, Tim Lehnen eloquently made the case for the non-profit that has driven Drupal's success for so many years: While the software makes up the bulk of open-source contributions, offering surrounding services that buttress the software's core is another key function that the Drupal Association performs. To that end, for many years, Tag1 has donated 80 hours of pro-bono work a month to ensure that Drupal.org and all the tooling the community relies on stays up and running. Tag1 is honored to increase our monthly contribution of pro-bono hours to the Drupal Association by 50% from 80 to 120 hours of expert work from our most senior resources. And now with our increased work hours and financial contributions, critical projects like the migration to GitLab can continue to move forward, even during a situation like the current pandemic.
Supporting Drupal's test infrastructure
In Drupal, a key aspect of code contribution is running tests that verify a patch will work against a massive variety of environments, be compatible with a spectrum of versions of Drupal, and not introduce any functional regressions in the code. One of the key questions many community members ask is why Drupal maintains its own testing infrastructure in lieu of a service such as TravisCI.
Unfortunately, whenever existing continuous integration solutions were tasked with running a Drupal core test for every Drupal patch, they would consistently time out, maxing out available resources. To solve the challenges associated with developing and testing at scale, the DA partnered with Tag1. We deployed our expertise in infrastructure, mission-critical application development, and performance and scalability to help run and maintain Drupal.org's servers and the DrupalCI test runner system. The CI system ensures that contributors have a reliable center for collaboration and a dependable test infrastructure for all of their patches and modules. Tag1's deep expertise has been critical to the success of the DrupalCI system, which we scaled dynamically to the extent that it is now concurrently running more than an entire decade's worth of testing in a single year.
The new testing infrastructure was an enormous undertaking for the Drupal Association due to its complexity. Narayan Newton opted from early days to leverage standard Unix tools to build out the environments for testing targets. And rather than use Kubernetes for the orchestration of tests, the Drupal Association opted to use Jenkins and the EC2 Fleet plugin for DrupalCI. Jenkins manages the orchestration of virtual machines (VMs) and initializes them as test targets before actually running the tests themselves in a clean room environment. As Narayan notes during our conversation, one of the most fascinating quirks of Drupal's infrastructure is that many of its core elements were installed before standardized tooling emerged to handle those use cases in a regimented way.
Supporting Drupal's migration to GitLab
In addition to our contributions to Drupal's underlying infrastructure, Tag1 also assists with key initiatives run by the Drupal Association such as the ongoing migration from Drupal's homegrown Git system to GitLab, a source control provider. According to Narayan, the migration to GitLab has been much more straightforward than previous historical migrations in Drupal's past, more specifically the original migration from Drupal's previous CVS source control system to Git, which it has used ever since. Code management in Drupal has long employed a bespoke Git approach with a homegrown Git daemon written by the community and cgit as the web-based front end for Git repositories.
One of the key benefits GitLab provides to the Drupal Association is the fact that the DA is no longer responsible for building and supporting a source control system for Drupal at the scale at which it operates. After all, GitLab has a dedicated site reliability engineering (SRE) team focused on ensuring source availability even at high loads. And as Narayan notes, GitLab has been responsive to security issues, in addition to facilitating "one of the smoothest migrations I've been a part of." But this doesn't mean there weren't complications.
Because GitLab has a superset of features that include some existing Drupal.org functionality, the Drupal Association, supported by Tag1, worked closely with the GitLab team to ensure that certain features could be disabled for use with the Drupal project, avoiding many of the issues that have plagued the GitHub mirror of Drupal since its conception. Narayan contributed key features to ensure that GitLab's integration points could be toggled on and off in order to enable the unique needs and requirements of the Drupal community and ecosystem.
Tim adds that in terms of lack of downtime, disruption, forklifting the entire Git code management infrastructure without disrupting the development community was a rousing success, especially given that there was no impact on a minor version release. In the process, the Drupal community has gained a number of key features that will enable accelerated development and conversation between contributors in ever-richer ways. In coming months, the Drupal Association will also facilitate the addition of GitLab's merge requests feature, which will introduce yet more efficiencies for those making code contributions.
Why #DrupalCares is so important
For us, Drupal is a key reason we exist, and the Drupal Association has done wonders to ensure the longevity of an open-source software project we hold dear. This is why in these troubling times for the Drupal Association, it could not be more important to uphold the ideals of open source and ensure the survival of our beloved community and ecosystem. Over the course of the past month, we've witnessed an incredible outpouring of support from all corners of the community, buttressed by the various matches provided by community members like none other than project lead Dries Buytaert. We at Tag1 Consulting have contributed toward #DrupalCares' $100,000 goal in order to multiply the impact of community donations and buttress our existing support.
Without your support, whether as a company or an individual, we may never see another DrupalCon grace our stages or celebrate yet another major version release that introduces innovative features to the Drupal milieu. And it's not just about the more visible elements of the Drupal experience like DrupalCon. It's also about the invisible yet essential work the Drupal Association does to keep the Drupal project rolling along. Thanks to the innumerable contributions the Drupal Association has made to maintain DrupalCI, the GitLab migration, Composer Façade, and a host of other improvements to Drupal's infrastructure and tooling, with the support of Tag1, the Drupal project remains one of the most impressive open-source projects in our industry.
Conclusion
Here at Tag1, we believe in the enduring value of open source and its ability to enrich our day-to-day lives in addition to the way we do business. We're dedicated to deepening our already extensive support for the Drupal Association in ways both financial and technological. And now it's your turn to return the favor. If you're an individual community member, we strongly encourage you to start or renew a membership. If you're an organization or company in the Drupal space, we encourage you to contribute what you can to ensure the continued success of Drupal. Together, we can keep Drupal alive for a new era of contribution and community.
Special thanks to Jeremy Andrews and Michael Meyers for their feedback during the writing process.