Transcript: Static Site Generators at VA.gov: Metalsmith, Gatsby, and the Future of Static Sites

This is a transcript. For the video, see LINK.

[00:00:00] Preston So: Hello. And welcome to another episode of Tag1 TeamTalks. Today is an exciting episode. I'm really, really happy about today's topic because we're going to be talking about static site building and static site generation at the Department of Veterans Affairs or the va.gov website.

[00:00:16] I'm Preston So. I'm the editor in chief at Tag1 Consulting. I'm also senior director of product strategy at Oracle and also the author of the only book on decoupled Drupal, Decoupled Drupal in Practice. Today I'm joined by Michael Meyers, Managing Director at Tag1 Consulting and our guest today, Neil Hastings, technical lead for the core CMS team at VA.gov. Let's go ahead and jump right into this topic today.

[00:00:41]So Neil, you know, I think static sites, right? JAMstack, these buzzwords have been coming up all the time. So one thing I wanted to kind of dig at is I think a lot of people have been, you know, hearing about this new, what I call kind of the second wave of static site generators, you know, not so much the, the old days of Jekyll or, or some of those older projects, but some of these new ones that are popping up, like Gatsby as part of the, what we call the JAMstack.

[00:01:08] So I wanted to ask what exactly is driving people towards this and why would anyone want to use something like a static site generator, especially this kind of new second wave that's coming up?

[00:01:20] Neil Hastings: Yeah, the JAMstack is really interesting and, I, you know, used Jekyll in the past when it came to the VA.org project, The, the site being up all the time is still mission critical.

[00:01:29] Like you said before, it's mission critical application. If we go down then a veteran could not be able to relay information back into the hospital and not know where to go for if a hurricane comes up. They rely on this information for a lot of really important information. So people are driven by, like, the separation of, of, front end display from content.

[00:01:48] It's just a natural separation, high performance. It's highly scalable, it's highly secure. You don't have a PHP website, you know, delivering, in our case, we're using AWS and S3 delivery, everything. So we didn't have a web server. I don't, I don't have 2:00 AM pages anywhere.

[00:02:02] I'll have to get up and go, you know, bring the site back up. Right. It's always there. So those are some great benefits we have with the static site generation.

[00:02:11] Preston So: And that's a really, you know, I think for all of us who have experience with those pages, right? I mean, I remember being on call 24/7 and having to get a page, you know, one story I always tell this is getting a page and I'm having to go into a McDonald's to get their free wifi and fixing up one of these active, actively running servers.

[00:02:31] So static files are definitely, we, something that I think is, infinitely scalable, right? Cause they're just, you know, static files. But one thing I wanted to do just before we jump into some of the kind of, pros and cons is, you know, the second wave of static site generators is kind of part of this new model called the JAMstack right.

[00:02:47] Java script, APIs, markup. A lot of people are talking about it. There's a large community starting to emerge around this. But let's talk a little bit about what are some of the disadvantages of this. And I think this also hearkens back to some of the disadvantages that came up from the first-generation of static site generators as well, where there is a manual step. Isn't that right?

[00:03:09] Neil Hastings: Yeah. That's right. So with any, you know, it's, it's a trade off, right? So with, with that great scalability of those static HTML files, they're static, HTML files. So that's where a content being pushed. You got to figure out how to get new updates out.

[00:03:22] What is your build pipeline look like? How complex is that? How fast or how fast is content going to be published? So a big piece we have at the VA is, managing editorial editor expectations of how long the content takes to get out. Our, our particular front end is driven by some static content, some React widgets, just kind of a, a mix of the, those two levels, the two different types of, you mentioned we have kind of a mix in there.

[00:03:44] There's definitely React widgets. We have a public API was also, a traditional static build in there. So that kind of, that adds a lot of complexity. And if any larger organization. Over time, we would kind of move from one, one level to the next level. And, as we go, yeah, so our particular team, our particular situation also, we actually have different teams that manage the different system.

[00:04:05] So I'm, I'm on the CMS team, which we go up to the GraphQL, we use GraphQL API, everything on the other side of the constant bill is, is actually a different team. We cross over from time to time to help each other. Mostly it's CMS team going to the build side. But at the same time, like I there's, I don't own that code so I can do pull requests, the repository.

[00:04:26] I, you know, my, my project manager doesn't own that situation. So we have two different teams. I manage that. So is that kind of different team also, which is also an advantage right because you can scale teams separately and you can build those out your, your various specialties of skills.

[00:04:40] You know, Drupal skills is different than a CSS, HTML skill. And instead of like having a Drupal person, who's also a theming themer, traditional Drupal themer, that person can be HTML. JavaScript can do the coolest things with React and not have to know, not have to know Drupal itself. So that's kind of a mix there.

[00:04:59] Preston So: And I think it's really interesting, Neil, you mentioned a couple of really interesting aspects to this VA.org project, which I think really bear mentioning, which is that you know, first and foremost, you've got a clean separation of concerns between the people working on your content and those who are working on the front end architecture, which is, I think one of the really big advantages of, of decoupling Drupal or, or using Drupal solely as a backend system.

[00:05:24] But then the other thing I think that is really interesting, that you mentioned is that you're actually able to mix and match and, you know, not only use Metalsmith, but also use React and put in some of these more dynamic components, which I think is part of the second wave of static site generation that we're seeing.

[00:05:41] So one question I had was, you know, I know that you all are on Metalsmith. I used to work for Gatsby, so, so I've got some experience with that. What are some of the other static site generators or tools or frameworks that, that you've worked with and that you like?

[00:05:55] Neil Hastings: Absolutely. So, I, there's a lot of the more traditional first layer ones.

[00:06:00] Jekyll a little bit with Hugo in the past. Gatsby is definitely one we're evaluating right now and kind of doing proof of concepts. We're looking at migrating off of Metalsmith. It's kind of an older - we're in an older version. They don't have things like incremental builds. Gatsby's incremental builds stuff.

[00:06:12] They just announced. It's really cool. We're really excited about that. Incremental builds is the future for us. Next JS is something we've played with a little bit. We actually tried to implement tomb, which is a Drupal module. That wasn't great for our use case. So we rolled that back recently.

[00:06:26] We're moving to, we - we're moving to a different model, but those are all ones we've used in the past. They're all great for different situations. Yeah.

[00:06:34] Preston So: I couldn't agree more. I mean, you know, I definitely used Jekyll and Hugo, in the past. They're both great, both solid. But I agree. I think some of the, you know, I know that some of these projects like, Gatsby, next JS, think Tome might be actually looking at this too, looking at kind of, you know, how do we limit the amount of time it takes for these builds to actually generate, because the time durations can be pretty long, especially for a site.

[00:06:57] I imagine that's as large as VA.org, right?

[00:07:00] Neil Hastings: Yes, that's right. So our, our build pipeline is very complicated because it not only pulls in content from Drupal, but also pulls in some content from a old markdown. That was the old version. So they had - originally had a Jekyll site. So the original site was a Jekyll site that content got migrated the Drupal, but not all the content.

[00:07:18] So some of the contents in a GitHub repository, markdown files, some of it's in Drupal, some of it's combining custom React widgets. So the build pipeline actually pulls all this stuff together into one VA.gov site. So on the VA gov site, I mean, it's, it's, I can pull up some analytics. I was gonna pull it up via the Google analytics, but if I'm at look, we have, we get millions of hits like a day.

[00:07:39] So this is, it's a very high load sit as you would expect. Our CMS. We currently have around 120 active users. We're editors, and we're going to expand, I want a thousand, within the, this year, active editors on the site. So that's a lot of stuff going on at once. That's just, we have a lot going on. And that the biggest, like we talked about a couple of times the biggest issue we're dealing with right now is the time it takes to publish. So our entire build pipeline takes almost an hour. to do. You think entire build, end to end.

[00:08:09] Before? So getting the site to live? So which, which is a long time.

[00:08:13] Right now, we, we build twice a day to it. So editors can only get our content from pupil live twice a day, the React widgets are live all the time. We have a facilities, API benefits, information. Those parts are live, live data from live API, but the Drupal content is only published twice a day. And getting that pipeline done is the biggest thing - is the problem. We're having right now.

[00:08:31] We've, we've tried to address this in several ways. We've improved graphQL queries, REFCO queries. They used to do this one large one. It used to take an hour just to run it. Now we run it. We've several, several small ones, and that takes about five to 10 minutes now to run a build to get off the content.

[00:08:47] That's just going to - just going to continue to grow. Right? So without. It, it, we have to do a full site rebuild every time we deploy our content. So every HTML page gets recreated. All the content gets pulled. It's basically doing the entire Drupal database dump into JSON and then another system is processing that data.

[00:09:03]That's not super efficient, right? So incremental builds is the future. We have to go there one page at a time. But tons of tons of problems there, tons of potential, you know, complexities there with dependencies of page, links and, not just like links within Drupal are one thing easy to manage.

[00:09:20] But remember we said, this is. Oh, it's content from all over the place. So how do we know if a page on Drupal has content pointing to a different content repository and when to update when so it's, it's an interesting problem to solve. Actually. I love these kinds of things. So I'm excited about it.

[00:09:35] Preston So: One thing that I'm hoping to see in the near, or, or maybe far future is a way that we can almost make builds so incremental that they're streams, to the point where, you know, you know, anytime you save something in your CMS, it just kind of, you know, sends that tiny little chunk over because, you know, Neil, you, you, you identified one of the really core issues, right. Which is that any website and by the way, but, you know, I have to make a correction here because, the show notes said VA.org, but we're actually talking about VA.gov

[00:10:04] So I just want to be very clear, that we're talking about VA.gov, not VA.org. So one of the things I think that's really hard is that with a, you know, as you said, mission critical application like VA.gov. This is, this is a content, you know, a Corpus of content that is constantly growing, right?

[00:10:21] You're going to have more and more information constantly being uploaded. You also got to potentially archive that information so that people can refer back to previous versions of content. So there's never going to be a situation where you're going to reduce the amount of content that's ever represented in one of these static site builds.

[00:10:36] Neil Hastings: That's, that's a hundred percent true. We are a government organization. So we do have laws that government wasn't allowed and not allowed to do. And archiving content is in there and there's visibility of content and accessibility laws. We have to make sure we're following. So the archive ability is we are every part of that build process I was talking about is we use S3 bucket version.

[00:10:55] So every deploy to S3 is a new version. So that that whole site is archived there. Within Drupal. Nothing's deleted, everything's - we have an archive state, in our workflow, but nothing's ever deleted. So contents always going to grow, right. We use revisions for everything. So revisions are just going to be constantly your growing databases.

[00:11:13] It's going to be awesome.

[00:11:16] Preston So: I definitely want to talk more about the build pipeline, because I think there's a lot of interesting things to talk about in there. But one thing I think is also really interesting and this, you know, it goes back to obviously the spirit of open source and a Tag1. And of course, you know, the idea of open and publicly available civic technology.

[00:11:33] So what I understand is that. VA.gov is actually completely open source front-end and back-end across the stack. And I believe that this is available even on GitHub. Can you tell us a little bit about, you know, what what's, what's kind of, well, first of all, what's out there in the open. How have people been contributing and also, what's it like to be in the open like this?

[00:11:54] Neil Hastings: Yeah, it's actually what drew me to this project in the first place. I was looking, I wanted to do government work, but so much of it is traditional behind the, behind the scenes. You know, nothing's visible and no transparency. This is an amazing project. We have a GitHub the Department of Veteran Affairs GitHub page.

[00:12:09] And, are there are, if you go to that page alone, that org there's tons of stuff on there. I mean, for anybody the main repos on there, there's a Vet dash website and that's the front end build. There's a VA.gov/cms. That's the Drupal CMS and there's a VA.gov/team one. And that's what most teams use to manage their issues.

[00:12:30] There's a lot more behind the scenes, a lot more libraries and lots of that stuff. But, so anybody in the world can go to and spin up the site. They can, they can, as long as you have access to GitHub, you can clone the CMS you can, which is Drupal. You can run. We have, we use Lando for, with Docker. So you spin up the Lando instance.

[00:12:49] We have a startup script that pulls down a sanitized database, which is it's all public data. It's all data. Just the only thing that sanitizes the user information, right. Everything else is in there. So you can go pull that in, spin up a Drupal site, spin up that site. You have the Drupal CMS, the way it is for our editors.

[00:13:05] Exactly the same. Same with the front end. You can spit it out the front end. You can spin up the vet's API, which is a Ruby back-end all that you can spin up and any create the whole infrastructure outside. Of our pipeline. That's what we do. So if we have a CI pipeline that spins it up separate, we use the same scripts everywhere, which was really cool to be in the open.

[00:13:22] And not only is our code in the open, but our entire process in the open. So we, on our CMS team repo, a CMS repo, but we actually have, you can go in there now and search for, in we also use ZenHub also to manage when you ZenHub, it's a plugin for Chrome that manages issues. It kinda, it kinda gives you a little board within GitHub.

[00:13:41] But we use that to mentor epics and our user stories and tasks, and all the PRs are public, everything. So anybody can go in there and do a review. We've had a couple of people when we were doing interviewing for new people to come on. We actually, some, someone would go in there and do PRs. So like just before they even got the job.

[00:13:57] Right. So they'd go new PRS. We ended up hiring one of the guys that did that. So it's, it's definitely cool to have that out there. It's a little nervous because you get to be, you get to be a little more diligent about what you write in your issues and your commits, and you gotta remember, this is a government, a very serious government organization.

[00:14:11] So, we use emojis and we smile. We kind of be, try to be falling as much as we can, but you just still had to be cognizant of your audience. Right. So, and I think it's great. So you know, all of our, our initiatives are out there and like, so I think that's kind of important, knowing that. These are American tax dollars.

[00:14:30] I'm spending, they're paying my salary. They're just doing all this work to paying for it. They're paying for all this stuff. So I think it it's, everyone should know what's going on and we should be able to get, get to it in my opinion. So I'm excited. That's pretty part of the organization, the organization and what we're doing.

[00:14:43] So, yep.

[00:14:44] Preston So: Well, I got to say, I really wish that more people had that perspective of open government, not only in the public sector, around the world, but also in the corporate world as well. You know, I think we would all benefit from, learning from, you know, all of the work you're doing in the processes that you've set up.

[00:15:00] I just took a look myself at the GitHub repository, incredibly well-documented, you know, incredibly friendly, very clear in terms of how things work. And, I want to encourage everybody to go onto GitHub and visit the Department of Veterans Affairs Organization and check out some of the open repos that are there.

[00:15:18] That's really cool. So we talked a little bit about. Just to, to, you know, we, we touched on the CI pipeline and I want to get back to that a little bit later. And we also talked a little bit about, you know, what it's like to be operating this kind of open government, code base. Let's talk a little bit about the you know, the ways in which your team works together.

[00:15:37] I think, you know, obviously as you said, millions of people are hitting this website all the time, and I just wanted to ask, you know, just before we go, go back to the technology a little bit is, you know, what is it like to be working on the VA.gov team as a content editor, as a technologist? There's so many teams, there's so many tools, right.

[00:15:56] But how has the number of teams interacting with this architecture and the team size in terms of developers? How has that impacted the decisions that you've made?

[00:16:06] Neil Hastings: Yeah, that's a great question. Well, we are very blessed to be open and transparent. We're still a government organization and it's still the contract. Similar. So, a lot of the, we like to say your, sometimes your technology choices, are dependent on contract and the team's ours. But being part of this overall platform, we kind of call the platform team within the VA to VA.gov; all these different companies is just at least five or six different contracting companies, very large ones, all work, all very working there very well.

[00:16:37] I don't feel the, the isolation between teams, even between like, even with, on this, on the CMS team, there's two different contracting companies, right? So that's just, that's more of a government structure the way that works. It's a lot of complexity as far as who owns different pieces. Like I talked about before our code pipeline is incredibly complicated and there's not one team that owns the entire thing.

[00:16:59] So there's at least three different teams. We have CMS team, we have a frontend tools team. We have a, an ops team, all three of those teams actually contribute in some way. I said three, what was wrong was actually currently five teams because we have two different product teams that build products on top of the platform that, and they actually also build on top of the same pipeline.

[00:17:21] They had the temp, the HTML templates and stuff like that. They work on the rescue brokers and create HTML templates. So there's all these different teams. And, what's been amazing is that we all came together to, you know, realize the problems we have of, of the build pipeline speed, how long that takes. So we're, we're solutioning together, working at the con where we're gonna fight through the next steps together.

[00:17:42] We're going to implement it together. So while it's, it's, it's definitely complicated and we're having the different levels of product managers at different levels, even product owners, there's groups within the VA that own the different products themselves. So all those different competing priorities come up, but -we'll, you know, it works out pretty well in the end, for the veterans I think.

[00:18:00] [00:18:00] Preston So: [00:18:00] And I think that's ultimately what, you all are doing this for, right. Is for American veterans. And then for those who have served our country. And I think it's really easy when we talk about the technology. When we talk about these build pipelines, it's easy to forget that there's a human side to all of this.

[00:18:16] Not only the people that we have to work with and the people we have to build these solutions for, but also the end users who are, who are going to be really benefiting from, the applications that we produce. So, so I want to, so I wanted to ask a little bit about some of the tech stack here. We, we, we touched on the backend, we talked, you know, we touched on the Drupal editorial piece.

[00:18:35] We touched on Metalsmith as well. You know, as you mentioned, it's pulling from multiple sources. There are these React widgets that are in there. So let's go back a little bit to the, to the CI pipeline. I mean, you walk this a little bit through the build pipeline. What are you using in terms of your continuous integration process?

[00:18:50] I see that, you know, you mentioned for example, that you're using Jenkins, but like what other kinds of tools are in the mix. And after that, I'd love to hear a little bit about your deployment pipeline

[00:19:00] Neil Hastings: Sure. So Jenkins is our main, main, our main system or building orchestration system right now.

[00:19:05] We're actually evaluating other solutions, our main ops team, or building an entirely new stack Docker, all Docker based Kubernetes based on AWS. So the other tools that we use, we use GitHub actions, on the Drupal side, a lot to run, a lot of code quality tests. We use Circle CI a bit though.

[00:19:22] We're honestly kind of moving off of Circle CI to GitHub actions more, that, and that's one of those government contractual things. I've just, that's the direction from above saying which way to go. So that's what I'm going to give it an action. It's just been, it's been great. It's been a pretty fast, for the CMS team itself.

[00:19:36] We actually use Tugboat, for our CI. And Tugboat, it's a tool made by Lullabot. They - it's all Docker hundred percent Docker based. We transitioned onto it recently, as in December is when we first transitioned and it's been an amazing tool to iterate. Well, our CI Pallium before is more traditional one big server.

[00:19:54] Using Apache it'd be hosts and stuff. Since moving over to Docker and Tugboat, we've been able to upgrade PHP Drush because we were on Drush 8 for awhile. Just recently last week, actually we got to Drush 10, so exciting. And all these tools we couldn't upgrade because of our, our previous stack.

[00:20:11] We break things now, we're upgrading them very quickly. So, the very exciting stack there, I could talk, I could do a whole conversation about Tugboat. Yeah, so those are the main tools we use. Like Jenkins is the, is the main one we have the main system that we deploy into is called BRD. It's called build a release and deploy.

[00:20:26] The build pipeline is based upon a GitHub commit. So commit gets created, runs a test suite, runs any kind of build that's needed, creates, creates the artifact and then pushes that to GitHub as a release. And then those releases might then be deployed by. Deployed job within Jenkins right now.

[00:20:44] Preston So: So, you know, you know, you used to share, you know, I think one of the things that a lot of people are struggling with right now, which is when do I make that jump, right? When do I make that jump from things like Apache and, this old way of doing things over into, more containerized architectures, more of the sort of test driven development, even that, that Tugboat, and those sorts of things enable.

[00:21:05] So let's, let's talk a little bit about, you know, we talked about. Your kind of build pipeline, which is, you know, obviously part of that, BRD or BDR. I always get those letters, BRD.

[00:21:16]Let's talk a little bit about the deploy pipeline. Cause I think that's a really interesting part of this. One of the things that I think many people who are listening to this or watching this might not realize is that, when it comes to static sites, there is both a build step and a deployment step.

[00:21:30] And both of those things are incredibly important and are very different from the previous ways of doing this kind of continuous integration. So one thing I know about, your deploy pipeline is obviously you're pushing up to S3. But let's talk a little bit about some of the really exciting things that you're doing in the deploy pipeline.

[00:21:47] You hinted earlier at doing things like link checking. Can you talk a little bit more about what's involved there? Yeah, absolutely.

[00:21:53] Neil Hastings: So our deploy pipeline before any code goes out with certain checks that, that occur, to make sure that, we don't bother like the veterans, like, so this is all my wrist is all about making sure the veterans get the services.

[00:22:04] We have a really complex link checking service that goes through and goes through every HTML page that was there. It, it makes sure it runs, how everyone's going to conduct a container. And I'm not sure the details of the top of my head, but it makes sure every link is real and it has to go to a 404 page.

[00:22:19] And that, that kind of 404 page might happen. If someone publishes a link to a page that's not published yet, or they publish content, that's not ready to be published many, many times we've caught content that someone published it, but it wasn't ready to be published. And it had a link to something else on it.

[00:22:33] And that actually stopped the deployed from going out. So we stopped someone from. Putting out, pushing out pages that weren't ready to be pushed out yet. Like when we migrating from some of the health care facilities from their old sites, which are these team sites to Drupal one time when someone mass published one of the facilities and it wasn't ready yet, but the link checker caught that and didn't go out.

[00:22:55] So there is a great, great check. Other things that does, like I said before, everything's versions because they have to be, so it keeps an archive copy of the old site. All these releases are done. We use all get, GitHub releases, so you can go to GitHub to the tags and releases and, and, you know, get all of the content that releases, which is kind of interesting.

[00:23:11] Is it also pushes a, a release to Composer. So the vet's website, which is the static site, we use composer to pull that into the Drupal side, to run to our, an ICI test against a mixture we can still build. So that's kind of an interesting mix of pulling a JavaScript, your website into a Drupal website, and we use composer to do that cause that way they tag releases within, and it takes you do tags and releases within GitHub.

[00:23:35] And then we have dependent by that runs with it, get up to get up service. If it depended by it does our PR every day, we just merged that PR in and the next deploy we're updated. Otherwise we're running tests against old stuff. And, we have some demo sites, RCI pipeline also holds demo sites. So when, like I talked about those facilities that are moving from old sites to Drupal, we use Tugboat to stage all the content.

[00:23:55] So they'll go on there and otherwise we're publishing content that's old and, yeah. So, but that deploy pipeline pushes stuff to static S3 and then that's basically it, right. We don't deploy to an Apache server. We don't have to worry about like, A file system changes this, those there's no server, you know, that's and that's a really different part.

[00:24:16] Like, yes, it's some people they but there's a server in AWS. Yes, there is. But from my perspective, there's no server, I just pushed you out, S3 object store and that's it, like, and it's super simplified perspective, right? There's always DNS. We have load balancers. And when you go to the VA.gov website, you're not actually going to ADP's directly, you're hitting some kind of F infrastructure that I'm not even aware of because it's, it's some kind of a file load balancer that gets routed.

[00:24:42] It gets routed and started, but eventually just ends up from my perspective of my user's perspective is like, really I really need servers. So, which is really cool.

[00:24:49] Preston So: Cool. And that's so cool. I mean, you know, I just, just now I loaded VA.gov, you know, hadn't visited it before it loaded so blazing fast. I mean, it was so fast.

[00:24:59] I had never seen a US government website load this quickly. In any situation ever, I, including like things like WhiteHouse.gov, which, which did load pretty quick back in the day. This is amazing. I mean, and I think one of the things that's really amazing about this architecture, Neil and I really do have to commend you and the team for this is that there is so much underlying complexity and so much under, under the hood, activity going on, as you mentioned, GitHub actions, doing all of these task runs.

[00:25:25] Also some of the really interesting things around dependency management with Composer and dependent box, but to the end user, it's just that the S3 bucket, it's just not just store there's literally, it's just a series of static files. And that I think is, is the beauty of, you know, the ways in which we're really using the static site generators in a way that is extremely alien to how they were originally used, which is like basic blogs, you know, basic, you know, static websites into this fully fledged application layer, which I think is, is really striking. So, so we've talked about, we've used the word site and application kind of interchangeably here. Right? And I think there's a sense of, you know, well, okay. S3 buckets, that's just, that's just the site, you know, it's just assets, but let's talk a little bit about that interactivity with those React, widgets, the end user experience.

[00:26:17] Now I understand that you, you all have a project known as Lighthouse and just for the, the readership, the viewership, the listenership, this is not Google's Lighthouse project, but this is the VA's own Lighthouse projects. Tell us more about this Lighthouse, API.

[00:26:34]Neil Hastings: A hundred percent.

[00:26:35] So, like I talked talk about before the build pallet can pull stuff from over the place and we have React widgets that pull real-time data.

[00:26:42] So the data it pulls, we have a public API that anyone can access and go create account on. It's the URL is developer.va.gov. You can go register an account. And, a little, some of the history behind that URL behind that site, it was originally set up for people to share their VA medical information with like Google health and like, those are the other.

[00:27:02] So when you set up an account on VA, you can actually link your medical information to it as a veteran. It's grown a lot, like, like if you, if you go in there, into the, and you kinda, you poke around some of the different areas we have, kind of looking at them myself. So there's lots of different areas.

[00:27:17] So, there's benefits information in there and all this is backed by a kind of, Nginx load balancer, and we have kind of which uses. What does it use?

[00:27:30] I can't remember the system at the top of my head. The, the api.va.gov is eventually backed by Ruby. By a rail system. And that's, again, it's all public and go access, which I think is also really cool. The open data OpenGov sort of initiatives. That's in there. Yeah.

[00:27:43] Preston So: And one thing I think that's really interesting about this is, you know, as we see more and more government agencies begin to open up, APIs and developer ecosystems, the possibilities are endless, right.

[00:27:53] I can just imagine somebody using the Benefits API in Lighthouse or the Health, API, and Lighthouse to build a chat bot for veterans, right. Or, or, or a voice user interface, right. Especially for veterans who have been injured during, you know, due to some sort of an IDE and they can't look at a screen for too long, they want to use something that's verbal instead of something that's visual.

[00:28:15]It's a really compelling, compelling thing. I actually want to dig into this.

[00:28:19] Neil Hastings: Amazing and the funniest, the chat, we actually have a chat bot, projection in the works. They're working on it right now. And the chat bot working with -------. not just a connecting real people together, but also kind of doing some more AI based stuff where tries to answer questions based upon the Q and A information. So there, within the CMS, there's a Q and A set up content type. We put all information in and reading from that and being able to answer a veteran's questions based upon that QA information. So that's something we're working on. There's a team working on that, where they integrate with our GraphQL system, again, to build out that that's another other piece of the puzzle.

[00:28:50] That's a great idea though. Yeah.

[00:28:51]Preston So: That's amazing. I would love to hear more about that. You know, I, I, I worked on a team that built out a very similar, quick question and answer machine for the georgia.gov website. Okay. It's actually believe it or not the subject of my upcoming book, this summer, about voice content.

[00:29:05] So it's interesting. Yeah. I definitely want to talk more about that. So, you know, as we know. You know, and as you've mentioned several times, right? There's - you can't have this kind of amazing architecture that's open and transparent, but also is doing this amazing orchestration of all of these different integrations and all of these different environments without having some challenges.

[00:29:26] So, I wanted to ask, you know, what are some of the, some of the, you know, I know you've mentioned some of the plans for improvements coming up. I know you're looking at Gatsby potentially for evaluation, but what are some of the kind of other challenges you face with the infrastructure? Like build pipeline problems, GraphQL server problems.

[00:29:43] And especially when it comes to, you know, obviously some of the users we have to serve who are those editors who build the content for, for our veterans.

[00:29:51] Neil Hastings: Yeah. Thanks. It, it, there are other trade-offs, even though are not up at 2:00 AM, I still have a server to manage. I still have expectations of me being up, but being performant.

[00:29:59]And we, and I, you know, it's not, I say me, it's my team. We have an amazing team. That keeps everything running very well. The GraphQL server. So I talked a little bit about how we changing the change, our queries around before, but the load on our Drupal server. It's, it's, it's a lot. We, we built pretty often.

[00:30:17] So the CI, all the CI pipelines hit our database, the same database that they use, and that seemed the videos the editors are using. Right. So that, that can be, a performance problem. We have, you know, our CPU issues, RDS it's really complicated. We're actually working a lot, you know, working on mitigating that as far as, like you said, that since we have the separation of the front end build and we have the content system, kind of editorial expectations, right?

[00:30:43] So traditional Drupal, traditional CMS, you press save, press publish, and it's there. You don't have that here. You have to wait for the build pipeline to go out. You have to wait, it could be an hour, it could be 12 hours until the next deploy goes out for your content. Right. So setting that expectation and communicating that information back to an editor on that page is very important.

[00:31:03] We have things like link checking and, and when you're on a page is a URL you're on actually live. So they can kind of look at that. But again, that's a whole totally different sort of thing. We have documentation all over the place. So, like I say, lots of different teams, lots of different documentation.

[00:31:17] So documentation within the CMS of ReadMe documentation, there's, there's diagrams with different, we have different tools. We use like mural and whimsical and lucid charts. And everyone kind of each team kind of uses their own thing once in a while. And multiple GitHub repository. So keeping all that information up to date is also a very challenging, because it's, it is constantly changing.

[00:31:36] We were agile, so it changes all the time and keeping that up to date for everybody, especially with the public. It's important and, it takes time.

[00:31:44] Preston So: Absolutely. So, I want to talk just briefly about some of the, and I know that we're starting to run out of time here, but you know, one of the things I do want to mention though is, you know, obviously this is a unique project, this sort of, you know, when I think back to a lot of the decoupled Drupal architectures I've worked on, or a lot of the static site generator architectures I've worked on, this is one of those really unique, kinds of kinds of architectures.

[00:32:07] And so, you know, Neil, I think this is a huge accomplishment, you know, obviously props to both you and the entire VA Tech team. What are some of the things that you all are most proud of? And what are some of the things that kind of, you know, get you up in the morning and, and, you know, get that coffee in your system and get you to work.

[00:32:25] Neil Hastings: Yeah. So, I mean, it's honestly, it's the people. Everyone across the team or platform team on all of our teams, we're all service minded, mission focused people. We understand the veterans and we understand the service we're providing. We understand how important it is at the end of the day, even with the arguments and pseudo political stuff.

[00:32:41] It, it, we all realize that we, we all can come together on that vision. Right. Which is great. It really helps keep me going. We, our, our team, like I said, we are not only is our project in open source, we'd be issued an open source. We do a lot of contributions back to drupal.org. We sponsor where we can.

[00:32:55] We have a lot of several open source projects. We have a GitHub page department of veterans affairs. You can look us up. We've. We, we continue that way. We have lots of really great ideas for open-source editorial experience projects coming in the future that I can't wait to be part of. We do speak at DrupalGov cons..

[00:33:10] We try to lead the way in, in Drupal government open source, and really we're focused on doing things the right way and necessarily always the easy way. So, w we have really great understanding from top down that, doing things the right way, saves money and time saves the money over time. So it may be more expensive now, but in a year it's going to be cheaper.

[00:33:31] And that's the mindset go back to you know, understanding we're spending tax dollars. So that's a mindset we have all the time. I'm doing things the right way, which is we have that support top down all the time.

[00:33:41] Preston So: Absolutely. And, and by the way, just to note for those who are listening here the VA has a Drupal page that's drupal.org/department-of-veterans-affairs, department of veterans affairs, separated by hyphens.

[00:33:52] Amazing, amazing contribution coming from one of the most important and essential agencies in our federal government. And I just love [that. You know, I think that one of the things, especially in this polarized political environment, I think one of the things that we can all agree on right, is, is taking care of those who have, you know, made the ultimate sacrifice for our country. So, next thing I wanted to talk about was the future, you know, I think, and this is where we'll probably end things. Unfortunately though, I imagine the, you know, we could probably have a three hour long conversation about these things.

[00:34:20] I'm very interested in, in, like, I really want to dig into a lot of these pieces more with you. But, like, you know, you, you mentioned some editorial improvements, you mentioned some interesting stuff coming up on the, on the open-source roadmap. You know, I know that you're looking at Gatsby, what are some of the things that you're looking at in the future?

[00:34:36] What is VA.gov going to look like potentially into three or five years?

[00:34:40] Neil Hastings: That's great questions and we're right in the middle of those planning sessions, this next quarter of brokering into teams and figuring out like we kind of have some North star getting principals. So the main one is we want, by the time one editor presses published, we want the content to be alive in a minute.

[00:34:55] We, we spent some time talking about how hard that's going to be today. And that that's really, that's the, a main thing, getting that content live as quick as possible. So what does that look like? Is it Gatsby? Is it, is it still Metalsmith? Is it a, what does that take? We don't know that yet. And that's actually the exciting part for me is I love that solution discovery and just figuring out what we're going to do some cool things. We have the Drupal side. We w we did a great amount of research user research last year, for a system we're calling EWA. So we can't get it off of E walk without the K EWA. And, we call it editor for editorial workflow and assignments. So building off of Drupal's workspaces, building off Drupal core on the revision system pre real site real-time previews, revisionable, previews.

[00:35:36] And I would like to share a link. The all it's, all public, all the real user research and designs you've already put into this, And that's coming up this next year. It's going to be really, really cool, exciting stuff. I have a pretty big background in, in revisional preview. I made a system in Drupal seven called site preview system SPS.

[00:35:53] I was part of a. Initiative on that. And that was a lot of, that was a lot of fun, real complex, real great project. Building on top of that building the type of work others have done, you know, standing on the giants of Drupal and just trying to get a little higher is how we're doing, it's it's, it's going to be pretty cool stuff.

[00:36:06] Preston So: Well, Neil, I had no idea that you worked on SPS because I was a user of site, previous system myself. So I use some of your code back in the day. I'm very happy to have done that. That's amazing.

[00:36:17] Neil Hastings: I hope it did good for you. I - that was that, that was a very fun project for me when I - we did that one. Yeah.

[00:36:22] Preston So: I mean, that was an incredible, you know, I think, I think SPS you know, for those who don't know much about kind of Drupal history and, and how, how things were about. Oh, gosh, five, five or six years ago now. Seven years ago. Yeah. You know, SPS really was revolutionary. I mean, the fact that you could have this completely comprehensive holistic preview approach that encompassed your entire site and gave you that complete peace of mind, it's, it's something that is still rare in the industry.

[00:36:50] I don't, I don't think, you know, it's still very rare today, so it's very difficult and it's a huge achievement. Well, you know, this is really cool. I'm excited. I've got all of these links, open myself, Neil, and we'll obviously provide all the links as well to our viewing audience, our listening audience.

[00:37:05] This has been an incredible conversation. Thank you all so much for hopping on this. A webinar or podcast today with Tag1 Team Talks. One thing I did want to say by the way, Neil is I've actually got a book coming out very soon on Gatsby. So, I'll send you a copy once it's ready to go.

[00:37:20] It'll probably be after you've already made your decision, but you know, maybe at all, it'll, it'll be something that could add to your shelf, you know, anyway, thank you so much. And if you enjoyed this conversation and you want to see more of this, if you want to bring Neil back to talk about Tugboat QA, if you want to bring Neil back to talk more about some of these exciting new projects that they're working on around the White House, APIs at VA.gov remember to upvote subscribe, let us know how you feel about this webinar.

[00:37:46] Share it with your friends and family, with your grandma. Who's still at home unable to leave because of coronavirus. Make sure to check out our past talks at tag1.com/tag1teamtalks. And as always, we'd love to hear your feedback, any suggestions you have about any topics. Write to us at tag1teamtalks@tag1.com.

[00:38:05] And just want to say once again, thank you so much, Neil. It was a real pleasure to have you. It's always a pleasure to hear about some of the amazing things going on and a thank you also to Michael Meyers and we'll see you next time.