Vendasta houses a great number of our frontend applications and libraries inside a monorepo that we have code-named “Galaxy”, powered by Nrwl’s NX tooling. The monorepo was introduced to do a number of things, including but not limited to:
- Increase code sharing
- Eliminate disparity of node package versions across all frontend projects
- Reduce variation in build process
- Speed up frontend development cycles
- Share linting and formatting configuration for all projects
Before the spring of 2020, Vendasta heavily invested in a hybrid model that had some core frontend projects living in a monorepo but some SDKs, libraries, and all applications living outside in their own repositories. It worked well for the time but had a few drawbacks, the biggest being the development cycle.
Developing in the hybrid world
Let’s start by breaking down exactly what’s going on in the graphic below.
The process for working in our legacy frontend stack
Here is what’s happening:
- Make a branch for the shared feature you’re adding or updating in the Frontend shared repo
- Make a branch in the project(s) where you need the shared feature
- Add or update the component you need
- Make the changes you need for the feature
- Push to your branch so you can wait for a temporary npm package to be built
- Install the temporary package in the project you need the feature
- Test your changes and if there’s an issue start back at (a)
- Merge changes into master shared frontend repo
- Wait for the master npm package to be built and published
- Install that master npm package into your project
- Merge your master branch into the project
Most of these steps require waiting on builds, waiting on publishing, and installing the changes into the project of choice. In the end, your hopes of reaching the final step depend on nothing going wrong along the way. As you can imagine, this type of a process leads to not wanting to make shared components in favour of just writing the components where they are needed, and copy/pasting them in other projects where they need to live. This isn’t the wrong way to do it, but if you ever want updates to be reflected across the entire platform, you’re stuck copy/pasting code even more and potentially fighting with changes other developers may have added in their projects.
Enter An Experiment in More Micro Frontends
In an attempt to try and increase code sharing and reduce the number of one-off components being made, we formed a frontend infrastructure team dedicated to owning a core library of components for others to use in our applications across the platform. To start, we explored several different ways of creating micro frontends to make it easier to serve smaller artifacts to projects so that developers could pick and import exactly what they needed into their projects. This included making use of a snazzy new product called bit.dev, which specializes in React and Angular micro frontends. Imagine that each component is its own npm package with its own sandbox to play in before importing it into your project.
This began to prove very useful as developers that required common shared components no longer needed to create them themselves or wait on the old dev cycle and then have to maintain these components. The frontend team would take special care in crafting each component to suit the needs of multiple consumers and publish it to bit for others to test and make use of. Maintenance was now extracted in a way that decreased one-off components but, unfortunately, this came with a huge drawback: each project had a different version of these components in a relatively short time period. This is something we knew was a problem in the past, so we quickly moved away from bit.dev in favour of another solution. Something that hadn’t been around in awhile that developers were starting to rant and rave about again.
The Monorepo
Micro Frontends quite obviously had bitten us several times, so we decided to make the shift to the polar opposite side of the development world: introduce a monorepo that’s built on Nrwl’s NX monorepo tooling.
Once inside the monorepo, the first order of business was to dog-food it by starting our work on the Galaxy Frontend Framework that would be used by all residents of the monorepo and by clients that live outside. This would allow us to start to work out some of the kinks of the monorepo, learn the process, and create more tooling before any other dev teams moved in. Our goal, after all, is for every single frontend project at Vendasta to live in the monorepo. An added perk is that any libraries in the monorepo are evergreen.
Our goal, after all, is for every single frontend project at Vendasta to live in the monorepo
After sorting out the project structure, properly making use of NX build settings, and starting to use a new tool called Codefresh for our CI/CD management, it was time to start bringing projects and dragging along the developers who work with them. Migrating projects into the monorepo is fairly straightforward but involves changing the directory structure of the project to match that of the NX monorepo, and then using git to copy the project over.
The Good, The Bad, and The Ugly
We can’t expect any new piece of technology to be a silver bullet. We have discovered some drawbacks which can be solved and have been finding more and more than the benefits are impactful and plenty. I like to understand this process in terms of the classic Western film, The Good, the Bad, and the Ugly.
The Good
- NX Tooling allows us to only build projects that are affected by our changes
- We have one source of truth for all tooling and build process
- All libraries are evergreen when consumed from within the monorepo by apps and other libs
- Finding the source for a lib is as easy as ctrl+f. No more sifting through countless git repos!
- A single package.json means that all apps and libs are on the same version of node and use the same version of shared packages
- Shared and consistent linting and formatting rules
- A better dev process (see next part)
As we can see from the graphic below, the dev process has been drastically improved for the residents of the monorepo. As a quick example, developers need to make a branch in a single repo to update or add a new component. Their changes are reflected immediately since code is now living side by side—no more waiting on npm publishing. Once they’re happy with their changes, the developer is ready to do a code review! If they need to, they can have libraries automatically published that can be consumed outside the monorepo.
The monorepo dev process happens all in one repo.
The Bad
- More projects in the monorepo means potential for longer build times when touching dependencies that affect many different projects
- Solution: This can be remedied by better hardware, updates to Angular (11 has build improvements), and parallelizing and sharding more of our process.
- Evergreen is not tied directly to monorepos but can be daunting to developers who want to make a change to a library consumed by many apps
- Solution: Increased testing and reporting can lead to more developer confidence when making these kinds of changes
- Solution: Increased testing and reporting can lead to more developer confidence when making these kinds of changes
The Ugly
- A single package.json is daunting and is a dependency of all projects. When triggering updates, all projects are considered affected and need to be rebuilt
- Some IDEs struggle with large monorepos. So far, from an anecdotal point of view, VSCode seems to handle this fairly well
Closing
Each engineering organization has different needs. Sometimes micro frontends are king, but for Vendasta we have navigated back into the world of a monorepo. So far the benefits we’ve encountered outweigh the negatives, which can always be resolved via some smart minds and the elbow grease of a couple of developers. In the end, the biggest improvements we’ve seen are a faster dev process, increased code sharing, and a reduction in disparity of dependencies. Stay tuned for more on the Vendasta Frontend stack.