This blog series highlights the important lessons that we learned when we thought we were practicing continuous delivery. This is the third in our series of confessions from continuous delivery experts. This story is about the experiences of the Snap CI team in moving a monolith codebase to microservices and why they chose to move back.
Snap CI was a continuous integration and delivery server for teams who work in the Cloud. If you were using GitHub, Snap CI could take your commit, put it through an automated test or suite of tests that you've written, and then help you deploy it to your Cloud target (like Heroku, AWS etc) .
Confession: We fulfilled all the practices on a checklist, but we failed on the core principle which is ‘to deliver changes quickly’
Snap CI’s team had very enthusiastic and experienced CD practitioners. The team wanted to do ‘everything CD’. Within six months after we began the product, we had a microservices architecture with nine different components, sophisticated messaging queues, monitoring, an automated infrastructure, and deployment pipelines for all of these different components.
“We were in CD Nirvana”
We had everything except for one important piece - delivering quickly. We were delivering to production every two weeks, which sounds “not so bad,” but was not a business decision. We were unable to deploy any sooner than that. The product manager was exceptionally frustrated that they were unable to do daily ‘continuous deployment’ as the business needed to.
Confession: We did not pay attention to the other core principle of continuous delivery, which was to deliver changes reliably
The second problem we had, was that when it came time to deliver, the likelihood of key features working was only about 50%. All that monitoring we had in place wasn’t giving us usable feedback. We were forced to rely on manual testing or using a small test group to inform us of any failures. We were unable to move the product forward for public use.
Assess and reassess your continuous delivery practices
It’s commonly thought that microservices are essential to do continuous delivery as a best practice. However, in our situation they were not right for us at that time.
We were enamoured by the idea of “microservices” and separated our components without pausing to think if they truly made sense for delivery. As a result, components were only separated superficially. When we changed one component, we still needed to change the data structure in the other components. And because of this, we were unable to deliver those pieces separately from each other.
At this time, we made a decision to go back to a monolith structure which put us in a better place to understand what was going on with the product. This gave us the ability to gradually release more often and allow the business needs and the needs of the product to dictate where those splits should be.
Lessons we learned
We learned that users don’t care if your team is practicing continuous delivery or using microservices. These are decisions a team makes to ensure what they build reaches users quickly and reliably. It’s easy to get caught up in the checklists, “doing CI and CD”, trying cool new microservices etc.
Do the important things first that will deliver code quickly and reliably and then evolve and move forward on your journey.
The second lesson we learned was that CD is a journey. It's an evolution, and you first need to set your goals driven by your business and then make your way there. It is tempting to move to the latest practice or new technology as soon as you hear about it. However I recommend that in the first few weeks, unless you are 100% sure that that's what you need, don’t make any major architectural decisions. Do the important things first that will deliver code quickly and reliably and then evolve and move forward on your journey.
Snap CI was created by Thoughtworks Products. It is no longer available and has been discontinued since Feb 2017.