A cornerstone of evolving and re-platforming older/legacy systems is understanding how the legacy system utilizes data, all the shared data flows, and how to bridge the gap between legacy system and new architecture.

One of the biggest risks as architects that we can take is thinking that we can easily start over and recreate something that was years (if not decades) in the making. If we go down that path and put our blinders on to what exists today, we run the risk of building systems that have missing features and take much longer than anticipated to even get to MVP.

When thinking about re-platforming systems, there are a lot of great resources online about enterprise architecture patterns that are particularly useful to evolve systems. A popular pattern that I like is the strangler pattern, popularized by Martin Fowler.

Essentially, this pattern seeks to minimize risk by slowly replacing pieces of legacy functionality with modern application design, as illustrated below:

While this is an extremely effective technique in code, it is difficult to achieve when thinking about the data storage evolution. The goals of modernization also typically include moving to more modern and scalable data storage. This can be anything from introducing search capabilities through a search engine like ElasticSearch or moving from a flat file indexed storage to a relational database.

In those cases, it is almost guaranteed that the legacy system will not know how to read or write from the newer data stores, and conversely, you are not going to want to read and write to the older data stores from the newer platform. Unfortunately, it is also very likely that the legacy system will need the updated data that the modern system writes.

How can we solve for this and still decouple the legacy and modern systems? Do we have to retrofit the legacy system to read and write from the new shared modern data store? The answer is no. We can keep the two systems separate from each other, but able to communicate in a decoupled way by introducing a messaging system in the middle.

With this approach, every time there is a new transaction/write from the modern application, a message or event would get sent through the eventing system. We would create a consumer for that event that knows how to write the data into the older data store that the legacy system relies upon.

With this in place, we can slowly migrate all writes from the legacy system while not breaking any downstream data needs or changing the way the legacy system reads data. This mitigates risk substantially since changing the shape of the data and the reads can cause significant unexpected side effects and relies on good regression testing strategies to be successful.

Most eventing systems are built to scale horizontally, on commodity hardware, so that we can set up multiple parallel listeners, queues, and topics to be able to process messages very quickly. Not only can this help take load off the legacy system, it can also help organizations scale their current ETL batch processing.

Eventing/Messaging Systems - a Pillar of Modernization by Russ Harding