Choosing an architecture

It’s all about the dependencies

Creating a bank from scratch is nothing like your usual Sunday stroll. Picture it more like going on a trek in a far and unknown jungle. As for all tough and long-running activities, it requires a good preparation and carefully chosen tools which allow to move fast and won’t break half way.

In software, the decisive tool isn’t the database or the framework we use, it’s not even the programming language, it’s the architecture. Simply put, it is how we organize our code such that it meets the technical and operational requirements of our business domain.

We defined at least five key requirements for our Core Banking System.

The choice we’re making now will be decisive for the pace at which we’ll be able to adapt to our customers, and provide them with the best experience.

Our business lies in our data. Money itself is only data, an account balance is a number somewhere in a database that lets you pay your rent, your bills and your food. But that’s not enough, we want to be fully transparent on our data, be able to justify each and every operation that happened on an account, leading to its current balance. Our architecture must allow that by design.

This is very good for traceability, everything that ever happened to the system is in our database. But now even the simplest query becomes very complex. Getting an account balance, for example, would force us to iterate through the event store to sum deposits and withdrawals. Not only this is not trivial to implement, it will also get slower and slower as events pile up.

That’s the reason why ES is used in combination with another pattern called Command Query Responsibility Segregation, or CQRS. This pattern handles reads and writes in the system with two very distinct entities. In this configuration, ES offers a very natural boundary between them. This architecture is usually referred to as CQRS/ES.

The read side waits for events from the event store, and has its own database. After each event, it updates his database accordingly.

BankAccount is called a projection of the event stream. This approach has tremendous advantages when it comes to performance and maintainability.

Another advantage of those projections is that they can live on their own, which brings me to another of our requirements.

Our system, like most systems, will need to handle a lot more reads than writes. Yet, we need to keep short response time on every request, especially on the write side. If we’re a little too long to respond to a card transaction authorization, the transaction is rejected, causing frustration to our customer.

That’s why the event store must be asynchronous. This way the write side can simply post events to it and move on, not waiting for all the projections to handle them. Moving projections to different machines, we can absorb a huge traffic with zero impact on the write side. If the read side becomes overwhelmed, we can even replicate the same projection on multiple machines, all connected to the event store.

While this is very good for performance, there’s now one problem.

We make our business decisions on the write side, based on a given command and the current state of the system. If this state is in the projections, we’re likely to do a dirty read, thus making a decision based on an outdated state.

To solve this problem, we divide the write part into small stateful components called aggregates. One of them would be “a bank account”. An aggregate handles commands and produces events based on its internal state. In the following example, the state is the bank account balance.

The aggregate uses its state to make business decision. For example, it can reject a withdraw command due to insufficient funds.

Since there’s only one instance of an aggregate per bank account, it ensures consistency. You’ll also note that we don’t need to rely on transactions and database locks.

Our final criterion was maintainability. The CQRS/ES architecture is harder to setup than traditional ones, but it improves the maintainability over time.

Most parts of the system are pure functions, which means they are very straightforward to test. An aggregate, which is where most of our business logic resides, takes commands and produces events with no side effects. Most tests will inject some commands and simply check the produced events.

Since most parts of the system are small and decoupled from each other, multiple developers can easily work on different parts without conflicts.

Projections are very tolerant to errors, since we can retroactively fix them. If we made the wrong rounding in the accounting projection for example, we don’t need to migrate the data after the fix, we only need to replay the events.

CQRS/ES is not perfect though, so we carefully considered all aspects.

CQRS/ES is a very opinionated pattern and some tasks may become more complicated than in traditional systems.

It should be made really clear that CQRS/ES rarely fits a whole system, and should be used sparingly. Using it for user management and roles for example wouldn’t make much sense, and would better fit a more traditional CRUD system.

Short answer, we don’t make breaking changes. Migrating previous events is dangerous, a bit like traveling in time and irremediably changing the course of events. Instead, we add a version to each event and handle the different versions in aggregates and projections. That means we must be very careful when designing those events in the first place. However, it must be noted that it isn’t more difficult than handling the versioning of a standard HTTP API for instance.

There’s an additional concept coming from DDD/CQRS called a “saga” or a “process manager”. It reacts to domain events and produces side effects.

Related posts:

I saw a room full of knitted brows, tightened shoulders, wringing hands, and twitchy eyes. I stood in the front of this church fellowship hall and felt the burden of division and pain. In the context…

We are pleased to announce that Cartesi is partnering with Binance Smart Chain (BSC) to allow advanced smart contract capabilities and massive computation for DApps atop Binance Smart Chain. Binance…

Hyperlinks are definitely a cool feature in websites but hidden hyperlinks or opening a page that opens 3 more immediately or opens up the app store on my phone make me want to burn down the people…