Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.

The pendulum swings! Microservices to Monoliths

Derek Comartin Leave a comment

We moved from monoliths to microservices a decade ago, and there has been a swing back to either consolidating microservices or moving to a modular monolith. This isn’t surprising. Why is this happening? I will explain why logical boundaries don’t have to be physical. Once we finally make this distinction, it opens up many possibilities.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=BFcxgcoO5Ns

Physical Boundaries

One of the good things that came from microservices is defining boundaries. Good fences make good neighbors, as the saying goes. Defining boundaries is a good thing, but how exactly are we defining them? Microservices, as defined by Adrian Cockcroft

Loosely Coupled service oriented architecture with bounded contexts

This means that we have services defined by a bounded context, which comes from Domain Driven Design, and these services are loosely coupled between them. Typically, this would imply an event-driven or message-driven architecture.

I typically define services (not the micro part) as the authority of a set of business capabilities.

What does a service do? What are the capabilities the service provides? At least with all the baggage of microservices came the idea of defining boundaries.

Unfortunately, microservices, as developers have implemented them, have forced boundaries to be physical boundaries. But when talking about what capabilities a service provides, I’m talking about logical boundaries. They aren’t the same thing. Physical boundaries are not logical boundaries.

Forcing this idea can cause a lot of issues and is not flexible. Typically you’d have a logical boundary (what a service provides) also be in its own source repository (git repo), which is built and turned into a deployment artifact run in some environment as process, container, etc.

Or you might have a mono repo where all the source is under a single repository, but the end result is still the same where a logical boundary is built and turned in its own deployment artifact.

If you’re logical boundaries are also physical boundaries, you likely end up having service-to-service communication that happens over the network.

This could be HTTP, gRPC, or any other type of synchronous request/response via network calls. What’s the problem with that? A lot. Check out my post REST APIs for Microservices? Beware! as I explain some of the pitfalls including, latency, failures, and more. This ultimately is a distributed monolith. check out the fallacies of distributed computing which is still very relevant. If you were to go back to Adrian’s definition, he mentioned loosely coupled, which this is not. Making RPC/network calls does not make anything loosely coupled.

Logical Boundaries

So what’s the difference between a logical boundary and a physical boundary? The best way to describe this is probably thinking about the full scope of a system. You likely have many different aspects to it. There’s probably some Frontend/Client/UI, a backend, and a database and other infrastructure like maybe a cache.

A logical boundary is the vertical slice across all these layers. A logical boundary owns everything related to the capabilities it provides. That includes the UI, the backend API, the database it is persisting to, and any other infrastructure it owns.

There are different ways of thinking about boundaries. A great illustration of this is the 4+1 Architectural View Model by Philippe Kruchten.

There are different ways you can look at a system. As I’ve been mentioning, there is a logical view, a development view (source code, repo), and a physical view (deployment).

A logical view can be the same as a physical view, but the point is they don’t have to be. Meaning, that a logical boundary doesn’t have to be deployed independently.

Composition

Once you realize this, you can see that you can compose things differently depending on your needs.

You could have multiple logical boundaries, within a single mono repo, which is built as a single unit of deployment. That single unit of deployment could be single process.

That means that if we were communicating synchronously between logical boundaries, we could be doing so in-process and not be making network calls. It’s just functions calling functions in-process.

I’m not suggesting to compose all your logical boundaries into a single deployable unit. I’m illustrating that logical boundaries aren’t physical boundaries. They don’t have to be one-to-one. You have options.

One logical boundary could have a single source repo that when built creates two different deployment units, let’s say a container.

One logical boundary could be across multiple source repos that each get built into their own separate containers.

You could have a single logical boundary that is spread across multiple source repos that gets built into a single container.

A better illustration of this would be if you have two different logical boundaries for a mobile app with a backend. Service A could provide a backend and a mobile front end. That same friend might be composed of another logical boundary to build the APK for Android.

This applies to infrastructure. Each service should own its own data, but that does not mean it needs to own the infrastructure. You could share infrastructure without sharing data. A single database instance could host different schemas owned by each logical boundary.

Could you have “noisy neighbors” where one service is consuming too many resources on your single database instance? Yes. At which point you could separate that out and use a different physical instance. The point is you don’t have to right from the get-go.

Jumping back to Adrian’s definition, he mentioned loosely coupled. If we leverage an event-driven architecture or messaging, we can remove any rpc or in-process calls. Even within a single process, we can use messaging to loosely couple between logical boundaries.

We have the option of changing our physical aspect by deploying logical boundaries independently if we need to for various reasons (deployment cadence, etc).

Microservices to Monoliths

Logical boundaries aren’t physical boundaries. They don’t have to be one-to-one. You can choose to compose logical boundaries together into a variety of different physical boundaries. Don’t limit yourself by forcing this restriction. Could you need a logical boundary to be a physical boundary? Sure. Then do it when it’s required. You have a lot of options when you don’t force this one-to-one constraint.

Join!

Developer-level members of my Patreon or YouTube channel get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Aggregate (DDD) isn’t hierarchy & relationships

Derek Comartin Leave a comment

How do you design an aggregate in domain-driven design? An aggregate in a cluster of related objects and used to manage the complexity of business rules and data consistency. Designing aggregates often incorrectly because of the focus on the relationship between entities.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=djq0293b2bA

Relationships

I believe most developers designing aggregates are thinking about data, hierarchy & relationships. As an example, here’s an aggregate diagramed by its relationships. It has an entity that is the Aggregate Root which any consuming code calls, and the relationship from the root to to other entities and value objects.

To put more context to this, I’ll be using an example in this post that came from a member of my Patreon that was asked on my private Discord. The scenario was you have a Route and many different locations for a Route. You can think of this as a bus route, and all the locations are bus stops. A location (bus stop) has a single vending machine.

This vending machine would be emitting event and telemetry data, such as temperature and alarming when it is out of a certain range.

Aggregate

Besides just making the statement that someone can purchase something from a vending machine, I’ve only talked about the entities and their relationships. This is precisely where things go wrong, only thinking about entities, data, and their relationships.

Yes, an aggregate is a cluster of related objects, but those objects should contain behavior.

If I were to model the above in code, it might look like this.

The above code only deals with creating the relationships between entities, and the only real behavior is from the vending machine that has the ability to trigger an Alarm.

We don’t have any business logic or any behavior. All the code we have is to build up an object model hierarchy. While this is a simple example, the point I’m trying to get across is that we’re modeling the relationships. Sure you can have “business logic” that is based on the rules of these relationships, but that’s pretty much it.

Do you need an aggregate for that? Well depending on what type of database you’re using, you could be enforcing these relationship constraints at the database level. You could also just be using transaction scripts and data models. For more on transaction scripts, check out my post Domain Logic: Where does it go?

Invariants

Why do you want to use an aggregate? To enforce invariants and to be a consistency boundary. Hierarchy & relationships are an aspect but not only within that context. An aggregate is a cluster of related objects, but related how? Related based on the invariants you need to enforce.

So if invariants are a key part of an aggregate, when do you need to enforce them? Only when you’re making state changes. If data within an entity is not related to any invariants or must be consistent within an aggregate, then it doesn’t serve a purpose within the aggregate.

Taking this further leads you to realize that exposing data such as the AlarmCount and LastAlarmDate serves no value.

CQRS

Now you’ve landed on CQRS because the model for commands and the model for queries are different. The model for your commands to make state changes can be an aggregates, and your queries can choose a completely different path.

As mentioned many times, CQRS isn’t about different data stores, it’s about different paths for reads and writes. Your queries can use the same underlying database, but they might not use an aggregate, rather they could query the data store directly.

Behavior

If we stop thinking about relationships and start thinking about behaviors and the invariants we need to enforce, the consistency we need within an aggregate, our model would look different.

The Vending Machine is the one that had the ability to create an alarm. It can be on its own and when an alarm is triggered it could be publishing an event.

We could have another aggregate that contains the Route and the Location that could consume that event if it actually needs to know the alarm was triggered.

What it likely cares about the event is for query purposes to know if there are routes that have any locations with vending machines that are in an alarmed state.

Aggregates

It’s typical to think about aggregates based on their hierarchy & relationships between entities. I think this comes from the nature of thinking about data first and not behaviors. But the point of an aggregate is a consistency boundary and enforcing invariants. The data that drives those invariants is what you care about based on the behaviors being exposed. If you don’t know what the behaviors are, you can’t know what invariants to enforce. If you don’t know that, then you’re just building an object model hierarchy, not an aggregate. Don’t add unneeded complexity when you simply can use a data model and a transaction script.

Join!

Developer-level members of my Patreon or YouTube channel get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Using your Database as a Queue?

Derek Comartin Leave a comment

Do you need a message broker, or can you use your database as a queue? Well, as in most cases, it depends. Let me explain how it’s possible and some trade-offs and things to consider, such as volume, processing failures, and more.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=DOaDpHh1FsQ

Database as a Queue

There was a blog post about ditching RabbitMQ for Postgres as a queue. TLDR: They claimed they had issues with prefetching with RabbitMQ and it was causing issues because they have long-running jobs. When they process a message, it can takes hours to complete. Another note is they are doing manual acknowledgments once a message is processed.

Invisibility Timeout

Acknowledgments are important because of the invisibility timeout of a message. When working with a typical queue-based broker, you need to send an acknowledgment back to the broker that you’ve finished processing a message.

The reason for this is there is a length of time (timeout) before the broker deems the message processing as incomplete so it will then allow that message to be processed again. Imagine if your process fails or you have some exception that occurs. You’d want the broker to allow the message to be re-processed. If your processing of the message was successful, you return to the broker to tell it that it can remove the message from the queue.

This time period is called the invisibility timeout. The message is still on the queue but it’s invisible to any other consumers until it’s either acknowledged or the timeout occurs.

Now you may hear people say you shouldn’t use a message broker for long-running jobs because of this. You’d have to make your invisibility timeout longer than the total processing time it takes. Otherwise, you’d potentially re-process messages. This is often the reason why those claims are made. There are other reasons as well as throughput considerations and more, but generally having long-running has implications you need to be aware of.

Competing Consumers

I mentioned throughput because one way to increase throughput, when required is using the competing consumers pattern. This means having multiple instances of the same application processing messages from the same queue. It’s basically scaling out horizontally.

So if you have two instances running, they can both be consuming messages from the same queue. This allows you the not process messages concurrently, increasing throughput.

The issue they described in the blog post was that they were prefetching multiple messages together from RabbitMQ. This means that if there were two messages available, a single consumer would pre-fetch 2 messages, not one.

And since consuming a single message could take hours, this would then reduce throughput because the other consumers available don’t see the message. As well as, the invisibility timeout would kick in since the second message that was pre-fetch would often exceed the invisibility timeout because the first message took so long to process. But the consumer would still process it since it fetched it, but the since the invisibility timeout expired, the message would get processed again as well.

Because of all these troubles, rightfully or wrongfully, they decided to ditch RabbitMQ and use Postgres as their queue, which they were already in their system.

Table

So how would you implement a queue using Postgres or a similar relational database? First, it simply starts with a table where you’d be persisting your messages.

The table would have some Primary Key and the message’s data, which is likely serialized.

Any new message would be inserted into the table. As for consumers, you need to be blocking so you can lock any available records for processing. To do this in Postgres, you could use the FOR UPDATE SKIP LOCKED statement and a LIMIT 0,1. This will allow us to select a single record that hasn’t already been locked.

Then once the message has been processed, you’d delete the message from the queue and commit the transaction.

While you don’t have an “invisibility timeout” you do have the same type of mechanism with a database, often called a wait timeout. You can only have a transaction or connection open for so long before the timeout occurs and it automatically does a rollback of the transaction or closes the connection. You’re still in the same situation where you might still be processing the message and now will end up re-processing it.

Trade-offs

So what’s the issue with using your database as a queue rather than a queue-based message broker? As you can see, it’s feasible to use a database. In some situations, you maybe simplify your infrastructure by using a database you already have. However, as always, context is king!

You need to know what you’re trying to accomplish and the limitations. Once you get out of the simplest scenario that I showed above, you will end up implementing a lot of functionality on top of your database that out-of-the-box queue-based brokers support (dead letter queues as an example).

You also need to understand your use case. If you’re going to have a lot of consumers, they need to be polling (querying) the table periodically. This can add a lot of load but also can decrease total throughput and increase latency because you’re polling your database.

When you have failures, how are you handling that with your queue if its in your database? Are you implementing retries? Creating another table as a dead letter queue? Do you need to implement different queue tables for different levels of priority of messages? The list goes on and on that out of the box you can do with your typical queue based broker.

So what’s the issue with using your database as a queue? None until you actually need a message broker.

Join!

Developer-level members of my Patreon or YouTube channel get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design