Category: Event Sourcing

Domain Events in Event Sourcing? Not Exactly!

Derek Comartin Leave a comment

What are domain events for? Domain Driven Design and Event Driven Architecture are nothing new, but they continue to become more popular. Unfortunately, some concepts and terms get a bit murky as they do. Specifically, the term domain events have caused a bit of confusion. I will clarify a misconception about the role and purpose of domain events and how that relates to Event Sourcing. Or does it?

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=dJBTNksQzys

Event Sourcing

As a primer on Event Sourcing, it’s a way of recording state. As a comparison, if you were recording the current state in a relational database or document store, you’d think of rows/tables or objects/collections.

As an example, let’s say we have products in a warehouse. We’re recording the SKU (productID), the quantity on hand, and when we last received and shipped the product from our warehouse.

Relational Database

With event sourcing, however, we’re recording events that represent the state transitions.

Event Sourcing Domain Events

Each product, represented as a row in our relational database table, has a stream of events in our event store.

For more basics on event sourcing, check out my post Event Sourcing Example & Explained in plain English

State

Event Sourcing is about using events as state. This doesn’t mean they are used as a form of communication.

Thankfully over the last several years, for the most part, the industry is accepting of not integrating at the database. Meaning instead of being able to interact with another service database directly, you must use a defined API. Gone are the days of calling another database directly.

Unfortunately, this idea seems lost when Event Sourcing—often thinking of events as am means of communication with Event-Driven Architecture. However, event sourcing is about state. Just because you have an event store, does not mean another service can reach out to your event store.

Domain Events

Domain events are inside events. They are internal to a logical boundary. Their purpose is for notification. To express that some business concept has occurred. There are situations where you may want to expose them to other boundaries. Often this is because they are well-established business concepts that aren’t going to change. In this case, exposing a domain event is reasonable. Treating a domain event also as an integration event. For more on this, check out my post Should you publish Domain Events or Integration Events?

So then, aren’t the events used in event-sourcing domain events? They are internal events. Yes, they can be, but they don’t have to be. Not every event persisted to your event store has to be a domain event. Likely they will, in most cases, represent business concepts, but as mentioned, they are also about state transitions.

You’re using events in event sourcing to reconstruct an internal state within an aggregate, so when you then perform an action (do this!), it can validate if it’s in a state to perform the action. And if valid, then appends the subsequent event that represents the state transition.

Events in event sourcing may be more fine grain than what is needed for communication purposes. Domain events may be less granular and represent the completion of a workflow.

As a typical example of a shopping cart and checkout process, let’s say we have an event stream that is:

  • ItemAddedToCart

  • ItemAddedToCart

  • ItemRemovedFromCart

  • ShippingInformationDefined

  • BillingInformationDefined

  • CheckOutCompleted

Are these events used to notify inside (and possibly) outside boundaries? We surely need them as our state transitions (event sourcing), but do we need them for communication (domain events)?

For communication, we probably want an OrderPlaced event to define the completion of our workflow. It likely will be a domain event (or integration event) to notify other boundaries. Still, all the others are probably too fine grain.

Domain events are about business concepts that the business cares about and understands. They are used for notifications/integrations within or outside a service boundary.

Events used for Event Sourcing are about state. They are capturing events to represent the transitions in state.

Domain events can be used for event sourcing, but not all events in event sourcing are domain events.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Event Sourcing Tips: Do’s and Don’ts

Derek Comartin Leave a comment

When people are getting into Event Sourcing, there are a few common questions that I often get or issues see people run into. CRUD Sourcing, Pre-mature optimization using Snapshots, and exposing your event streams for integration. Here are my top three Event Sourcing Tips to help you down the right path.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything in this post.

https://www.youtube.com/watch?v=SYsiIxJ-Nfw

CRUD Sourcing

My first event sourcing tip, which is probably the most common issue I see people run into when new to Event Sourcing is what is often called “CRUD Sourcing”. If you’re used to developing applications/systems in a Create-Read-Update-Delete style, then this means you’ll likely end up creating Events that are derived from Create, Update, Delete.

There’s a shift when moving from an application that simply maintains the current state via CRUD to having your point of truth be a stream of events. Often events are artifacts of the state change as well as the business event that caused that state change.

If you provide a UI that’s CRUD driven on Entities, you’ll end up with events that are derived from that. As an example, you’d start creating events such as ProductCreated, ProductUpdated, and ProductDeleted.

Still in line with this is if you have updates that are for properties on Entities, you’ll end up with events such as ProductQuantityUpdated or ProducePriceChanged.

In both cases, the events are simply representing state changes but not why the state changed.

What was the reason why a ProductUpdated occurred? It was updated, great, but why?

How about the ProductQuantityUpdated event, why did that change? Was it because there was an inventory adjustment? Did we receive more quantity of the product from the supplier? Why was it updated?

Being explicit about the events is important because we want to be driven by business concepts. To get out of CRUD we need to move to more of a Task Driven UI. This allows us to have the client/UI explicitly perform a Command/Task. For example, if the user performs an Inventory Adjustment as an explicit Command/Task, that’s a business concept. We will generate an InventoryAdjustment event.

Being explicit is important because you do not need to derive or guess based on the data of the event and why it occurred. You’ll have many more answers to questions when you look at an event stream when they are explicit. As an example, when’s the last time we did an inventory adjustment? When we do inventory adjustment, how many times are we decreasing the quantity on hand? You cannot answer these questions with a ProductUpdated or a ProductQuantityChanged.

Optimizations

Once people understand Event Sourcing and how it works, the most common question is:

That seems really inefficient to have to fetch all the events from a stream to build up current state! What happens if I have 1000’s events!

To understand how event sourcing works check out my post Event Sourcing Example & Explained in plain English.

As a quick primer, you have a stream of events for a unique aggregate, such as Product with a SKU of ABC123.

Event Stream

Anytime we want to perform a command which will append a new event to the stream, we’ll generally fetch all the events form the stream, build up the current state, then enforce any invariants for the command we want to perform.

In the stream above if we were keeping track of “quantity” as the current state, it would be 59.

So back to the common comment of “that’s really inefficient”, is a pretty valid concern. The answer to this problem is called Snapshots, but they are an optimization that you don’t necessarily need to apply right from the start or often.

In my experience, event streams are generally finite and have a life cycle with a beginning and end. There may be a long time duration for how long a stream is “active” but the events that are persisted are often limited.

If you have a lot of events and it’s taking a long time to rebuild the state, then creating snapshots can help. They are a way of creating a point-in-time representation of the state.

Event Stream Snapshot

After so many events append to a stream, you persist another event to a separate stream that is the current state, also recording at which version of the stream it represents. This way when you want to rebuild the current state, you first get the last snapshot and then get the events from the stream since that snapshot was created.

Now the question is, when do you create snapshots? If you think the event stream is going to contain a lot of events, how many for a given situation is a lot? Each different type of event stream is going to have different events which contain different data. There’s no magical number of events that is a threshold for creating a snapshot, it’s going to be use-case specific.

Event Sourcing Tip, don’t jump to snapshots immediately, look at how you’ve defined your streams and boundaries.

Communication & State

This last mess people get in with Event Sourcing is conflating events representing state as well as using events as a way to communicate with other service boundaries.

Your event store, the event streams, and the events within a stream represent the state.

Event Store for State

Often with Event Sourcing, you’ll create Projections as a way to represent the current state for Queries/UI/Reports. This way you can query a separate data store that’s pre-computed the current state. This means you don’t have to pull all the events to a stream to build the current state. It’s already pre-computed as events occur (usually asynchronously)

Event Store and DocumentDB for Projections

This means you’ll have two different databases. One for your event streams and one for projections. This is all within the same logical service boundary.

Now if you’re using an Event Store that supports subscriptions, this doesn’t mean that other logical service boundaries should directly access the Event Store.

A service can't access Event Store for integration

People often do this as a way of Pub/Sub which is used for communication. But there is a difference between Events used inside a logical boundary to represent state and events used to communicate with other services.

You wouldn’t have one service connect to another service’s relational database, would you? Then why would you access its Event Store?

Don't access other services DB directly

Domain Events used within a service boundary to represent the state are not integration events.

Domain Events: Inside Events

Integration Events: Outside Events

Domain Events aren't integration events

You want to define which events you want to publish to the outside world for integration. Domain Events and Integration events will be versioned entirely differently. The moment you expose a domain event to the outside world you potentially now have consumers that are going to rely on it. If you don’t publish your internal domain events, you can refactor and change events very differently.

If another service was interacting with your relational database, and you made a change to a column name, you’d break them. If you change an event that’s not backward compatible, you’re going to break consumers. Don’t expose internal domain events as integration events.

Event Sourcing do’s and don’ts

Hopefully, these three Event Sourcing Tips give some insights that can help you if you’re new to Event Sourcing or if you ever questioned how various aspects work.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design as well as access to source code for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

CQRS & Event Sourcing Code Walk-Through

Derek Comartin Leave a comment

Want to see an example of how CQRS & Event Sourcing work together? Here’s a code walk-through that illustrates sending commands to your domain that stores Events in an Event Store. The Events are then published to Consumers that updated Projections (read models) that are then consumed by Queries. This is the stereotypical set of patterns used when using CQRS and Event Sourcing together.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

https://www.youtube.com/watch?v=5aznkIEvkKc&list=PLThyvG1mlMzkRKJnhzvxtSAbY8oxENLUQ&index=7

CQRS & Event Sourcing

Because CQRS and Event Sourcing are so often talked about or illustrated together, you’ll end up seeing a diagram like the one below.

This is the stereotypical diagram to illustrate both concepts together. Unfortunately there often isn’t a distinction between what portion is CQRS and what portion is Event Sourcing. And as you’ll see later in this post, there’s another concept involved as well in this diagram as well.

CQRS

Command Query Responsibility Segregation (CQRS) is simply the separation of Writes (Commands) and Reads (Queries). In the diagram above, that’s illustrated by the horizontal paths on the top and bottom. I’ve talked about the simplicity of CQRS and the 3 Most Common Misconceptions.

Event Sourcing

Event Sourcing is about how you record state. It’s a different approach to persistence since most applications are built to only record the current state. Event sourcing is about using a series of events (facts) that represent all the state transitions that get you to the current state. If you want more of the basics, check out my post Event Sourcing Example & Explained in plain English.

Simplest Possible Thing

I’m going to use Greg Young’s Simplest Possible Thing sample that illustrates both CQRS, Event Sourcing, and Projections. This sample is rather old so I’ve upgraded it to .NET 6 and Razor pages.

The sample app is just showing an Inventory Item and has various commands that mutate state, queries that return current state, and event sourcing is used as a way to persist state.

Commands & Events

Commands are handled by Command Handlers. In this example, they are simply using a repository to get out the InventoryItem which is a domain object, and then invoke the proper method on the InventoryItem. The Repository.Save() will persist all the events (generated though ApplyChange() on the InventoryItem you will see below) to an in-memory event stream (collection).

The Inventory Item domain objects contain all the behavior for doing any state transitions.

The AggregateRoot base class contains the ApplyChange method, which stores the event being applied. It also calls the appropriate Apply() method on the InventoryItem. You will notice there are Apply() methods for only some of the events. This is because our InventoryItem only cares about maintaining its internal state (projection) that is required to perform logic.

What all this code is illustrating is the Command side of CQRS as well as Event Sourcing. Commands are explicit actions that we handle to perform some type of state change. For state changes, we’re using explicit events to capture what actually occurred from the result of a command.

CQRS & Event Sourcing Code Walk-Through

Projections

Before I illustrate the Query side of CQRS, first we’re going to build a Projection that acts as a read model representing the current state of the Events.

When events are saved by the Repository, it then dispatches the events to consumers. In this example, it is done in-memory however if you were using an actual Event Store, you’d likely be using a subscription model to have your projections run in isolation asynchronously.

Consumers handle the events to update the read model (projection). In this example, there are two different consumers for updating two different projections. One projection is for showing a list of Inventory Items, the other is for showing an individual Inventory Item.

This is just using a FakeDatabase that is really just an in-memory collection & dictionary.

CQRS & Event Sourcing Code Walk-Through

Query

Now that we have projections (read models) we can look at how the Query side of CQRS uses the projections.

The razor page is using the ReadModelFacade (and underlying FakeDatabase) that has the projection for our InventoryItem details.

CQRS & Event Sourcing Code Walk-Through

CQRS & Event Sourcing

Hopefully, this illustrated the differences between CQRS, Event Sourcing, and how they are used together while also using Projections. While this diagram is often used to describe CQRS, realize there are multiple aspects that are at play and not just CQRS.

Source Code

Developer-level members of my YouTube channel or Patreon get access to the full source for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Related Links

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design