Building a system that Junior Developers can be productive in

How do you get junior developers or someone new up and being productive within your system? Never mind domain knowledge. There is a lot of technical tribal knowledge about how you handle logging, persistence, validation, and various patterns and practices you apply within your specific systems architecture. Here’s how I think about it by creating silos.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

Coupling

In a simplistic view, you might think you have a web application or HTTP API with a controller that invokes a service layer that interacts with your database.

You could think of A and B as types, layers, components, whatever. The point is they illustrate the coupling between them.

In reality, however, it’s not a single usage. Typically, you will have a high degree of coupling to some type/layers, etc.

This is typical when you have some abstraction or layer around data access. Yes A is coupled to B, but at a very high degree, making “B” more fragile to change. If you introduce a bug or non-backward compatible change to “B”, it can affect many different usages from A.

Check out my post, Write Stable Code using Coupling Metrics, as I’m describing Afferent and Efferent coupling.

Another common situation is having a lot of indirection where a request goes through various layers and types.

Pile on top of that a high degree of coupling, and you’re in a system that will be hard to change.

If you’re a junior or someone new on a team, you’ll likely feel overwhelmed by the number of places you need to make changes to (layers) or will be so afraid of making a change because you don’t know the implications.

What often happens to combat this is developers will look at existing functionality and how it’s implemented. They then will re-create the same type of changes throughout various layers or piggyback off another feature, which ultimately conflates different features.

Silos

You want to be working in isolation. You want someone new on your team or junior developers to have the freedom to work in isolation without having to worry about every part of the system. To do that, you need to manage coupling. You want to be able to add functionality to your system without affecting any other part of your system. I like to think of this as working in silos.

A silo is independent and oftentimes will resemble other silos.

Their structure is the same. However, internally what they implement can be different. This makes them great for being a template to kick off a new feature. The template will show all the different patterns, practices, and tools you use to start quickly. Think of it as scaffolding a new feature.

This allows new people on your team to instead of piggybacking on existing features or having to change and touch code through different layers, they can create their changes in isolation. You’re minimizing coupling as much as you can.

This means that one silo won’t affect another. Because you aren’t coupled between them. The best way to illustrate this is with the publish-subscribe pattern.

In an e-commerce system, you publish an OrderPlaced event when an order is placed. You then can have 0 or many different consumers of that OrderPlaced event.

In the example above, there are two consumers. One is for sending out a confirmation email, and another consumer is used for sending HTTP requests to other systems as a webhooks system.

They are independent and have no coupling between them. This means we can add a new consumer at any time, completely independent of the other consumers.

This is the type of independence you’re after. However, not everything is going to be decoupled by the publish-subscribe pattern, but the intent is still the same: limit coupling.

Silos may share some common concerns. For example, they may share the same underlying data model, but they likely don’t all work on the same data. Or if they do, you’ll likely share a domain model in that case.

They aren’t share nothing. You will have s some cross-cutting concerns, but generally, those are more technical concerns rather than business concerns (excluding a domain model).

Because there’s limited coupling, this means you can add silos and remove silos without affecting others, similar to the illustrating with publish-subscriber consumers.

Extensible

Ultimately this makes your system extensible. You can leverage templates that illustrate the common structure but need the implementation details created. Every team has various coding standards, how they use certain libraries, frameworks, and tools. This allows you to define those and have the focus simply be on the implementation of the functionality required.

You might be thinking, “Are you just describing vertical slice architecture?”. Yes, to a degree. Vertical slices are a great way to think about silos. Meaning a feature or a feature set is a silo. It’s also helpful to think about CQRS in accomplishing this as well. However, the end goal is controlling coupling. You’re junior developers (and everyone on your team) will thank you.

Join!

Developer-level members of my Patreon or YouTube channel get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Speeding up Queries with Materialized Views

Many applications are more read-intensive than write-intensive. Meaning your application performs more reads than writes. However, we often persist data optimized for writes making querying and composing data intensive at runtime. What’s a solution? I will review different ways of creating materialized views and some trade-offs.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

Materialized Views

If you have a large system decomposed into many different boundaries, you’ll have data spread across many different databases. When you need to perform any query, often for a UI, you need to do some type of UI/View Composition by composing all the data from various sources.

This is one of the most common challenges when working in a large decomposed system. Check out my post on The Challenge of Microservices: UI Composition for various ways of handling this.

One issue with this composition is often performance because even if you’re querying each database concurrently there is often a lot of computation that has to occur to transform into the file result back to the client.

A solution to this is to do that computation ahead of time so that when a client makes a request, you already have all the data composed exactly as you need it or to query it.

This sounds like a cache, right? It is, but we’re not caching exactly the same structures as what we have in our database, rather, we’re caching the end result that’s already done all the composition and transformation in it’s final for we want to return to the client.

Meaning, we’re storing a data structure that’s optimized for specific queries or use cases.

As an example, let’s use the typical Sales Order in a relational database.

If we want to display a list of Sales Orders and the order total, we can query the order table, joining with the order line items and doing a Sum() of the Quantity and UnitPrice. No problem.

When an order is completed, its state is set and won’t change. Meaning we can do some pre-computation when an order is placed and summarize (denormalize) into a separate table that’s specific to this use case.

Rather than have to query and join the tables, we just query this individual table directly. Now this example is rather trivial, but you can see how doing pre-computation can be helpful. Another example is when the order is placed if we have to calculate the tax on the order. Instead of having to do that at run-time or via a query, we would just calculate it once and persist it.

You also don’t have to persist any of this data separately. You can often leverage the existing structures you have. Meaning some columns are persisted as calculated columns that are specific for queries.

CQRS

CQRS fits naturally because the point is to separate commands and queries as distinct operations. Queries can be optimized separately from commands.

When a command is executed that makes some type of state change, we can perform the write operation and also do that pre-computation and also persist the data that are specifically for queries.

That way when we execute a query it can use that specific tables/columns that are explicitly for queries and that are optimized for them.

One caveat to this is if you are doing this precomputation at the time of the write, you could be adding latency to your writes. However, in a lot of systems, you’re often more ready-heavy than write-heavy. But it’s worth thinking about if you need low-latency writes.

Another option is to remove that pre-computation at the time of the write and do it asynchronously.

When you perform your state change from a command to your database, you then publish an event that the query side will consume. It will handle then updating your query optimized read side.

Because this is asynchronous, your read database is eventually consistent. This has implications about not being able to immediately query the read database after you complete your command. Check out my post Eventual Consistency is a UX Nightmare for ways of handling this.

It’s worth noting, just like above, these don’t have to be different physical databases. They could be the same underlying database instance but separate tables and/or columns as I explained above.

If you’re familiar with Event Sourcing, then you actually have a leg up as your events in your Event Store are the point of truth where you can derive all other types of views of that data.

This is called a Projection and it allows you to read the events from your event stream and do all the same type of precomputation and persist it however you want for query purposes. You’ll also notice there is no message broker in the mix because you don’t need one. You’re event store often supports different subscriptions or, at worst, poll it to get the events you need to process to build your read model.

Lastly, a common scenario is for reporting purposes, where you need to compose data from various sources. This is where I find the value in Event Streams where various logical boundaries and published events that are persisted.

I’m not talking about a queue-based message broker, as you want these events to be re-consumed at any time. They don’t have a lifetime and can be persisted in the event stream log forever. This allows you to rebuild your materialized view at any time.

Materialized Views

You have a lot of options for creating materialized views that are optimized for queries. It’s interesting that many applications are more read-intensive than write-intensive, yet we typically persist data in a structure that’s optimized for writes. You can find the balance and optimize both where needed, especially in hot paths within your application that are query heavy.

Join!

Developer-level members of my Patreon or YouTube channel get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Implementing Undo functionality

You’re writing an email and click “Send”. Then immediately think, “oh no!” because you sent it to the wrong person. Or maybe you’re checking out of an e-commerce website, click “Place Order” then a couple of minutes go by, and you think, “I shouldn’t have ordered that,” and have buyer’s remorse. You want to cancel or undo the action you performed in both situations. Now put your developer hat on. How would you implement undo functionality in these types of cases?

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

Undo

A good example of undo functionality is canceling the sending of an email. If you’re familiar with Gmail, it has a feature to undo a recently sent email.

Gmail sending undo functionality

Once you send your message, you have the option to undo your sent message.

Undo Functionality

To illustrate this in our implementation, I also realized that Twitter could not undo sending a tweet. So I’m going to re-create a simple example of that.

I’ve created a Blazor server app allowing you to enter text and click send now. In the example below, I sent a message “this is a test message”.

On our backend, we are generating a Tweet message (command) that is being sent from our API to a message broker.

Send Command to Broker

We have a consumer that asynchronously handles the Tweet command and persists that to our database. It then also publishes a Tweeted event back to our broker.

Process command from broker

Then there is a consumer for the Tweeted event that handles that event from our message broker and, using SignalR, pushes that message back to the client.

Consume event to push to client UI update

Here’s what the code looks like when processing the Tweet command.

So that’s how it works currently. How do we implement an Undo? So that our “Tweet” can be canceled for a window of time after we send it.

Implementation

All the code for this demo is available to Developer-level members of my Patreon or YouTube channel.

To implement undo functionality, we can use delayed delivery from the Tweet command we initially sent to our message broker. This prevents the command from being consumed until the delay period has passed. It acts as a timeout. Once the timeout has passed, then a consumer will process our Tweet command.

During that timeout period, our API can expose another command to cancel the original Tweet command.

Delayed delivery has a bunch of different use cases. Check out my post Avoiding Batch Jobs by a message in the FUTURE for more examples.

We can now implement a TweetWithDelay command and a new UndoTweet command. And if a Tweet is undone, we will publish a TweetUndone event.

The above code uses an NServiceBus saga. When a TweetWithDelay is handled, it sets the Saga State to capture the incoming text/message of the Tweet. Delayed Delivery occurs when it calls RequesTimeout for 10 seconds. Once 10 seconds has passed, the Timeout method will be invoked, and we’ll create a regular Tweet command and send it. That will invoke the code above that we already had. If however the UndoTweet command is consumed, it will complete the Saga and publish our TweetUndone event. When a saga is completed, the Timeout won’t be executed.

Notice we didn’t have to change any of the existing code. This is all brand-new functionality.

In our front-end Blazor, we now can have a “Send with Delay” button that will send our new command to the broker.

What this looks like visually, I will send a tweet with “send now,” and then I will send another with “send with delay”.

When that occurs, we can show a cancel/X on the far right to allow the user to send the UndoTweet command. If you don’t within 10 seconds, as mentioned earlier, the timeout will occur, and the SendTweet will be sent, and we will remove the cancel/X on the far right.

Now if I send another tweet with a delay

However, with this tweet, I did click the undo button, which sent the UndoTweet command, which happened before the timeout occurred, so it completed our Saga, and the tweet was never processed, and we removed it from the UI via SignalR.

Client Side

You may wonder why not implement undo functionality exclusively on the front end? Why not just implement the timeout in the front end and make the API call after the timeout?

You could, but the downside is that no request would ever be sent if you close the browser before the timeout occurs. In the example of Gmail, do you want to click “Send” and then wait 30 seconds to close your browser/tab? Not really.

Join!

Developer-level members of my Patreon or YouTube channel get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design