Brief Summary
This video compares event-driven architectures (EDA) powered by Apache Kafka with request-response architectures, highlighting six key factors: reactivity, coupling, consistency, historical state, architectural flexibility, and data access/reuse. EDA uses asynchronous event exchange for loosely coupled, reactive systems, while request-response involves direct, synchronous communication between services. EDA excels in flexibility, historical data access, and data reuse, making it suitable for complex data ecosystems. Request-response is better for real-time operations but can be challenging for data access across multiple services.
- Event-driven architectures (EDA) are loosely coupled and asynchronous, using event streams for communication.
- Request-response architectures involve direct, synchronous communication between services.
- Key factors for comparison include reactivity, coupling, consistency, historical state, architectural flexibility, and data access/reuse.
The video introduces event-driven architectures (EDA) powered by Apache Kafka, where systems communicate by exchanging events. It contrasts EDA with request-response architectures, such as REST and RPC, where services communicate directly. The comparison is based on six factors to highlight the strengths and weaknesses of each architecture. An example scenario involves a storefront where customers place orders via their cell phones, illustrating both request-response and event-driven approaches.
In an event-driven architecture, the storefront writes an order record to an "orders" topic upon receiving it. The Fulfillment service then consumes this record, stores it, and processes it for shipping. This demonstrates reactivity, where actions are triggered by events. In contrast, a request-response service requires the storefront's business logic to initiate the next step. The storefront either instructs the Fulfillment service to process the order or the Fulfillment service requests work from the storefront. This direct communication defines the reactive behavior in request-response systems.
Event-driven architectures are inherently asynchronous, decoupling services in space and time. By writing a record to a topic, the storefront is decoupled from the Fulfillment service. If the Fulfillment service is offline, it can resume processing orders when it comes back online. The consumer doesn't need to know the producer's identity, and they are only coupled via the Kafka topic. In request-response, services need to know each other's API details and locations at runtime, creating tight coupling. Both services must be online simultaneously, and if one goes offline, the service must retry, highlighting the synchronous nature and tight dependencies.
The asynchronous nature of event-driven architectures leads to eventual consistency. The Fulfillment service will eventually be consistent with the orders placed in the storefront, but the time this takes depends on event volume and consumer processing rate. Request-response services can achieve easier consistency, especially if all data is kept within a single service. However, if the Fulfillment service requests work from the storefront, the order data will be eventually consistent, which is typical when data is copied between services.
In an event-driven architecture, if a customer modifies an order, the storefront updates the data and writes a new record to the topic. The Fulfillment service consumes this update and adjusts accordingly. The Kafka topic maintains a complete history of state, allowing new services, like an inventory service, to consume data from any point in time. This provides flexibility for reprocessing data in case of failures. Request-response systems typically overwrite data in place. The storefront must communicate modifications to the Fulfillment service. Historical state is not commonly stored alongside current state in typical database patterns, though it is possible.
Event-driven architectures offer significant architectural flexibility. New services can be easily built, and data from across the company can be integrated. For example, incoming inventory data from warehouses can be added to the inventory service to create a complete picture. Order data can be reused for multiple use cases. In contrast, request-response systems rely on service APIs as building blocks. Creating a new inventory service requires communication with multiple existing services, such as Fulfillment and storefront, through their APIs. This can be challenging if service boundaries don't align well, requiring multiple service calls to complete tasks, especially when data is spread across systems.
Event-driven architectures are powerful for data reuse. Existing event streams can power various applications, including data lakes, warehouses, machine learning models, and generative AI vector databases. Tools like Kafka Connect facilitate creating connectors to ingest data, and native consumers offer flexibility. The work done to define and populate events into streams can be reused extensively. Request-response architectures primarily serve real-time operational data. Populating data lakes requires setting up ETL jobs, which can be challenging. The curated APIs for business use cases don't necessarily transfer over when trying to format data for analytics, machine learning, and generative AI.
Event-driven architectures, like those powered by Apache Kafka, provide a loosely coupled and asynchronous framework for building reactive applications. Event streams act as reusable building blocks for powering applications and data-intensive use cases. Request-response services provide business functions for other services to call, but uptime and liveness are critical, as failures can affect dependent services. While request-response services can power distributed applications, accessing important business data for various use cases remains complex and challenging.
Summarize any YouTube video in seconds
Get summaries like this one for any video, in your language, with the free Summ app.
Transcript
Full transcript of “Event-Driven Architecture (EDA) vs Request/Response (RR)”. Click a timestamp to open the video at that moment. Need another video? Use the YouTube transcript generator.
0:00 in this video we're going to look at event-driven architectures or Eda that powered by Apachi Kafka this is where systems communicate with one another through exchanging events we're going to compare and contrast Eda with request response architecture where the services communicate directly with one another examples of request response include rest as well as RPC or remote procedure calls we're going to look at six factors to see how these architectures vary and to give you a better idea of the strengths and weaknesses of each let's start with an example imagine you have a storefront that shows customers products and lets them place an order say via their cell phone on
0:45 this side we're going to look at it through request response and on this side we're going to look at it through event driven architectures so let's say our customer here places an order what happens next well the order is going to be written into this Forefront database and now for our first Factor we're going to look at reactivity event-driven architectures are reactive upon storing this order here the storefront writes a record representing that order to this orders topic the Fulfillment service in turn will consume This Record possibly stored in its own data store and then process it for fulfillment and shipping out that customer for a request response service
1:33 the business logic of the storefront service needs to drive the next step so the storefront will either tell the Fulfillment that there's an order to process it'll request that it processes it and get a response back indicating if it could or could not alternatively the Fulfillment service could ask the storefront is there any work that I need to do and could of course then be returned that work to do for itself in either case these two services are communicating directly with one another through their apis the second Factor we're going to look at is coupling both in space and in time edas are inherently asynchronous by writing this record into
2:19 this topic we've decoupled the storefront from the Fulfillment if the Fulfillment service is offline it can come back online and resume processing orders if the storefront Services offline fulfillment can keep processing all the orders that it's in its topic until it runs out the consumer doesn't know the identity of the producer service nor does it have any direct coupling on it they're only coupled via this CFA topic for request response the services these two Services here they need to know the specific details of each other's API as well as their location at runtime these services are tightly coupled to each other and furthermore both of these systems need to be online and available at the same
3:05 time if either one goes offline then the service will have to try again to recommunity this asynchronous temporal decoupling of event-driven architectures gives us a restriction known as eventual consistency so this fulfillment service will be eventually consistent with the orders that were placed in the storefront service but that might take some time how much time it takes will depend heavily on the volume of events and the consumer's processing rate in request response Services you
3:51 often have an easier time with consistency if you keep all of your data inside of a single service by definition it's consistent because there's only one copy however if this fulfillment service let's say were to request work to do from the storefront and the storefront were to respond with a list of orders to fulfill the order data that ends up in here will be eventually consistent with the data that's coming from the storefront this is true anytime you make a copy of data the fourth point we're going to look at is historical state so let's consider that this customer who placed an initial
4:37 order wants to modify it and they want to get the the all-in-one shipping everything shipped in one box so they say they log in they edit their order it hasn't shipped yet and they submit that into the storefront the storefront will update that data in place and then write a new record into this topic with that information available the Fulfillment service upon consuming that we'll see the modifications and if it's already shipped the package it may just have to ship out a second one if it hasn't shipped it yet it can amalgamate the two together and ensure that they get shipped out at the same time this
5:23 Kafka topic contains a complete history of State for as long as we want to keep it a new producer let's say an inventory service could spin up and consume this data these orders uh from the beginning of time it could consume it from the middle or it might just consume it from the latest it's entirely up to this service it can reread it as many times as it want in case there is a failure or a bug and you want to reprocess it you have that flexibility and that option for request response systems typically we overwrite this data in place so when we've modified that order we simply overwrote the old one the storefront service
6:09 though upon not noticing that it's overwriting existing orders would need to communicate to fulfillment and say listen this order has been modified and you need to account for it right now typically we don't store historical State alongside the current state in in a lot of common database patterns you can do it but it is fairly uncommon to do so this brings us to our fifth factor which we will call architectural flexibility event-driven architectures the streams make it very easy to build new Services as we just showed here but it also makes it easy to take service or to take data from across the company from different services so here's a different topic perhaps this is uh
6:56 incoming uh inventory stop that's being received at the warehouses being created by some producer somewhere else and we can add that into our inventory service to help get a complete picture not only of inventory going out but also inventory coming in to create a complete computation of the two we're able to reuse order data for multiple use cases in a request response system we are relying on the service apis these service apis are the building blocks for our applications so API blocked and here it would be data are the building
7:43 blocks so let's say we create a new inventory service over here in the Fulfillment world this new service also has its own API and we'll need to communicate with the Fulfillment service we'll need to communicate with the store front service perhaps and of course if we have this shipping service over here that's measuring uh tracking the incoming deliveries we would also need to communicate with this API here to to see hey what products do we have coming in so this new service is relying on all these other existing apis to solve its business functionality if your boundaries align well between your
8:30 services this can work but often the service boundaries don't necessarily align how you like and you might have to call multiple services to get your work done and those services in turn might have to call other services so a fairly trivial task might actually be pretty challenging to implement given how your service boundaries have evolved over time this this problem is the most noteworthy when the data your service needs to do its work is spread across multiple systems and you're forced to make many different service calls and this brings us to our final uh our final uh Point our final Factor data access and data reuse event-driven architectures are extremely powerful for
9:17 data reuse right we've already shown that you can power these different applications but you can also power things such as your data Lakes warehouses uh you can machine learning models you can pump this into your generative AI uh Vector databases all of these existing event streams are very easily uh routed into these uh data intensive applications you can use something like Kafka connect to easily create these connectors to get that data in there you can use native consumers you have a lot of flexibility but all of the work that you did already to get these events defined and populated into these
10:02 streams can be reused again and again and again in contrast request response architectures are primarily about serving realtime operational data so if you want to populate your own data Lakes over here let's say and you have a lot of these individual request response Services you're going to have to set up ETL jobs or extract transform and load and these can be their own challenge to set up and so a lot of the work that you've already done in curating these apis for your business use cases don't necessarily transfer over when you're trying to get your data into a format to
10:47 do all sorts of other important data work like your data Lake analytics machine learning and generative AI so let's wrap this up event driven architectures such as those powered by Apachi Kafka provide a Loosely coupled and asynchronous framework for building reactive applications these event streams act as reusable building blocks for powering applications as well as data intensive use cases and for connecting everything all together request response Services provide business functions for other services to call uptime and liveness are critical as a failure in one service can instantly affect all of the other dependent services while request response services are indeed capable of powering distributed
11:33 applications it Still Remains complex and challenging to Simply access important business data for your own use cases if you would like to learn more please check the video details for related content and if you've enjoyed this video uh please like And subscribe stay tuned for future episodes where we'll talk more about event streams event-driven architectures and how to build a healthy data ecosystem
