Brief Summary
This video concludes the systems design concepts series by discussing the trade-offs between monoliths and microservices, and introduces Docker and Kubernetes as tools to manage the complexities of microservices. It emphasises that microservices are essential for large companies needing independent scaling and faster deployments, but they introduce complexity. Docker helps standardise dependencies through containerisation, while Kubernetes orchestrates these containers, managing deployments and ensuring high availability. The video concludes with the advice to avoid premature optimisation and to choose the architecture that best fits the project's current needs.
- Monoliths offer simplicity for smaller applications but lack independent scaling.
- Microservices enable independent scaling and development but increase complexity.
- Docker containerises applications, standardising dependencies and simplifying deployment.
- Kubernetes orchestrates containers, managing deployments, scaling, and ensuring high availability.
The video introduces the final systems design concepts discussion, focusing on monoliths versus microservices. The creator expresses excitement about transitioning to interview question-based videos, which was the primary motivation for redoing the series. Despite feeling unwell, the creator aims to deliver a concise explanation of the concepts.
The video explains that a monolith is a single, massive binary encompassing all application functionalities, deployed and scaled as a unit. Using Instagram as an example, a monolith would include features like file uploading, user following, ad serving, and third-party integrations within one application. The primary advantage of a monolith is its simplicity, with a single repository simplifying dependency management. This approach is suitable for smaller applications where the build is not required for significant scale, reducing the overhead of managing multiple services.
The video outlines the cons of using a monolithic architecture, which have led to the adoption of microservices in large-scale applications. A single bug can bring down the entire application due to the interconnected nature of the codebase. Scaling is not independent, meaning resources cannot be efficiently allocated to specific features that require more capacity. For instance, if photo uploads are more frequent than user follows, a monolith cannot dedicate more resources solely to uploading, as each instance handles all functionalities.
The video describes microservices as an architecture that allows for more efficient resource allocation in large-scale applications. Using Instagram as an example, the application is divided into separate services such as uploading, following, serving ads, connecting to third parties, and handling metrics. This decoupling enables independent scaling, allowing more resources to be allocated to services with higher demand. Microservices can be developed in separate repositories, preventing global bugs and enabling teams to work independently using different technologies, as long as they agree on a common interface or API.
The video highlights the challenges associated with microservices, including the increased complexity of managing multiple services. Each service can fail independently, requiring monitoring of each component. Deploying these services to different locations with varying dependencies and resource requirements can become complicated.
The video introduces Docker as a technology that helps manage the complexities of microservices by standardising how dependencies are installed and development environments are set up. Docker uses containers, which encapsulate all the necessary dependencies, allowing applications to run consistently across different environments. This abstraction eliminates the need to manually install dependencies on each server, saving time and effort. Unlike virtual machines, containers can scale resources up or down as needed, optimising resource usage and reducing costs.
The video explains that Kubernetes is used in conjunction with Docker to manage and orchestrate containers. Kubernetes addresses the challenges of deploying and managing containers across multiple hosts, ensuring applications run as expected. It allows users to specify a desired state, and Kubernetes manages the deployment, scaling, and recovery of containers to meet that state. Kubernetes monitors the health of containers and restarts them if they fail, and it can also redistribute containers to different hosts if a host goes down, ensuring high availability.
The video concludes by summarising that microservices are essential for large companies due to their scalability and independent deployment capabilities. However, the complexity introduced by microservices necessitates tools like Docker and Kubernetes. Docker is a containerisation tool, while Kubernetes is an orchestration framework. The video advises against premature optimisation, suggesting that a monolith may be more suitable for smaller projects.
Summarize any YouTube video in seconds
Get summaries like this one for any video, in your language, with the free Summ app.
Transcript
Full transcript of “Monolith Vs. Microservices + Docker + Kubernetes | Systems Design Interview with Ex-Google SWE”. Click a timestamp to open the video at that moment. Need another video? Use the YouTube transcript generator.
0:00 hey guys and welcome back to the channel uh I think we finally made it to our last systems Design Concepts video which is freaking awesome because I've really been looking forward to getting to the next round of videos which is going to be all the interview questions it's kind of the main reason that I wanted to redo everything cuz I just rambled a bunch of those and never really Drew anything out cuz I was lazy so uh yeah I'm going to get to those but unfortunately today I am super hung over and uh there's a decent shot I just throw up while recording this uh so as a result I'm going to try and get through it as quickly as possible so let's go ahead and get to the iPad all righty guys so today we're going to be talking about monoliths vers microservices and I'm going to begin by talking about the monolith and I have this handy little note here CU I know many of you might just think about my junk when I say the word monolith but no that's not in fact
0:47 what I'm talking about what I am talking about is effectively the idea of having one kind of you know massive binary that incorporates all of the functionality of your application and basically as a result of having just pretty much one piece of code that's going to run everything it basically means that you're going to deploy it together you're going to scale it together etc etc etc so if we were you know to use Instagram as an example you know we might have literally just one binary and within that we would have the ability to upload files to our service we'd have the ability to follow other users uh you know basically serve people ads connect to third parties deal with metrics and so of course there are going to be some pros and cons when designing an application this way uh the main Pros is probably just going to be Simplicity if you just have one repository uh you can
1:33 keep all of your dependencies in check and this is probably less relevant for something like a systems design interview but it is an objective upside of something like a monor repo or a monolith in general um and then additionally of course you know if you just have a smaller application you don't need a build for that type of scale why would you split everything out into different Services you're going to have a lot less to keep track of if you just have you know one thing that you're managing and one thing that actually needs to be kept working so of course there are going to be Cons with this approach as well well which is the reason that people have begun using microservices at least in applications that require tons of scaling so one is that uh in the case that everything is in the same repository one bug can really just take everything down you know you screw up a dependency chain and all of a sudden everything is screwed up and you've got a s 2 on your hands the
2:18 other is that you basically can't scale things out independently so what if for whatever reason people were uploading photos a million times more than they were following one another in the case of Instagram that may actually be true you would probably want a lot more servers devoted to exclusively uploading things than you would following things but in the current monolith State basically every single binary that you deploy has the ability to do both and you know you just don't really need that so let's now go ahead and talk about microservices because this is going to allow us to be more efficient with our resources when we have an application at huge scale so uh in this case you know perhaps I'm talking about your private parts micro but let's go ahead and and talk about them so in the case of Instagram I've kind of broken it down into those five different Services you
3:05 know whatever you want to call them theoretically and basically the gist is you know if certain things are going to require more load than others you could basically go and devote more nodes in your distributed cluster to actually running that service specifically the issue with the monolith is yeah in theory you can have just a ton of deployments of that monolith but at the end of the day all of those services are going to be stealing resources from what you actually want to be devoting that particular deployment to and as a result decoupling these things within microservices might make life a little bit easier so the first thing is that it doesn't have to be the case that they're separate repositories but if they are developed in separate repositories microservices will allow you to avoid basically like Global bugs and additionally teams can kind of develop independently from one another without any considerations into you know like hey we have to use this dependency or we
3:51 have to use this technology you're basically decoupled and you can just agree on one common interface uh with which you know to have your microservices connect basically just creating apis that's what they're for um and of course like I mentioned each service can actually scale independently which can be more cost efficient and ultimately allow you to basically get more usage out of your servers which is pretty important when you have tons of users on the cons side though every single piece can fail because now we have to keep track of a bunch more stuff you know for example our ad service can just have a bug it can go down entirely and now all of a sudden we have to monitor you know five different Services instead of one and be like okay is this okay is this okay is this okay and then of course when we will have to deploy all of these different services to different places and all of those uh
4:37 microservices need different dependencies and different resources that can get complicated so to close out this video I'm going to talk about two different types of technologies that are going to greatly help us out with microservices and actually dealing with the complications that they introduce the first is going to be Docker so if you haven't seen it before the logo looks like a little bit of a whale but basically the gist of Docker is this every single one of our microservices assuming they have different Technologies is going to have unique requirements you know maybe one of them is in Python maybe one of them is in Java maybe even within python they just have different dependencies that they want and the gist is if I were to just deploy that to like an actual server computer then I would have to jump to that server download all the dependencies and that's really complicated and I would have to do it for every single one of them instead what Docker does is it basically allows
5:24 us to standardize how we are going to install those dependencies and get our development environment going and the way that it does this is through this usage of a container so think of a container as this interface right and the reason I've drawn this out as a square is because let's imagine every single computer knows how to run the square nice and easily but then within the square you know you can have anything going on you might have all different types of dependencies in here you might have different operating systems even like different versions of Linux that you're actually running your program on and so Docker basically abstracts away from the actual host that the container is running on uh the dependencies and just allows it to run the the container and so you don't have to go and install the dependencies manually on each host which saves a ton of time additionally another really nice thing about containers is that as opposed to something like a virtual
6:10 machine you don't necessarily have to like specify how many resources you need in advance you know how much memory how much CPU usage anything like that they can actually just scale up and down as more resources are needed and so of course in a world where we want to cut as many costs as possible when running our applications containers are hugely useful this is going to be critical when you've got many of them on the same host which is very common when you are using something like microservices that being said great now we have all these Docker containers but what happens if uh we need to choose you know for a new Docker container which host to actually put it on additionally what happens if a given Docker container crashes or stops running or a host goes down how can we make sure that all of our programs are running as expected this is where something called kubernetes comes in so
6:56 I think a lot of people when they hear about Docker and kubernetes think that they're actually the same thing or you know there's something that you use one or you use the other but in reality you actually tend to use them in conjunction with one another so kubernetes is pretty simple we have all these containers and like I mentioned we don't know where to actually put them you know all of this is commodity Hardware so we don't really care which host it's going on and additionally we want something to actually manage failures for us because otherwise we have to monitor everything ourselves and that can be really tough and I had the Eraser out there like an idiot so basically what kubernetes allows us to do is specify this concept of a desired State and kubernetes via the control plane over here and etcd which is basically a version of raft right this is a distributed consensus uh
7:41 coordination service I don't think it actually uses raft under the hood it probably uses its own algorithm but uh the point is it's like zookeeper is keeping track of basically what systems are up what systems are down and as a result it can then send that information over to the kuet on every single host host one one could be a VM or it could be a physical host host 2 and then basically KU uh kuet will go spin up all of these pods and each of these pods is effectively just running a Docker container if a pod goes down the kuet can notice it and restart it and similarly if a host goes down the control plane can notice it and put that stuff over on a different host and so by doing this kubernetes makes things a lot easier to run and manage so guys let's go ahead and quickly conclude because we've gone through all of these
8:26 great systems Design Concepts and I would just love to be able to wrap it up so I can go throw up first off basically the gist is if you're a massive company if you're a Google if you're a Facebook or Microsoft or anything like that microservices are basically inevitable you can't really develop every single thing in the same large binary it's going to take a forever to compile forever to deploy and it's going to be impossible to scale independently with different pieces that being said because microservices introduce all of this additional complexity when it comes to actually keeping track of and managing your services Docker and kubernetes or at least tools similar to those become very very important Docker like I mentioned is known as a containerization tool and kubernetes is going to be more of like an orchestration framework at least as far
9:12 as I'm aware correct me if I'm wrong and so to finish off this series I would love to give two quotes and then we will go ahead and talk about how they're relevant here the first is a serious one and a very good one which is the premature optimization is the root of all evil as I spell optimization wrong bad that's why Donald n he's a genius and uh is right about that if you don't really need microservices you don't have to actually go and build them out sometimes a monolith is going to make your life easier if you're just building for a personal project and you anticipate you're going to have a 100 users most you probably don't need a bunch of disjoint Services just write it all on the same app and of course for my ex-girlfriend yours is perfect the big ones hurt anyways sometimes you don't always want to design for scale if you don't need it and uh this was a convoluted way of me trying to make a joke about it but generally guys what I
9:59 want to say most is thank you all for watching this series I hope it's been helpful I continue to get really really great feedback from all of you and more importantly you guys continue to tell me that it's really benefiting you tons of you have gotten job offers and I plan on making an appreciation post for all of you at some point that I can keep track of anyways guys have a great rest of your weekend and I'll see you in the next one
