Installing DeepSeek R1 on the Raspberry Pi is EASY

0:00 hello everybody in this video what we're going to be doing is setting up deep seek R1 on a Raspberry Pi now if you're watching this video you probably already know what deep seek is so I'm not going to dive into it whether that be the really good stuff or the not so good things about it what I will say is I would prefer not to use their web version because anything you type to it is saved and logged by servers controlled in China that being probably the worst thing about it probably the best thing about it to me is the fact it's open source we can download it self-host it it doesn't need to contact the Internet it's pretty cool now this guy does Focus here on the Raspberry Pi but this can be mimicked just about anywhere it's actually incredibly easy to just install on your desktop Windows

0:46 Linux or Mac OS operating system and chances are you will probably get better performance if you do it that way the main reason I'm doing this on a Raspberry Pi right now is mostly to kind of one the tutorial and two to demonstrate that it can actually run their lowest distilled model on a pi and get relatively decent performance out of it and by relatively decent I mean you'll get output it just might not be very fast now for this we are going to be using AMA this is the main download site for this tool this is what we're going to use to actually run the model you can see has a bunch of different options here including deep seek uh there's going to be a script to download this but before that if we go to deep seek R1 here you could see all the different models 7B is their default it's a pretty good one it works

1:32 perfectly fine even the 8B one works perfectly fine on this M4 Mac Mini that I'm using those models generally give you better results but they do require better hardware for the pi the 1.5b is what we are going to have to use and we have a command right here to go ahead and run it but first you're going to want to actually download and install deep seek this guide is assuming that you have a Raspberry Pi with some sort of Debian based operating system on it this Raspberry Pi is R my personal favorite which is the latest version of Ubuntu Server if you have Raspberry Pi Os or something like that it's going to work fine and putting whatever operating system you want on a pi is really easy just use the SD card and the pi imager but I'm going to go up in my logs CU I've already been testing and playing around with this after you run a pseudo apt update and app oh first update and

2:20 upgrade your system so that's apt update and apt upgrade all these commands and everything that I'm using are going to be down below this right here is the script it's pretty easy we want curl so you don't have that go ahead and install it and we're just pulling the installation script directly from AMA at install sh and just hit that to run it and it's going to download and install and when it's completely done you're going to see this right here it's going to give you a warning saying it cannot detect any Nidia or AMD gpus that is fine as it will run in CPU only mode and that is one of the key reasons why we need to use the smaller model here so now actually running the model is a pretty simple process all we need to do is grab this Command right here oh llama run deep seek R1 1.5b minimize that real quick and drop it in here if it's your first time

3:05 running it it's going to have to download the model it's pretty small so it should be pretty quick but if I just hit enter here it's going to load up the model and like that we are talking to deep seek so I can say something like hello it's going to think about it and it's going to give me a pretty generic uh response I could ask uh kind of a more complex question why should I use a GPU with R1 and there we go so it's going to do what you expect it to do it's going to go through its entire thought process I will note here if I jump into another terminal shell for this Raspberry Pi here you can see it is cooking it's really using every single drop of resources it has available to come up with this response but it it's doing pretty good it's not super quick

3:51 but it's going quick enough where I can't keep up with reading it now chances are you're not going to want to spend the majority of your time in CLI or the command prompt actually interacting with this thing so let's set up a nice little tool called open web UI which will give us something that looks more like the Deep seek or even the uh chat GPT interface and it does give us a little bit more options to do things like upload a PDF to actually Analyze That and to do that we are going to launch it in Docker so if you do for slby that's how you get out of the model and I do have some stuff here but we're going to ignore that for now I will not if you're not familiar with Docker we have a wonderful uh kind of beginner's guide on what it is it goes over all the basics that you're going to need at least to get started so if you have no experience with Docker I recommend you

4:37 check out that video now you could do this wherever you want on your system if I print my working directory you can see I'm just in my home directory here for this I'm just going to make a new directory and I'm going to call this open web UI so then within this open web UI directory I'm going to change directory into it and now here I'm going to create a new compose yaml file which is what we're going to use to spin up the docker containers so Nano is my text editor of choice compose do yaml and then just copy and paste this it will be in the description this is a really simple way to go ahead and run it basically all this does is we're going to launch the service open web goey it's going to pull the image from this Source we're giving it the container name of open web GUI or UI sorry the volumes it's going to point to whenever you see

5:22 a DOT slash that just means the current working directory so this is going to make a new data folder in that open webui folder I just created and this side is just going to point to where the actual app needs to look for this information ports the first one is the important one that's 3,000 that's the one I'm going to be accessing it from if you need to change it you can do not change this end that is um internal port for the actual container that we are going to be launching here extra hosts it's going to need the host Docker internal Gateway allow it to access that and then we will restart it and let stopped pretty simple so output that we didn't even need to change anything crl X but but one thing we are going to need to change is if we launch this now it's not going to be able to access our Ama or any of the models within it so we do

6:08 need to make a slight edit to our olama service paste that in when you installed olama using these scripts from their website it created something called a system CTL service it's just kind of a way to have various services and things running in the background and a lot of these Services have configuration files that we could go ahead and edit so you can see pseudo system CTL edit AMA dos service jump in there uh when you first look at this you're not going to have this here like I said I was playing around with it earlier popop it's going to look like this and you can see anything between here and the comment below will become contents so we are going to want to drop in this right here this is the service we're going to add the environmental variable AMA host at what is basically

6:55 Local Host so it's allowing Docker to see this service so we'll output that and then the first time you do that we're going to need to restart both the system CTL Damon as well as the olama service so this Command right here pseudo system CTL Damon reload and system CTL restart Alama hit enter and there we go now when we launch this uh Docker container it will be able to access it and make sure we're still in the same directory we are so to actually launch this container all we need to do is run a Docker compose up- D compose up launching the contents of the compos file and D detached meaning it's going to be running in the background for us so hit enter you can see it created the network basically so

7:41 it can see the AMA service and then it created the container open web goey or UI I keep saying guey so now within our web browser here I am going to click oh not right there I'm going to click right here and then paste in my local URL and then the port 3000 or not the the URL of the machine we just installed this on which is the pi hit enter oh let's give it access and try that again and there we go so you're going to get something that looks like this we're just going to click on get started and here you're going to fill out your name email and all that fun stuff this is all local so you can put whatever you want there we go so once you have everything filled in we can go ahead and create our admin account it's going to give us some new stuff I do recommend you read it and check it out we're going to be just scraping the service of this tool there's really a lot it can do but there

8:28 we go and you can see right here up in the top we have our models to select just for Giggles I did try the uh the 7B model wouldn't recommend it so I'm going to select the model that we downloaded the 1.5b and then we can interact with it right here just as we were in the terminal for example if I just use one of these maybe show me a code snippet of a website sticky header in both CSS and JavaScript so if I hit send it's going to think about it and actually if I go ahead and open up Tabby here you could see that it is definitely working uh the response might be a little slow we could still see the thinking process here if we click this so it's thinking and thinking through everything if we don't want to see that you could hide it one kind of cool thing about this uh Docker container that we spun up here and it's

9:14 been about a minute it's still going but you can see it is outputting quite a bit here to uh accomplish that goal and you can kind of see the speed of it so it's not moving too fast and just just for comparison's sake here uh I'm going to copy this and show you the speed locally on uh on this Mac so if I bring this up and let's open a new tab here we could do a deep seek R1 which is even a better model this is the 7B model so we're going to run it locally here real quick and then paste in this message hit enter give it a second to kind of start and you can see how much quicker it is even with the better model on the uh M4 Mac Mini here versus it still thinking about it

9:59 yeah versus this right here and again this is a better model so we're probably going to get a slightly better result in this uh HTML file it's going to create here so sticky header Styles this is Javascript CSS styling so yeah it's definitely a little bit different and yeah here here it goes this probably took it or once it tells me what's going on here probably took it about 45 seconds to render this out versus like three or four minutes with the pi on the smaller model so I cleared out that chat cuz it was taking way too long and what what we're going to do now is hopefully something that doesn't take even longer is upload a PDF and ask it to kind of analyze it for us so I'm going to upload a random file and I found this right here it's the only PDF I have on my system so I'm probably going to have to like blur out most of the output but

10:46 let's say can you summarize select our model and send the message 1054 let's see how long it actually takes yeah I'm going have to blur all this out I'm not going to show you but it worked pretty good but 3 minutes it took it actual use practicality is there was an a payment that was made provided by this company to this company for the product being described as uh the product with a total price of the price so it works pretty good again like I said in the beginning of the video doing this on better Hardware is going to give you better results and it's going to be much quicker but if you have a Raspberry Pi that you just want to throw this up on have a nice little chat UI you can access from anywhere on your home network this is great yeah that's how you do it again everything I mentioned

11:32 will be linked down below and with all that I do hope you have an absolutely beautiful day and goodbye