Running Qwen 3 Coder for Free: The Future of AI is Here

0:00 Hey, welcome to the gray Cat. Have you ever seen the news about Alibaba's Qwen 3 Coder model and wondered if you could run it on your own machine? I certainly did. I have an M4 Max MacBook with 48 Gigs of shared RAM and 40 GPU cores, so a 32-billion model can fit into it. This led me down a rabbit hole. Could they actually replace the paid cloud AI assistance I use, like the one built in the Cursor with a powerful model running 100% locally. This video is the story of that experiment. I'm going to walk you through the entire process. The successes, the surprising roadblocks, and the ultimate solution I found. We are going to see just how practical it is to run the free, private, and powerful AI coding partner on your own hardware.

0:53 To run a massive 32-billion parameters model locally, you need the right software. My tool of choice for this experiment is LM Studio. Think of it as a desktop application that lets you easily discover, download and run open source large language models. It handles all the complicated setup in the background. The first step was to download the Gwen 3 Coder model inside the LM Studio and search for it, and I found a quantized version. Quantization is a process that shrinks the model, making it possible to run on consumer hardware. In my case, on a MacBook GPU. After downloading, I loaded it into the memory. The performance in the chat window was surprisingly good, hitting around 90-100 tokens per second.

1:39 I also took a moment to extend the context window to its maximum, which gives the model more room to understand larger files. The most important step in the LM Studio is starting the local API server. This creates a local web address that other applications can talk to, which is the key to connecting our local model to a code editor. My goal was to integrate this local model into Cursor, my primary code editor. I went straight to the settings, pasted the local server address from LM Studio localhost:1234/v1 and added a dummy API key, which is all LM Studio requires. But when I tried the chat, it immediately errored out. After some digging in the Cursor community forums, I found the problem:

2:26 For some reason, Cursor needs to verify the API server from an external address. It doesn't work with a simple localhost connection. The recommended workaround was to use a tool called Ngrok, which creates a public internet URL that tunnels back to your local machine. So I installed ngrok, configured it, and got a public URL for my local server. This time Cursor seemed to connect. Autocompletion started working, but I couldn't be sure if they were coming from my local Gwen Code model or Cursor’s default cloud models. The inline editor (Cmd+K) also worked, but it gave different strange responses each time. Things got weird when I gave it a custom instruction and it started repeating “I'm the gray cat” over and over.

3:11 The agent feature, which is supposed to perform complex tasks, failed completely. It was clear this setup was unstable and not a real solution. Just as I was about to give up, I remembered an extension I'd wanted to try called Cline. It's often advertised as a Cursor killer, but luckily it's also compatible with VS code, which Cursor is built on The installation and was incredibly smooth. I opened the extensions panel, installed Cline, and went to its settings. To my surprise, it had first class support for local models. I selected LM Studio from a dropdown list and it automatically filled the correct local server address and settings. Most importantly, it didn't require the ngrok public URL.

3:57 I was so relieved to finally shut down that extra process. With Cline and my Qwen 3 model properly connected, it was time for a real test. I opened up the code for our fun Slack bot, The Gray Cat, which can do a lot of “useful” things including image generation. The first thing I noticed was the speed. It felt much slower than the paid cloud tools like Claude or Cursor’s auto mode, but the quality of the answer was excellent. Well, excellent. Maybe an overstatement, but what I mean is that the results are decently good for a model that is running locally. I decided to give it a real task, modifying the bot to generate images of different sizes. It handled their requests almost perfectly, writing the necessary code.

4:43 After testing, it turned out that the model had suggested Dell E3 image sizes that are no longer supported. I pointed out the error and the model quickly corrected the code to use valid sizes. Then I asked it to add and that's the --square flag that would for say 1024x1024 image size, and it impleented that feature almost instantly. The intelligence and coding ability were genuinely impressive, perhaps even smarter than what I often get from the standard cloud models. So is running Qwen 3 Coder locally a viable replacement for paid services? It's a trade-off. The biggest win is ownership and privacy. Your code never leaves your machine. This experiment proved that a free local model

5:29 can be incredibly intelligent and useful for day-to-day coding tasks. However, the free part comes with the big asterisk – the hardware. I was able to run the 32-billion model on the system with 48 Gigs of RAM, but it was noticeably slower than the cloud. And then there's the dream of running even larger models. The Gwen team has released models as large as 480 billion parameters, but running that would likely require a top of the line machine, like a Mac Studio with immense amounts of RAM. Still, for the price of a one time hardware investment, you get a perpetually free and private coding assistant. The Cline extension made the setup painless without needing a login, which is a huge bonus.

6:14 Overall, I'm very happy with this experiment. We saw how to use LM Studio to download and serve a powerful open-source model like Qwen 3 Coder. We ran into frustrating limitations of trying to connect it directly to Cursor and discovered the ngrok workaround, but ultimately we found a much better solution with the Cline extension, which made the local setup simple and reliable. After the initial tests, I tried it in more battle-like conditions. The quality was pretty good, although the speed could be better. The VRAM usage also increased from 16 to over 23 gigs, so I had to close excessive windows of Cursor, Figma and Chrome to work comfortably. And yes, the fans of the MacBook were spinning rapidly.

7:01 I will definitely be using Qwen 3 Coder more in my daily workflow to see how it holds up. It's an amazing time for open-source AI with powerful tools becoming more accessible every day. The future of AI isn't just a corporate cloud. It's running right here on your own machines. Thanks for watching and subscribe Well.