For Pistilli, choosing local models versus online chatbots has implications beyond privacy. “Technology means power,” she says. “And so who[ever] owns the technology also owns the ability.” States, organizations, and even individuals is perhaps motivated to disrupt the concentration of AI power within the hands of just a couple of corporations by running their very own local models.
Breaking away from the massive AI corporations also means having more control over your LLM experience. Online LLMs are continually shifting under users’ feet: Back in April, ChatGPT suddenly began sucking up to users excess of it had previously, and just last week Grok began calling itself MechaHitler on X.
Providers tweak their models with little warning, and while those tweaks might sometimes improve model performance, they also can cause undesirable behaviors. Local LLMs can have their quirks, but at the least they’re consistent. The one one who can change your local model is you.
After all, any model that may fit on a laptop computer goes to be less powerful than the premier online offerings from the foremost AI corporations. But there’s a profit to working with weaker models—they’ll inoculate you against the more pernicious limitations of their larger peers. Small models may, for instance, hallucinate more ceaselessly and more obviously than Claude, GPT, and Gemini, and seeing those hallucinations can enable you construct up an awareness of how and when the larger models may also lie.
“Running local models is definitely a very good exercise for developing that broader intuition for what this stuff can do,” Willison says.
How one can start
Local LLMs aren’t only for proficient coders. In the event you’re comfortable using your computer’s command-line interface, which permits you to browse files and run apps using text prompts, Ollama is an ideal option. When you’ve installed the software, you possibly can download and run any of the a whole bunch of models they provide with a single command.
In the event you don’t wish to touch anything that even looks like code, you would possibly go for LM Studio, a user-friendly app that takes a whole lot of the guesswork out of running local LLMs. You’ll be able to browse models from Hugging Face from right throughout the app, which provides plenty of data to enable you make the fitting selection. Some popular and widely used models are tagged as “Staff Picks,” and each model is labeled based on whether it will possibly be run entirely in your machine’s speedy GPU, must be shared between your GPU and slower CPU, or is just too big to suit onto your device in any respect. When you’ve chosen a model, you possibly can download it, load it up, and begin interacting with it using the app’s chat interface.
As you experiment with different models, you’ll begin to get a feel for what your machine can handle. In accordance with Willison, every billion model parameters require about one GB of RAM to run, and I discovered that approximation to be accurate: My very own 16 GB laptop managed to run Alibaba’s Qwen3 14B so long as I quit almost every other app. In the event you run into issues with speed or usability, you possibly can at all times go smaller—I got reasonable responses from Qwen3 8B as well.
