Ollama and Using AI Locally — In Other Words, for Free

What does that mean? First of all, you’re not limited by any tokens and can use AI to your heart’s content :-) That sounds nice in writing, but what’s the reality? I tried it for you so you don’t have to. The Ollama software itself takes up a few gigabytes on disk. Then you have to choose a suitable AI model, which takes another 8–20 GB of disk space. That’s a lot. You also have the option to use models in the cloud, but for that you have to pay an Ollama subscription for $20, and I don’t really understand why I should pay Ollama instead of, say, Claude or ChatGPT directly.

So let’s get back to downloading an AI model to your computer. The first snag is already in choosing the model. The selection is broad, but in all tests (benchmarks) the models rank very low. So you’re not getting a full Einstein, more of a FrankEinstein. Especially if you’re used to some of the better models, for example Google’s very popular Gemma 4 won’t impress you much. You might think there’s a lot of buzz around DeepSeek... well, it does even worse in all tests.

And what about Gemma 4? Was it usable? Not really. I let Gemma look through one project, had it inspect the code, and gave it a task to add a small feature. It spent about 10 minutes getting familiar with the project and then took just as long to tell me it was done. But it hadn’t done anything at all. I tried giving it the task again. After 10 minutes it asked me what project I was talking about and had no idea what it was supposed to do. I switched to the standard Visual Studio Code model in the free version, which wasn’t local, and the task was done in no time.

And that’s not even mentioning the fact that when you give Gemma/Ollama a task, it uses as much of your computer’s resources as possible, so you can hardly do anything else. And I have a pretty powerful machine. If you don’t have at least an i7 and 16 GB of RAM, I wouldn’t even try it.

I decided to try just a regular chat, because Ollama also offers a UI environment similar to ChatGPT. Very stripped down, but the chat is there. I asked why it uses so many computer resources. At that moment everything stopped, all the computer fans spun up, the computer turned bright red (okay, that’s a bit of an exaggeration), and after 10 minutes of waiting I got a fairly extensive research result saying that it’s a good idea to choose a lightweight model.

For me, it’s unusable and trying it was a pretty big waste of time, but I’m glad I’ve been through it and don’t have to keep thinking about it. If you decide to give it a try, I wish you luck in reaching a different conclusion.