Short answer: Ollama is a free, MIT-licensed tool that downloads open AI models and runs them on your own computer, with a terminal chat and a local HTTP API on port 11434. You install it, type ollama run plus a model name, and chat offline once the download finishes. The model you can run depends mostly on your memory: a 1 to 2 GB model is comfortable on 8 GB of RAM, while 20 GB models need a 32 GB machine or a big GPU.

One honest note before we start: Ollama is not installed on our test Mac, by policy. Every Ollama command below is copied from the official docs at docs.ollama.com and the GitHub repository, and we say plainly which parts we could run ourselves.

What Ollama is, in plain terms

A large language model (LLM) is the AI part of a chatbot: a big file of numbers that predicts the next word. An open model is one whose file you can download, such as Google's Gemma, Alibaba's Qwen or OpenAI's gpt-oss. Running one yourself used to mean compiling C++ code and hunting for the right file format.

Ollama packages that work into one program. It runs a small server (a background process that answers requests) on your machine, pulls models from its own library at ollama.com/library, and gives you a command line plus an API. The README lists llama.cpp as its supported backend. The project had 186,906 GitHub stars when we checked, and the latest stable release is v0.40.16 (9 November 2026), with v0.40.17-rc0 out as a pre-release.

Recent versions also offer optional cloud models and sign-in. You do not need an account for local models: the quickstart's API-key step says "For local models, skip this step."

What you need

  • macOS: Sonoma (v14) or newer. Apple M-series chips get CPU and GPU support; Intel Macs run on the CPU only.
  • Windows: Windows 10 22H2 or newer, Home or Pro. NVIDIA cards need driver 551.61 or newer. The install needs at least 4 GB of disk space before any models.
  • Linux: x86_64 or ARM64. NVIDIA GPUs need compute capability 5.0 or higher and driver 550 or newer (570 or newer for compute capability 5.0 to 6.2); AMD GPUs use ROCm v7.
  • Memory and disk: the quickstart's first model, gemma4:e2b, is "about 7.2 GB" and Ollama recommends "8 GB of available VRAM, or unified memory on a Mac." Models in general "can be tens to hundreds of GB in size."

VRAM is the memory on a graphics card. On Apple Silicon the CPU and GPU share one pool, called unified memory.

Step-by-step setup

Step 1: Install Ollama

Each platform has a one-line installer and a manual download; pick one.

macOS and Linux (official script):

curl -fsSL https://ollama.com/install.sh | sh

Windows (PowerShell):

irm https://ollama.com/install.ps1 | iex

These pipe a script from ollama.com straight into your shell. If you would rather read it first, open the URL in a browser, or use the manual downloads: Ollama.dmg for macOS (drag the app into Applications) and OllamaSetup.exe for Windows, which installs into your home folder without administrator rights. For a manual Linux install on x86_64, the docs give:

curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst \
    | sudo tar x -C /usr
ollama serve

On ARM64 swap in ollama-linux-arm64.tar.zst. With an AMD GPU, also extract ollama-linux-amd64-rocm.tar.zst the same way. The Linux page also has a systemd service file if you want Ollama to start at boot. You should end up with an ollama command in your terminal.

Step 2: Confirm the server is running

The command line talks to the background server, so the server has to be up first. On macOS and Windows, opening the app starts it. On Linux, the quickstart says to start it with ollama serve "if it is not already running" (the Linux page also shows how to run it as a systemd service). Then, in another terminal:

ollama -v

You should see a version number. Ours would read 0.40.16 if it were installed; yours depends on when you install.

Step 3: Run your first model

This one command downloads the model on first use and opens a chat:

ollama run gemma4:e2b

The quickstart says: "Ollama downloads the model and starts a chat on your computer. Type /bye to leave." The part after the colon is a tag, which picks a size or build. Leave it off and you get latest; on the Gemma 4 page, gemma4:latest carries the same ID (009acb0d7fe1) as gemma4:e4b. Typing plain ollama instead opens a setup menu, where you choose a local model or sign in for cloud ones.

Step 4: Choose a model that fits your RAM

The model has to fit in memory with room to spare for your operating system and for the context window (how much text the model can see at once, measured in tokens). The download sizes below come from the Ollama library pages on 14 November 2026; which memory row each model sits in is our own rule of thumb, not an Ollama recommendation. Some tags show a range. For gemma4:e2b, the 4.6 GB end matches the q4_K_M build listed on the same page and the 7.5 GB end matches the MLX build (Apple's machine-learning format), so plan for the higher number.

Memory you have (our rule of thumb)Models that fit (download size)
8 GBllama3.2:1b (1.3 GB), qwen3.5:0.8b (1.2 to 1.3 GB), deepseek-r1:1.5b (1.1 GB), llama3.2:3b (2.0 GB), qwen3.5:4b (3.3 to 4.0 GB)
16 GBgemma4:e2b (4.6 to 7.5 GB), qwen3.5:9b (6.6 to 7.6 GB), gemma4:e4b (6.6 to 9.5 GB), gemma4:12b (7.7 to 8.0 GB), deepseek-r1:14b (9.0 GB), phi4 (9.1 GB)
32 GBgpt-oss:20b (14 GB), gemma4:26b (16 to 19 GB), qwen3.5:27b (17 to 20 GB), gemma4:31b (19 to 20 GB), deepseek-r1:32b (20 GB)
64 GBdeepseek-r1:70b (43 GB)
96 to 128 GBgpt-oss:120b (65 GB), qwen3.5:122b (81 GB)

The pairings are our rule of thumb, not an Ollama spec: aim for a model file well under your total memory, because the context window and your other apps need space too. Context also grows with GPU memory. Ollama's docs say it defaults to a 4k context below 24 GiB of VRAM, 32k between 24 and 48 GiB, and 256k at 48 GiB or more. Ollama itself recommends 8 GB of available VRAM or unified memory for gemma4:e2b; an 8 GB laptop has less than that free once the system is running, which is why we list it in the 16 GB row. Start one row below what you think you can handle, then move up.

Step 5: Call the local API

Any program on your machine can use the model over HTTP, which is how editors, chat front-ends and your own scripts connect. The README's example:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'

Change gemma4 to the tag you pulled, for example gemma4:e2b. With "stream": false you get one JSON object back, with the answer in message.content. Ollama also serves OpenAI-style endpoints such as /v1/chat/completions, so many existing tools work by changing the base URL. From Python, the official client is one install away (use a virtual environment):

pip install ollama
from ollama import chat

response = chat(model='gemma4:e2b', messages=[
    {'role': 'user', 'content': 'Say hello in one sentence.'},
])
print(response.message.content)

By default the server listens only on 127.0.0.1, so other devices on your network cannot reach it unless you change OLLAMA_HOST.

Step 6: Know where models are stored

Models are big, so it helps to know where your disk space goes. From the official FAQ:

  • macOS: ~/.ollama/models
  • Linux: /usr/share/ollama/.ollama/models
  • Windows: C:\Users\%username%\.ollama\models

To use another drive, set the OLLAMA_MODELS environment variable and restart Ollama. On Linux with the standard installer, the ollama user needs read and write access: sudo chown -R ollama:ollama <directory>.

Step 7: Remove models you no longer need

Deleting a model is the quickest way to reclaim tens of gigabytes:

ollama ls
ollama rm gemma4:e2b

ollama ls lists what you have downloaded and ollama rm deletes one. To free memory without deleting anything, ollama stop gemma4:e2b unloads a running model; otherwise models unload after 5 minutes idle. To remove Ollama itself, use Add or remove programs on Windows. The macOS and Linux pages list every file to delete; on macOS that includes ~/.ollama, which holds your models.

Check it worked

curl http://localhost:11434/api/version
ollama ps

The first returns JSON like {"version":"0.40.16"} with your installed version. The second lists loaded models. Its PROCESSOR column says 100% GPU, 100% CPU, or a split such as 48%/52% CPU/GPU; a split means the model did not fit in GPU memory, so a smaller tag will run faster.

Troubleshooting

Error: could not connect to ollama server, run 'ollama serve' to start it
The text comes from Ollama's source (cmd/start_darwin.go and cmd/start_default.go). The command line is installed but the server is not running. Open the app, run ollama serve, or on Linux run sudo systemctl start ollama.

Error: listen tcp 127.0.0.1:11434: bind: address already in use
Reported in GitHub issue #14511. A server is already running, usually the desktop app or the systemd service, so you do not need ollama serve. To restart with new settings, quit the app (or sudo systemctl restart ollama) instead of starting a second copy.

model requires more system memory (18.3 GiB) than is available (12.6 GiB)
Quoted from issue #15704, an open report of this message appearing wrongly inside LXC containers; your numbers will differ, and on a normal install it means what it says. Only Ollama v0.24 and earlier print it (on Linux and Windows); v0.30 and later report the same problem as llama-server reported out-of-memory during startup. The model plus its context does not fit. Pick a smaller tag from the table, close memory-heavy apps, or lower the context length.

{"error":"model 'gemma4:e2b' not found"} with HTTP 404
This is the API's reply (from server/routes.go) when you request a model you have not pulled. Run ollama pull gemma4:e2b and make sure the name in your request matches ollama ls exactly.

curl: (7) Failed to connect to localhost port 11434 after 0 ms: Couldn't connect to server
We hit this on our test Mac, where Ollama is not installed. The Python client gives its own version: ConnectionError: Failed to connect to Ollama. Please check that Ollama is downloaded, running and accessible. https://ollama.com/download. Both mean nothing is listening on port 11434: start the server as in Step 2.

Garbled text like ←[?25h←[?25l on Windows
The troubleshooting docs blame a terminal bug in older Windows 10 builds such as 21H1, sometimes with The parameter is incorrect. The docs' fix is to update to Windows 10 22H1 or newer.

What to do next

FAQ

Is Ollama free and open source?

Yes. The code is MIT-licensed and local use costs nothing. Cloud models are a separate, optional service that needs an account.

Does Ollama send my prompts anywhere?

Not for local models. The FAQ says: "Ollama runs locally. We don't see your prompts or data when you run locally." To switch cloud features off completely, set OLLAMA_NO_CLOUD=1 or put "disable_ollama_cloud": true in ~/.ollama/server.json, then restart.

Can I run Ollama without a GPU?

Yes. Ollama falls back to the CPU and system RAM, and the quickstart warns that "responses may be slower." Intel Macs always run CPU-only. Smaller models from the first rows of the table are the realistic choice without a GPU.

How do I let another computer on my network use it?

Set OLLAMA_HOST to 0.0.0.0:11434 using your platform's method from the FAQ, then restart. Only do this on a network you trust: local requests need no API key, so anyone who can reach the port can use your models.

How this guide was tested

Checked on 14 November 2026 on macOS 26.5 (Apple M4). Ollama is not installed on this machine and was not installed for this guide, so the install, ollama run, ollama rm, model storage and API steps were checked against the official docs only: docs.ollama.com, the docs folder of the ollama/ollama repository, and the Ollama library pages. Version and star data came from gh api repos/ollama/ollama and its releases endpoint. What we ran ourselves: the official Python client (ollama 0.6.3 from PyPI, in a virtual environment with Python 3.14.3) and the README's curl request, both against an empty port 11434. That produced the two connection errors quoted in Troubleshooting. Error strings marked as coming from source code or GitHub issues were read there, not reproduced. We re-test these steps monthly and record any change in this page's changelog.