Ollama

Ollama — explained in more detail

Ollama is a compact runtime that lets language models run directly on your own hardware — no cloud, no token costs, no data leaving the device.

Operation

The main interface is a slim command line: ollama run llama3 pulls the model from a curated library and starts a chat. Available models range from small (Llama 3 8B, Phi-3) to large (Qwen 72B, DeepSeek), each in quantised variants of different precision.

API server

Alongside the CLI, Ollama runs an HTTP server with an OpenAI-compatible interface. Tools originally built against the OpenAI API — Continue.dev, Aider, custom scripts — can be pointed at the local instance just by swapping endpoint and model name.

Requirements

Speed and feasible model size depend directly on the hardware. 7B models run with 8 GB of RAM; 70B models need 64 GB plus a strong GPU. Anyone planning to work locally should check the machine’s spec before settling on a model.

More on this topic

See everything in one place:Local LLM→

Encyclopedia · 02 May 2026Running LLMs locally — hardware, tools, modelsHow to run language models on your own hardware — VRAM requirements, tooling (Ollama, LM Studio, llama.cpp, vLLM) and which models fit which GPU.GlossaryGPT4AllGPT4All is an open-source desktop application by Nomic AI for running large language models locally and offline on your own computer. It works without a GPU and without a cloud connection, so data never leaves the device.GlossaryJanJan is an open-source desktop application that lets you run large language models locally and fully offline on your own computer. It uses the llama.cpp engine and is positioned as a privacy-friendly ChatGPT alternative.