Ollama Fix, Crash & Optimization Guide

Ollama is a desktop and command-line tool for downloading and running open-weight large language models locally. It is popular with developers who want a private, offline LLM served on a local API.

Install / First Setup

• macOS / Linux: download the installer, or use the official install script.
• Windows: install the Windows app from the Ollama website, then run from the Start menu or terminal.

Pull and run a model:

ollama pull llama3
ollama run llama3

The local REST API listens on http://127.0.0.1:11434 by default. To start the server explicitly:

ollama serve

Common Issues & Fixes

"could not connect to ollama server" / server not running

Cause: The background Ollama service is not started. Fix: Start it with ollama serve in one terminal, or launch the desktop app so the service runs. Then retry ollama run.

Model runs on CPU, not GPU (slow)

Cause: GPU layers are not offloaded, or the model is larger than VRAM. Fix: On Linux set OLLAMA_NUM_GPU and let Ollama auto-offload; reduce num_gpu via a Modelfile PARAMETER if it OOMs. Ensure current GPU drivers are installed. On macOS, Metal is used automatically on Apple Silicon.

"model not found" / pull fails

Cause: Typo in the model name, or no network access to the model library. Fix: Verify the model tag with ollama list and the library name. For offline use, pull on a networked machine and copy the model, or use ollama create from a local Modelfile.

Context window too short

Cause: Default context length is applied. Fix: Set it when running: /set parameter num_ctx 8192 in the interactive session, or define PARAMETER num_ctx 8192 in a Modelfile and ollama create a custom model.

Out of memory during load

Cause: The model exceeds VRAM and Ollama cannot offload enough layers. Fix: Use a smaller or quantized (Q4) variant, or lower num_gpu so more layers stay on CPU/RAM (slower but fits). Close other GPU apps first.

Performance & Optimization

• Low-End (8 GB RAM, no discrete GPU): Use 3B–7B Q4 models; expect CPU-only speeds. Keep context modest (2048).
• Mid-Range (16 GB RAM, 6–8 GB VRAM): 7B–13B Q4 models fit on GPU; raise num_ctx to 4096–8192.
• Workstation (32 GB RAM, 16–24 GB VRAM): 13B–34B Q4/Q5 models run on GPU; raise context to 8192+. Monitor VRAM with nvidia-smi / Activity Monitor and tune num_gpu to stay just under the limit.
• Quantization (Q4/Q5) dramatically cuts memory versus fp16 with a small quality trade-off.

Version & Compatibility Notes

Ollama ships native builds for macOS, Windows, and Linux. GPU support depends on current NVIDIA drivers (CUDA) on Linux/Windows and Metal on macOS. Environment variables such as OLLAMA_NUM_GPU, OLLAMA_MAX_LOADED_MODELS, and OLLAMA_HOST control behavior; exact names and defaults can change—consult the official Ollama documentation for your version.

FAQ

Q: What port does Ollama use? A: The local API listens on 11434 by default (http://127.0.0.1:11434).

Q: How do I run a model with a bigger context? A: Use /set parameter num_ctx <n> interactively, or bake it into a Modelfile with PARAMETER num_ctx <n> and ollama create.

Q: Can Ollama use a model from Hugging Face? A: Yes, via a Modelfile that points FROM at a GGUF file (local or a HF URL), then ollama create.

Q: How do I make Ollama listen on the network? A: Set OLLAMA_HOST (e.g. 0.0.0.0:11434) before starting the server; only do this on trusted networks.

Q: Why is the first generation slow? A: The model is being loaded into memory on first use; subsequent prompts are faster.

Q: How do I list installed models? A: Run ollama list to see pulled models and their sizes.

Related Guides

• vLLM Fix & Optimization Guide
• LM Studio Fix & Optimization Guide
• Hugging Face Transformers Fix & Optimization Guide
• Dev RAM Calculator

Accuracy Note

Commands and paths reflect common, real-world setups as of 2026-08. Always verify against your installed version and OS. When in doubt, consult the official Ollama documentation.