Ollama Setup Guide: Run AI Models on Your Own Hardware
Ollama is the simplest way to run large language models locally on your own hardware. With a single command you can download and run models including Gemma 4, Llama 4, Mistral, and dozens of others without any cloud dependency, API keys, or data leaving your machine. This guide covers installation on macOS, Linux, and Windows, model selection based on your hardware, and production configuration for self-hosted AI services.
Installation and First Run
Install Ollama on macOS with brew install ollama, on Linux with curl -fsSL https://ollama.com/install.sh | sh, or on Windows with the official installer from ollama.com. After installation, run ollama run gemma2:4 to download and interact with a small model immediately. The entire process from installation to first response takes under five minutes on a broadband connection.
Model Selection by Hardware
For machines with 8GB RAM, use 2-4B parameter models like Gemma 2 2B or Phi-3 Mini. With 16GB RAM, run 7-8B models like Gemma 4 or Llama 3.1 8B. With 32GB RAM, run 13-14B models or quantized versions of larger models. For 64GB+ systems, run full 30-70B models. Apple Silicon Macs benefit from unified memory architecture that makes GPU acceleration seamless with no configuration required.
Privacy and Security
Ollama runs entirely locally. No prompts, responses, or model interactions leave your machine. There are no analytics, telemetry, or usage tracking. The models themselves are downloaded once and cached locally. For air-gapped environments, models can be pre-downloaded and transferred via USB. This makes Ollama suitable for processing sensitive documents, medical records, legal materials, and personal information.
Key Findings
- Ollama installation to first response takes under five minutes with a single command
- Models run entirely locally with zero data leaving the machine and no telemetry
- Hardware requirements start at 8GB RAM for small models scaling to 64GB+ for full-size models
Timeline
Ollama initial release for macOS
Ollama 0.1.20 adds Windows support
Ollama supports 100+ models including vision models
Ollama adds Gemma 4 support on release day