What you’re learning
Ollama is a convenient model manager and runtime. llama.cpp gives you more direct control over model files, quantization, and runtime options. You can use it through command-line tools or its local server. The core concepts transfer between both.
Work through it
- Start with the idea. A model’s weights are the learned parameters. A GGUF file packages model weights and metadata for llama.cpp. Quantization stores weights with fewer bits, which reduces memory use and can affect output quality.
- Choose an install path. Use the official llama.cpp install/build guide for your operating system. If you build from source, the CPU build uses CMake; GPU acceleration requires a matching backend and supported drivers.
- Get a small GGUF model. Choose a model and quantization from its official model publisher. Check the file size, license, and hardware notes before downloading. For this lesson, begin with a small instruct model you can already run in Ollama, if a compatible GGUF is published.
- Run and compare. Launch the `llama-cli` program with the model file path using the current official examples. Give it the same prompt you used in Ollama. Note the startup options and response speed.
Example command / prompt
cmake -B build cmake --build build --config Release
Keep this in mind. Those are the official general CPU build commands; they are not a universal installer. GPU builds require a compatible backend such as CUDA or Vulkan, and setup differs by operating system. Begin with CPU mode if you are unsure, and follow the project’s current build guide.
Try thisRun one prompt through Ollama and llama.cpp using the same model family. Record which settings are visible and whether changing quantization changes speed or answer quality.