GPU acceleration
Answer speed is dominated by where the model runs. On a CPU, the first answer after a pause takes one to three minutes. On a mid-range NVIDIA card, seconds.
How it's selected
One setting in your .env:
COMPOSE_PROFILES=cpu # or nvidia, or amd
The compose file ships three variants of the model container and starts exactly one of them. They share the same name, address and downloaded models, so switching between them keeps everything and re-downloads nothing.
COMPOSE_PROFILES is unset, none of them start. The rest of the stack
comes up, and every question fails. This is the most common self-host mistake.The desktop app's guided setup detects this for you on first install. Everyone else picks it below.
Automatic detection
curl -L -o detect-hardware.sh \
https://github.com/ChroniclerLM/Releases/releases/latest/download/detect-hardware.sh
chmod +x detect-hardware.sh
./detect-hardware.sh
It checks the platform, whether containers can actually reach a GPU, and how much
memory Docker has, then writes COMPOSE_PROFILES into the .env beside it. It
never touches your database password, and never changes your model.
Useful flags: --recommend-only (print, change nothing), --coding (bias the
suggestion towards a coding model), --env-file PATH (target a different .env).
The script needs bash — it runs on Linux, and on Windows from Git Bash or a WSL shell. If you can't run it, set the value by hand as below.
NVIDIA
Confirm the host sees the card
nvidia-smi
No output means a driver problem — fix that first, nothing below will help.
Confirm containers can see it
The driver being installed on the host is not enough; Docker needs the NVIDIA Container Toolkit to pass the card through.
docker run --rm --gpus all ollama/ollama:0.32.4 nvidia-smi
If that prints your card, you're ready. If it errors:
- Linux — install the toolkit, register it with Docker and restart:Terminal
sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker
Install instructions: NVIDIA Container Toolkit. - Windows — update the NVIDIA driver (the WSL2 driver includes container support), make sure Docker Desktop is on the WSL 2 based engine, and restart Docker Desktop. There is nothing extra to install.
Switch the profile
COMPOSE_PROFILES=nvidia
docker compose up -d
AMD
Linux only, and best-effort. ROCm reaches the card through /dev/kfd and
/dev/dri, which Docker Desktop on Windows cannot pass through.
ls /dev/kfd /dev/dri
If both exist:
COMPOSE_PROFILES=amd
docker compose up -d
This variant pulls a different, larger image (the ROCm runtime), so the first start after switching takes a while.
Verify it's actually being used
docker compose ps
The running container should be ollama-nvidia or ollama-amd, not
ollama-cpu. Then ask a question and watch:
nvidia-smi # NVIDIA: memory in use jumps while generating
docker compose logs chronicler-ollama | tail -30
The Ollama log says which backend it loaded the model onto. The honest test is the clock: if a warm answer still takes a minute, it's on the CPU.
Switching later
Edit COMPOSE_PROFILES, then docker compose up -d. Compose replaces the model
container in place. Downloaded models live in a shared volume, so nothing is
re-downloaded and no data is lost.
Known limitation: model list in the admin console
Admin → Model probes hardware from inside the backend container, which has no access to the GPU. It therefore reports no graphics card and greys out the GPU-class models even on a machine that has one.
Models that run on the CPU are unaffected and still selectable — the probe reads RAM correctly, it's only the graphics card it can't see. To use a GPU-class model anyway, set it on the server instead of in the console: see Choosing a GPU model by hand.