Performance and memory

Chronicler is slow for one of three reasons, in this order of likelihood:

1. There's no GPU, or there is one and it isn't being used.
2. Docker doesn't have enough memory, so the model reloads constantly.
3. The model is too big for the machine and is spilling to disk.

Fix them in that order. GPU acceleration covers the first, this page covers the rest.

1. Give Docker more memory

This is the setting people mean when they say "allocate more RAM", and it lives in Docker, not in Chronicler. Nothing in the compose file can raise it.

[wsl2]
memory=16GB
processors=8
swap=8GB

wsl --shutdown

free -h

Check what Docker actually got:

docker info --format "{{.MemTotal}}"

Under 4 GiB and the backend will be killed as soon as a model loads.

2. Tune the model runtime

These go in the .env next to your compose file. Each is optional — leave it out and Ollama's own default applies. Run docker compose up -d after a change.

OLLAMA_KEEP_ALIVE — the one worth changing

Ollama unloads the model five minutes after the last question. The next question pays the full cold load again: one to three minutes on a CPU, several seconds on a GPU. On a dedicated server there's no reason to keep paying that.

OLLAMA_KEEP_ALIVE=-1

-1 keeps the model resident permanently. The cost is exactly the model's memory (3–20 GB depending on which one), held whether anyone is asking or not. On a shared machine that has other work to do, prefer something like 2h.

OLLAMA_NUM_PARALLEL — for several people at once

Each parallel slot gets its own context memory, on top of the model weights. Doubling it roughly doubles the per-context memory. With one or two users, leave it alone; a spare 1 also frees memory on a small machine.

OLLAMA_NUM_PARALLEL=4

Only useful with memory to spare. If the machine is already tight, more parallel slots make everything slower, not faster.

OLLAMA_MAX_LOADED_MODELS

Chronicler generates with a single model, so leaving this alone is right for almost everyone. Setting it to 1 on a small machine guarantees a second model never sneaks into memory.

OLLAMA_CONTEXT_LENGTH

Included for completeness. Changing it is almost never the fix you're looking for.

3. Tune the database

Also in .env. The defaults are Postgres' own, which are famously conservative.

POSTGRES_SHARED_BUFFERS=2GB
POSTGRES_WORK_MEM=32MB

A common rule of thumb for shared_buffers is 25% of the memory you're willing to give the database. work_mem is allocated per sort, not per connection, so a large value times many concurrent queries is how servers run out of memory — 32 MB is generous, 1 GB is a mistake.

Only worth touching once you have thousands of documents. Below that, the default is fine and the model is your bottleneck.

4. Right-size the model

A model that doesn't fit is the worst case: it spills to disk and each answer takes minutes. Match the model to the memory you actually have — see Models. Smaller and resident beats bigger and swapping, every time.

Worked examples

OLLAMA_KEEP_ALIVE=30m
OLLAMA_NUM_PARALLEL=1

OLLAMA_KEEP_ALIVE=-1
OLLAMA_NUM_PARALLEL=2
POSTGRES_SHARED_BUFFERS=2GB
POSTGRES_WORK_MEM=16MB

OLLAMA_KEEP_ALIVE=-1
OLLAMA_NUM_PARALLEL=4
POSTGRES_SHARED_BUFFERS=4GB
POSTGRES_WORK_MEM=32MB

Check what you changed

docker compose config | grep -A6 "OLLAMA_"        # what compose resolved
docker exec chronicler-ollama env | grep OLLAMA   # what the container actually got
docker exec chronicler-postgres psql -U chronicler -d chronicler_app -c 'show shared_buffers;'
docker stats --no-stream                          # what everything is using now

A setting you left out of .env won't appear in the container at all. That's correct — it means the runtime's own default is in force.

Speed that isn't about memory

• Scope your questions. A conversation pointed at three documents is far faster than one pointed at everything.
• Index during quiet hours. Bulk uploads, especially scanned ones, compete with chat for the same CPU.
• Use an SSD. Loading a 20 GB model off a spinning disk is a bad minute.