Local-first coding assistant stack: Ollama serving Gemma 4 to Cline and Open WebUI

What they did

They wanted an agentic coding loop and a general chat assistant without sending repository code to a hosted API. Starting from the Ollama README, they installed the runtime on their workstation, pulled a Gemma 4 model, and confirmed the daemon answered on localhost:11434 with a single non-streaming /api/chat curl call. From there they fanned the same endpoint out to two clients: Cline inside VS Code for multi-file edits and terminal commands, and Open WebUI in the browser for longer research conversations and document Q&A. They used the ollama-python and ollama-js libraries for a small internal script that summarizes diffs, and kept model pulls pinned to explicit tags so the IDE agent and the chat UI did not drift onto different weights. The whole setup stayed on one machine, with the model list and conversation history living in local storage rather than a vendor account.

How it went

Working end to end: the curl smoke test, the Python and JS snippets, and both clients all hit the same local endpoint. Cline handled routine refactors and test fixes acceptably on a mid-size model; Open WebUI gave a usable chat surface with no signup. Polish is hobbyist-to-solid rather than production: no auth in front of the API, no queueing, and quality depends heavily on which Gemma 4 variant fits in VRAM.

Worth knowing

The README excerpt shows only the llama.cpp backend and no hardware sizing, context-window tuning, or GPU/CPU offload guidance, so throughput on laptops is unverified. The REST API on port 11434 is unauthenticated by default and should not be exposed beyond localhost. Agentic tool use in Cline is weaker on small quantized models than on frontier APIs, so expect more retries on multi-step edits. Model tags move; pin them explicitly.

Wiring

Install Ollama via the platform script (curl | sh on macOS/Linux, irm | iex on Windows) or the ollama/ollama Docker image, then `ollama run gemma4` once to pull weights and confirm the daemon. The server listens on http://localhost:11434 and exposes /api/chat; verify with a non-streaming curl POST containing model, messages, and stream:false. Clients attach in parallel: Cline is configured in the IDE to use the local Ollama endpoint as its model provider; Open WebUI runs as a separate container or process pointed at the same host URL; scripts use `pip install ollama` or `npm i ollama` and call chat() with the same model tag. Keep the daemon bound to 127.0.0.1, pin model tags in a shared config file, and run the curl smoke test before either client so failures are isolated to the server rather than the integration.

Try it yourself

Set up a local-first AI dev stack in this repo using Ollama. Steps: (1) verify `ollama` is installed and the daemon responds at http://localhost:11434; if not, print the install command for my OS and stop. (2) Pull a Gemma 4 tag that fits my available VRAM and record the exact tag in a new `docs/local-ai.md`. (3) Add a `scripts/smoke_chat.py` using the `ollama` Python package that sends one non-streaming chat message and prints the reply, plus a matching `scripts/smoke_chat.mjs` using the `ollama` JS package. (4) Write `docs/local-ai.md` covering: how to point Cline at the local endpoint, how to run Open WebUI against the same daemon, the model tag pinning policy, and a warning that port 11434 must stay bound to localhost. (5) Add a `make smoke` target that runs both smoke scripts. Do not modify application source; only add docs and scripts.