Private offline multi-model chat client with local Ollama and image generation

What they did

They set up LLM X, a browser-based chat client that makes no external API calls, and wired it to local model servers. They installed Ollama and pulled a model, then configured OLLAMA_ORIGINS so the hosted web app (or the Chrome extension) could reach the local server. They also started LM Studio with CORS enabled and, for image generation, launched AUTOMATIC1111 with the API and CORS flags. Optionally they enabled Gemini Nano in Chrome Canary via the on-device model flags. They then opened the web app or extension, went offline, and chatted with multiple models at once, with chats and images stored locally in IndexedDB.

How it went

Works fully offline with chats and image generations kept private on-device. Multi-model replies from different local servers can run simultaneously, and the Chrome extension connects out of the box. Polish is good, though setup depends on correct CORS and origin flags per server.

Worth knowing

The hosted web app requires a secure context, so remote servers need HTTPS. Each backend needs its own CORS/origin configuration, and Gemini Nano requires Chrome Canary with specific flags and a model download. Image generation needs AUTOMATIC1111 running with API and CORS enabled.

Wiring

Install Ollama and pull a model (e.g. ollama pull llava). For the hosted web app, run OLLAMA_ORIGINS=https://mrdjohnson.github.io ollama serve; for the extension use OLLAMA_ORIGINS=chrome-extension://iodcdhcpahifeligoegcmcdibdkffclk. Start LM Studio with lms server start --cors=true. For images, run ./webui.sh --api --listen --cors-allow-origins "*". Optionally enable Gemini Nano in Chrome Canary with BypassPerfRequirement and the Gemini Nano API flag. Open the web app or extension, let auto-connect find the server, and chat. For source builds use yarn install && yarn preview, docker compose up -d on port 3030, or yarn chrome:build and load unpacked from extensions/chrome/dist.

Try it yourself

Set up LLM X locally against my Ollama server. Check my repo for any existing config, then: 1) confirm Ollama is installed and pull a small model, 2) start Ollama with OLLAMA_ORIGINS set for both https://mrdjohnson.github.io and the Chrome extension origin, 3) optionally start LM Studio with CORS and AUTOMATIC1111 with --api --listen --cors-allow-origins "*", 4) verify the app connects and can chat with two models at once, and 5) document the exact commands in a README section.