오프라인 리포지토리 도구용 단일 파일 로컬 LLM + 음성-텍스트 변환 러너

# 오프라인 로컬 추론 설정 (llamafile + whisperfile)

## 1. `.gitignore` 업데이트

```gitignore
# Local model binaries and runtime artifacts
.local/
```

## 2. `scripts/fetch-local-models.sh`

```bash
#!/usr/bin/env bash
# Downloads a small pre-built llamafile and a whisperfile binary into .local/.
# Both are chmod +x'd so they can be executed directly.
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
LOCAL_DIR="${ROOT_DIR}/.local"
mkdir -p "${LOCAL_DIR}"

# --- llamafile (Qwen3.5 0.8B Q8_0) -----------------------------------------
# Hosted on the mozilla-ai Hugging Face repo. Small enough to run on CPU.
LLAMAFILE_NAME="Qwen3.5-0.8B-Q8_0.llamafile"
LLAMAFILE_URL="https://huggingface.co/mozilla-ai/Qwen3.5-0.8B-llamafile/resolve/main/${LLAMAFILE_NAME}"

# --- whisperfile -----------------------------------------------------------
# Pre-built whisperfile binary (tiny.en model baked in).
WHISPERFILE_NAME="whisperfile-tiny.en"
WHISPERFILE_URL="https://huggingface.co/mozilla-ai/whisperfile/resolve/main/whisperfile-tiny.en"

fetch() {
  local url="$1"
  local dest="$2"
  if [[ -f "${dest}" ]]; then
    echo "✓ already present: ${dest}"
    return
  fi
  echo "↓ downloading ${url}"
  curl -fL --retry 3 --retry-delay 2 -o "${dest}.part" "${url}"
  mv "${dest}.part" "${dest}"
}

fetch "${LLAMAFILE_URL}"   "${LOCAL_DIR}/${LLAMAFILE_NAME}"
fetch "${WHISPERFILE_URL}" "${LOCAL_DIR}/${WHISPERFILE_NAME}"

chmod +x "${LOCAL_DIR}/${LLAMAFILE_NAME}" "${LOCAL_DIR}/${WHISPERFILE_NAME}"

echo "Done. Binaries in ${LOCAL_DIR}:"
ls -lh "${LOCAL_DIR}"
```

## 3. `scripts/serve-local.sh`

```bash
#!/usr/bin/env bash
# Launches the llamafile server on a configurable port with a modest context window.
# Logs to .local/server.log.
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
LOCAL_DIR="${ROOT_DIR}/.local"
LOG_FILE="${LOCAL_DIR}/server.log"

PORT="${PORT:-8080}"
CTX_SIZE="${CTX_SIZE:-4096}"
LLAMAFILE="${LOCAL_DIR}/Qwen3.5-0.8B-Q8_0.llamafile"

if [[ ! -x "${LLAMAFILE}" ]]; then
  echo "llamafile not found at ${LLAMAFILE}. Run scripts/fetch-local-models.sh first." >&2
  exit 1
fi

mkdir -p "${LOCAL_DIR}"
echo "Starting llamafile server on port ${PORT} (ctx=${CTX_SIZE}) → ${LOG_FILE}"

exec "${LLAMAFILE}" \
  --server \
  --host 127.0.0.1 \
  --port "${PORT}" \
  --ctx-size "${CTX_SIZE}" \
  --nobrowser \
  >>"${LOG_FILE}" 2>&1
```

## 4. `scripts/transcribe.sh`

```bash
#!/usr/bin/env bash
# Runs whisperfile over an audio file and writes a .txt transcript beside it.
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
LOCAL_DIR="${ROOT_DIR}/.local"
WHISPERFILE="${LOCAL_DIR}/whisperfile-tiny.en"

if [[ $# -lt 1 ]]; then
  echo "Usage: $0 <audio-file>" >&2
  exit 1
fi

AUDIO="$1"
if [[ ! -f "${AUDIO}" ]]; then
  echo "Audio file not found: ${AUDIO}" >&2
  exit 1
fi
if [[ ! -x "${WHISPERFILE}" ]]; then
  echo "whisperfile not found at ${WHISPERFILE}. Run scripts/fetch-local-models.sh first." >&2
  exit 1
fi

OUT="${AUDIO%.*}.txt"
echo "Transcribing ${AUDIO} → ${OUT}"
"${WHISPERFILE}" -f "${AUDIO}" -otxt -of "${AUDIO%.*}"
echo "Done: ${OUT}"
```

## 5. 모델 클라이언트를 로컬 엔드포인트로 연결

앱의 모델 클라이언트가 OpenAI 호환 엔드포인트를 사용하도록 설정합니다. 환경 변수로 호스팅 백엔드로 전환할 수 있게 합니다.

```ts
// src/lib/model-client.ts
const BASE_URL = process.env.MODEL_BASE_URL ?? "http://localhost:8080/v1";
const API_KEY  = process.env.MODEL_API_KEY  ?? "local-no-key";

export const modelClient = {
  baseURL: BASE_URL,
  apiKey: API_KEY,
  model: process.env.MODEL_NAME ?? "local",
};
```

호스팅 백엔드로 전환하려면:

```bash
MODEL_BASE_URL=https://api.example.com/v1 \
MODEL_API_KEY=sk-... \
MODEL_NAME=some-hosted-model \
npm start
```

## 6. 스모크 테스트

서버가 응답할 때까지 대기한 뒤 테스트를 실행합니다.

```ts
// tests/smoke.local.test.ts
import { describe, it, expect, beforeAll } from "vitest";

const BASE_URL = process.env.MODEL_BASE_URL ?? "http://localhost:8080/v1";
const TIMEOUT_MS = 60_000;

async function waitForServer(url: string, timeoutMs: number) {
  const deadline = Date.now() + timeoutMs;
  while (Date.now() < deadline) {
    try {
      const res = await fetch(`${url}/models`);
      if (res.ok) return;
    } catch {
      // not up yet
    }
    await new Promise((r) => setTimeout(r, 1000));
  }
  throw new Error(`Server at ${url} did not respond within ${timeoutMs}ms`);
}

describe("local inference smoke test", () => {
  beforeAll(async () => {
    await waitForServer(BASE_URL, TIMEOUT_MS);
  }, TIMEOUT_MS);

  it("responds to a chat completion", async () => {
    const res = await fetch(`${BASE_URL}/chat/completions`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${process.env.MODEL_API_KEY ?? "local-no-key"}`,
      },
      body: JSON.stringify({
        model: process.env.MODEL_NAME ?? "local",
        messages: [{ role: "user", content: "Say hello in one word." }],
        max_tokens: 16,
      }),
    });
    expect(res.ok).toBe(true);
    const data = await res.json();
    expect(data.choices?.[0]?.message?.content).toBeTruthy();
  });
});
```

## 7. README 섹션

````markdown
## Offline local inference

This repo can run fully offline using [llamafile](https://github.com/Mozilla-Ocho/llamafile)
for chat and [whisperfile](https://github.com/Mozilla-Ocho/llamafile#whisperfile) for
speech-to-text. Binaries live in a gitignored `.local/` directory.

### Setup

```bash
./scripts/fetch-local-models.sh   # downloads llamafile + whisperfile into .local/
./scripts/serve-local.sh          # starts the server on :8080 (override with PORT=...)
```

Then point the app at the local endpoint (defaults already do this):

```bash
MODEL_BASE_URL=http://localhost:8080/v1 MODEL_API_KEY=local-no-key npm start
```

To switch to a hosted backend, override the env vars:

```bash
MODEL_BASE_URL=https://api.example.com/v1 MODEL_API_KEY=sk-... npm start
```

### Transcription

```bash
./scripts/transcribe.sh path/to/audio.wav   # writes path/to/audio.txt
```

### Windows notes

- **Rename the `.exe`.** On Windows, downloaded binaries arrive as
  `Qwen3.5-0.8B-Q8_0.llamafile.exe` and `whisperfile-tiny.en.exe`. Rename them to
  drop the `.exe` suffix (or update the scripts to match) so the launcher treats
  them as llamafiles rather than plain executables.
- **4 GB executable limit.** Windows cannot execute a single `.exe` larger than
  4 GB. The Qwen3.5 0.8B Q8_0 build is well under this limit, but if you swap in
  a larger model, either pick a smaller quantization or use the
  `llamafile` + separate `.gguf` split so the executable stays under 4 GB.

### Smoke test

```bash
./scripts/serve-local.sh &          # start the server
npm test -- tests/smoke.local.test.ts
```

The test waits up to 60s for the server to respond before running.
````