llm_web

Local LLM chat web page powered by Ollama.

Run

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe .\server.py

Open:

http://127.0.0.1:8787

LAN access uses the notebook IP:

http://<notebook-ip>:8787

Chat history is stored locally in data/chats.json. The data/*.json files are ignored by Git.

Models

The web UI can switch between installed Ollama models:

qwen2.5:7b: default, best speed/quality balance on this notebook
llama3.1:8b: available for comparison
qwen2.5:14b: test-only; can be slow or memory hungry on 16 GB RAM
qwen2.5:3b: fallback model

Recent quick benchmark:

qwen2.5:7b   18.0s
llama3.1:8b  32.4s
qwen2.5:14b  82.7s

RAG

RAG is enabled by default. The server indexes local .md, .txt, .py, .js, .html, .css, and .ps1 files, excluding .git, .venv, __pycache__, and data.

Use the Reindex files button after adding or editing reference files.