49 lines
1.0 KiB
Markdown
49 lines
1.0 KiB
Markdown
# llm_web
|
|
|
|
Local LLM chat web page powered by Ollama.
|
|
|
|
## Run
|
|
|
|
```powershell
|
|
python -m venv .venv
|
|
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
|
|
.\.venv\Scripts\python.exe .\server.py
|
|
```
|
|
|
|
Open:
|
|
|
|
```text
|
|
http://127.0.0.1:8787
|
|
```
|
|
|
|
LAN access uses the notebook IP:
|
|
|
|
```text
|
|
http://<notebook-ip>:8787
|
|
```
|
|
|
|
Chat history is stored locally in `data/chats.json`. The `data/*.json` files are ignored by Git.
|
|
|
|
## Models
|
|
|
|
The web UI can switch between installed Ollama models:
|
|
|
|
- `qwen2.5:7b`: default, best speed/quality balance on this notebook
|
|
- `llama3.1:8b`: available for comparison
|
|
- `qwen2.5:14b`: test-only; can be slow or memory hungry on 16 GB RAM
|
|
- `qwen2.5:3b`: fallback model
|
|
|
|
Recent quick benchmark:
|
|
|
|
```text
|
|
qwen2.5:7b 18.0s
|
|
llama3.1:8b 32.4s
|
|
qwen2.5:14b 82.7s
|
|
```
|
|
|
|
## RAG
|
|
|
|
RAG is enabled by default. The server indexes local `.md`, `.txt`, `.py`, `.js`, `.html`, `.css`, and `.ps1` files, excluding `.git`, `.venv`, `__pycache__`, and `data`.
|
|
|
|
Use the `Reindex files` button after adding or editing reference files.
|