Add model selection and local RAG
This commit is contained in:
1 parent
2b2e5fcb0f
commit
bf39c2e070
6 files changed
+369
-32
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# llm_web
|
||||
|
||||
Local LLM chat web page powered by Ollama and `qwen2.5:7b`.
|
||||
Local LLM chat web page powered by Ollama.
|
||||
|
||||
## Run
|
||||
|
||||
@@ -23,3 +23,26 @@ http://<notebook-ip>:8787
|
||||
```
|
||||
|
||||
Chat history is stored locally in `data/chats.json`. The `data/*.json` files are ignored by Git.
|
||||
|
||||
## Models
|
||||
|
||||
The web UI can switch between installed Ollama models:
|
||||
|
||||
- `qwen2.5:7b`: default, best speed/quality balance on this notebook
|
||||
- `llama3.1:8b`: available for comparison
|
||||
- `qwen2.5:14b`: test-only; can be slow or memory hungry on 16 GB RAM
|
||||
- `qwen2.5:3b`: fallback model
|
||||
|
||||
Recent quick benchmark:
|
||||
|
||||
```text
|
||||
qwen2.5:7b 18.0s
|
||||
llama3.1:8b 32.4s
|
||||
qwen2.5:14b 82.7s
|
||||
```
|
||||
|
||||
## RAG
|
||||
|
||||
RAG is enabled by default. The server indexes local `.md`, `.txt`, `.py`, `.js`, `.html`, `.css`, and `.ps1` files, excluding `.git`, `.venv`, `__pycache__`, and `data`.
|
||||
|
||||
Use the `Reindex files` button after adding or editing reference files.
|
||||
Reference in new issue
Block a user