A retrieval-augmented generation (RAG) chatbot for WordPress documentation can run entirely on local hardware. The pattern is consistent: index your site’s posts and pages, convert each chunk of text into a numeric embedding with a local model, store those vectors in a small database, and let a local large language model (LLM) answer questions using the retrieved passages as context. Two documented reference implementations show how the pieces fit — Tuppu Agent, a Node.js server that combines SQLite, Xenova embeddings and a llama.cpp-compatible endpoint, and the MDDB WordPress agent guide, which wires WordPress export, a Markdown database, Ollama and Open WebUI together.
This guide is for technically curious beginners and intermediate practitioners who want a working mental model and a reproducible path, not a vendor pitch. It describes what each component does, what hardware the documented stacks expect, where the common failures occur, and what tradeoffs to accept before you start.
What a local RAG chatbot for WordPress actually does
Retrieval-augmented generation separates a chat system into two phases. During indexing, the site’s content is split into chunks and each chunk is converted into an embedding — a vector that represents its meaning. During answering, the user’s question is embedded the same way, the closest chunks are found by similarity search, and those chunks are pasted into the prompt sent to the LLM. The model then answers from the supplied context rather than from its training data alone.
This matters for documentation because an LLM’s parametric memory is stale and unverifiable. RAG keeps the answer grounded in the current site content, and because everything runs locally, the content never has to leave the machine. The tradeoff is that you now maintain an index that must be refreshed when posts change.
Two documented stacks to learn from
Tuppu Agent is a minimal RAG server written in TypeScript. According to its repository, it indexes WordPress posts through the REST API with an RSS fallback, performs chunked semantic search using cosine similarity over embeddings, uses the local Xenova/all-MiniLM-L6-v2 model for embeddings, keeps everything in a single-file SQLite database (tuppu.db), and calls an OpenAI-compatible /v1/chat/completions endpoint such as a llama.cpp server. It also ships a small no-build browser UI served at /. The project is MIT-licensed, requires Node.js 18+, and downloads its Xenova models on first use into the Node cache. The repository notes that no Python or GPU is required and that CPU-only operation is viable even on a low-power Debian VPS or NUC.
The MDDB guide takes a different route. It exports WordPress content to Markdown with wpexportjson, imports it into mddbd (a Markdown database that stores content and vector embeddings), generates vectors with Ollama, and exposes a chat interface through Open WebUI. Its published hardware table lists a minimum of 16 GB RAM and 4 CPU cores, with 32 GB and 8+ cores recommended, plus 20 GB of free disk space. A GPU is optional; the guide states that Ollama works on CPU but is roughly ten times slower, with GPU-backed responses generating in one to three seconds instead of ten to thirty.
Prerequisites and hardware expectations
Before writing any code, decide which stack matches your environment. The Tuppu approach is lighter and more self-contained: Node.js, npm, disk space for the SQLite file and model caches (the repository warns of hundreds of megabytes on first run), and optionally a local llama.cpp server. The MDDB approach assumes Docker and a more generous machine, and its own documentation recommends Linux (Ubuntu 22.04+) or macOS/WSL2.
For CPU-only deployments, the MDDB guide recommends smaller models such as llama3.2:3b instead of llama3.2:8b, and notes that the embedding model nomic-embed-text is lightweight enough to run well on CPU. Tuppu’s embedding model, all-MiniLM-L6-v2, is similarly small. The practical rule is that embedding is cheap and generation is expensive; size your hardware around the LLM, not the vector search.
Step 1 — Get your WordPress content into a machine-readable form
Both stacks begin by pulling content out of WordPress. Tuppu Agent uses the WordPress REST API and falls back to RSS if REST is unavailable, which makes it resilient on sites where the REST endpoints are restricted or cached oddly. The MDDB workflow instead exports to Markdown using wpexportjson and then imports that Markdown into mddbd.
Whichever route you take, verify the export before embedding anything. Fetch a single post through the REST API in a browser or with curl and confirm that the title and body are present. If REST returns an empty array or a 401, check whether a security plugin has disabled the API or whether authentication is required. An index built on an empty export will produce a chatbot that confidently says nothing useful.
Step 2 — Generate embeddings locally
Embeddings are the core of retrieval. Tuppu Agent uses Xenova’s JavaScript port of all-MiniLM-L6-v2, which runs in Node without a Python environment. The first run downloads the model weights into the Node cache, which is why the repository warns about disk usage. MDDB relies on Ollama’s nomic-embed-text model for the same job.
Two settings deserve attention here. First, chunk size: documents are split before embedding, and chunks that are too large dilute the vector, while chunks that are too small lose context. Second, consistency: the same embedding model must be used for both indexing and querying. Mixing models produces vectors in incompatible spaces and retrieval degrades to noise.
Step 3 — Store vectors and retrieve relevant chunks
Tuppu Agent stores embeddings in a single SQLite file and ranks results with cosine similarity. SQLite is not a dedicated vector database, but for a documentation site of ordinary size the simplicity is a feature: one file to back up, no separate service to operate. MDDB uses its own document database with an HTTP API on port 11023 and a gRPC API on port 11024, as shown in its Docker Compose example.
At query time, the incoming question is embedded with the same model, compared against stored vectors, and the top matches are returned. Most implementations then pass only those passages to the LLM. Keep the retrieved set small enough to fit the model’s context window; stuffing too many chunks into the prompt increases latency and can bury the relevant passage.
Step 4 — Connect a local LLM
Tuppu Agent expects an OpenAI-compatible server at an endpoint such as http://127.0.0.1:8080/v1, which is the shape exposed by a llama.cpp server. MDDB uses Ollama for both embeddings and generation. In both cases the LLM receives the retrieved context plus the user’s question and returns an answer.
Because the endpoint is OpenAI-compatible, the same application code can point at a different local server without rewriting the client. That portability is worth preserving in your own implementation: keep the base URL and model name in configuration rather than hard-coding them.
Step 5 — Add a chat interface
Tuppu Agent includes a minimal no-build UI served at /, which the repository says works on any Debian or Ubuntu box. It defaults to English responses and can be internationalized in the web UI. MDDB pairs with Open WebUI as the chat front end. A simple interface is enough: an input box, a message history, and a way to show which source passages were retrieved. Showing sources is not cosmetic — it is how users verify that an answer is grounded.
Verifying the pipeline works
Test each stage independently rather than only chatting with the finished bot. Confirm that the export contains posts, that embeddings were written to the database, that a known query returns the expected chunk, and only then that the LLM produces a coherent answer. A useful check is to ask a question whose answer appears verbatim in one post; if the bot answers correctly, retrieval and generation are both working. If the bot answers plausibly but wrongly, retrieval is probably returning the wrong chunks.
Common failure points and fixes
- Empty or partial index. REST API blocked, pagination not followed, or RSS fallback silently returning fewer items. Re-run the export and count records.
- Nonsense answers. Embedding model mismatch between indexing and querying, or chunks too large. Re-embed with one model and reduce chunk size.
- Slow responses. CPU-only generation. The MDDB guide’s own estimate is a tenfold slowdown without a GPU; switch to a smaller model or add a GPU.
- Missing model files. First-run downloads can fail behind a proxy or firewall. Check the Node or Ollama cache and retry.
- Stale answers. The index is not refreshed when posts change. Schedule a re-index.
Limitations and tradeoffs
Local RAG is not free of cost. The MDDB hardware table implies a machine with 16–32 GB of RAM and, ideally, a GPU with 8 GB or more of VRAM; a CPU-only setup works but is slower. Tuppu Agent is lighter because it uses a small embedding model and a single SQLite file, but it still needs a separate LLM server to generate answers. Neither stack provides automatic content governance: if your documentation contains outdated or contradictory pages, the chatbot will retrieve and repeat them. Retrieval quality also depends on chunking and on how well the embedding model handles your domain vocabulary. Treat the chatbot as an interface to your content, not as an authority above it.
Next steps
Start with the lighter Tuppu Agent repository if you want to see the whole loop in one Node.js project, or follow the MDDB guide if you prefer Docker, Ollama and Open WebUI. In either case, build the index first, verify retrieval with a known question, and only then tune the LLM prompt. Readers are welcome to share their environment, the stack they chose, and any errors they hit in the community discussion.
FAQ
Do I need a GPU to run a local RAG chatbot for WordPress?
No. Both documented stacks can run on CPU. Tuppu Agent's repository states that no GPU is required and that CPU-only operation is viable even on a low-power Debian VPS or NUC. The MDDB guide notes that Ollama works on CPU but is roughly ten times slower, and suggests smaller models such as llama3.2:3b for CPU-only setups.
Does the chatbot send my WordPress content to a cloud service?
Not in the documented configurations. Tuppu Agent embeds locally with Xenova and calls a local OpenAI-compatible endpoint such as a llama.cpp server; MDDB uses Ollama for embeddings and generation. Both are designed to keep the content on the machine running the stack.
What WordPress content can be indexed?
Tuppu Agent indexes WordPress posts through the REST API with an RSS fallback. The MDDB workflow exports content to Markdown with wpexportjson before importing it into mddbd. Pages and custom post types depend on how the export or REST query is configured.
How do I keep the chatbot's answers current?
Re-run the indexing step when posts change. Neither documented stack refreshes the index automatically, so a scheduled re-index is the practical solution; otherwise the chatbot retrieves and repeats outdated content.