VoiceThere

Docs knowledge base (RAG)

Overview

The landing page assistant on this site answers from these docs, and you can do the same for yours. The pieces are a small MCP server that searches your pages, a container image that has your pages baked in, a project service that runs the image, and an LLM agent that calls the search tool before it states a fact. The agent code needs no changes. Nothing is stored at runtime: the index is a read-only file set that you build once per version of your docs.

Search runs inside the container, so there is no embedding API key and no outside call per question. The server and the base image are public in the @voicethere/agent repository as ghcr.io/voicethere/docs-rag-mcp.

Your docs as a corpus

The index builder reads one of two inputs.

  • A folder of .md files. The page id is the file path, the title is the first # heading, and --base-url turns the path into the page URL that the agent can read out.
  • A corpus JSON file with documents, each one an id, a title, a url and the page text as markdown. Use it when your pages come from a site generator and you would rather index the rendered text than the sources. This site does that: the corpus is built from the rendered pages, so a price shown on a page is the price the assistant quotes.

Build your image

Start from the base image, copy your docs in and build the index in the same Dockerfile. The base image has the search server and the embedding model, runs as a non-root user and refuses to start without an index.

FROM ghcr.io/voicethere/docs-rag-mcp:0.25.0
COPY docs/ /docs/
RUN docs-rag build --docs /docs --base-url https://docs.example.com/ --out /index
RUN docs-rag eval --index /index --questions /questions.json --min-recall 0.9

Pin the base to a release tag. Push the result to any registry; a public image needs no credential. Two environment variables shape what the model sees: DOCS_RAG_DESCRIPTION describes the search tool (for example Search the Acme docs) and DOCS_RAG_TOOL_PREFIX renames both tools (acme_ gives acme_search_docs). If you use the llm-docs-rag template, update its allow list and prompt to match.

Search runs on one CPU thread by default, which suits a container with a CPU limit below one core. Set DOCS_RAG_THREADS to a higher number only when the container has that many cores to itself; more threads than the limit allows makes every search slower.

Check retrieval with an eval

docs-rag eval takes a list of questions, each with the id of the page that answers it and optionally a fact that page contains. It reports recall@4 and MRR for vector search, text search and the combined search, and exits with an error when combined recall@4 is below --min-recall. Put it in the Dockerfile and a docs change that makes retrieval worse fails the build instead of reaching your users. Write the questions the way a visitor would ask them, and take the page id from the page that really holds the answer. About forty questions across your main topics is enough to catch a regression.

[
  { "q": "How much does the Advanced plan cost per month?", "docId": "pricing", "fact": "$45/mo" },
  { "q": "Which command deploys my build?", "docId": "quickstart", "fact": "deploy --wait" }
]

Run it as a project service

In the project's Services tab, or with the CLI, set the image, one port named mcp on 8080 and an HTTP health check on /healthz. Choose always on so the first visitor does not wait for a cold start. Then deploy.

voicethere projects services set docs \
  --image registry.example.com/acme/docs-mcp:1 \
  --port mcp:8080 \
  --health http:/healthz:8080 \
  --run-mode always_on \
  --cpu 500 --memory 1024 --start-timeout 120
voicethere deploy --wait

The agent finds the service on the private port and sends the service token, so leave MCP_TOKEN unset in the image. The agent log shows an llm tools discovered line once the tools are listed.

The llm-docs-rag template

The llm-docs-rag template in @voicethere/agent 0.21.0 and later is an LLM agent wired for this. Its voicethere.llm.json turns the built-in knowledge store off and lists the service as a tool source that allows only search_docs and get_doc. Its prompt tells the model to search before it states any product fact, to give the number from the result and to name the page URL. Edit prompt.md for tone and scope, and keep that rule.

Each page is cut into passages of about 300 words along its headings. A question is searched twice: once by meaning, with the local bge-small-en-v1.5 embedding model, and once by keyword, with SQLite full-text search that matches exact numbers and words. The two ranked lists are fused, and the server returns up to ten passages, at most two from one page, capped at 4000 characters. Every hit carries the page title, its URL and its id, so the agent can cite the page or call get_doc to read all of it.

Since every hit carries its page URL, the agent can point the caller at the page instead of reading its address out. With @voicethere/agent 0.22.0 and later the URL in the reply is taken out of speech and sent to the client as an assistant_content link with the reply's turnId; the Links in replies section has the message format. Two settings in voicethere.llm.json make those links trustworthy:

  • "links": { "requireToolSource": true } sends a URL only when it appears in a tool result of the same turn, so a link always comes from a search_docs or get_docresult and never from the model's memory.
  • "toolChoice": "required-first-round" makes the first model call of each turn call a tool, so the search runs before the answer. It needs tools.maxRounds of 2 or more; see Always call a tool first.

Tell the model in the prompt to end its answer with the URL of the result that answered the question, exactly as returned. The client shows it as a link under the reply, and the caller hears only the sentence.

Limits

  • A project service can use at most 1024 MiB of memory and 1000 millicores. The server needs about 260 MiB after the first searches, so the limits leave room.
  • The embedding model is English-first. Other languages work for keywords and identifiers but match worse by meaning, so run the eval on your own questions before you rely on it.
  • The index is built at image build time. To publish new docs you build and push a new image and redeploy; there is no live re-index.
  • While the service runs it costs credits like any project service, see Resources and pricing.

← All documentation