MCP & RAG Specialization

The training-cutoff problem

Even a strong base model like Qwen 3-Coder only knows what existed in its training data. If the Vulkan working group ships a new extension today, the model won’t know about it — and it may confidently invent struct members or bitmasks that don’t exist, which can cost real debugging time.

MCP and RAG are the two main ways to give a model access to information beyond its training data.

Two kinds of context

It helps to separate two kinds of knowledge a model needs:

  1. The Vulkan spec itself — the same for every developer. This is what MCP connects the model to.

  2. Your engine’s specific conventions — private to your project, and something no general-purpose model was trained on. This is what RAG connects the model to.

MCP: connecting to the spec

The Model Context Protocol lets your AI query real-time data instead of relying only on training data.

Example. Say you need the valid VkImageUsageFlags for a storage image on Android. Instead of looking it up yourself, you ask the assistant to query your local SDK’s vk.xml for it. Rather than guessing from training data, the assistant reads the actual XML file and returns the real bitmask definitions — correct even for extensions released yesterday.

RAG: connecting to your codebase

Where MCP handles structured spec data, RAG handles your own source, internal docs, and architectural conventions.

Example. You’re writing a new Buffer wrapper and want it to match your existing Texture class’s style. With RAG set up, you ask the assistant to use Texture as a reference and generate a Buffer class following the same RAII and allocation patterns. Instead of a generic C++ class, the system retrieves your actual Texture implementation and generates something that matches your codebase’s existing conventions, without retraining the underlying model.

How a RAG pipeline works

Understanding the pipeline is useful because it’s how you turn a general-purpose model into something that knows your engine specifically, using only your local files.

Phase 1: curating source documents

Identify the documents worth indexing for your engine:

  • Header files (.h, .hpp, etc.) — your internal API surface.

  • Engineering guidelines — docs covering memory management or synchronization conventions.

  • External docs — local copies of third-party library manuals (Slang, VMA, Volk, SDL3) or vendor optimization guides not already covered by MCP tools.

NB: There’s a tradeoff between index breadth and retrieval speed — a wider index can be more accurate but slower to query, and it also costs tokens, leaving less budget for the actual work in a session. Keep the index focused rather than indexing everything.

Phase 2: embedding and indexing

  1. Chunking: files are split into smaller overlapping pieces (a function, a class definition).

  2. Embeddings: a small embedding model (bge-small-en, nomic-embed-text) converts each chunk into a vector.

  3. Vector database: vectors are stored so that conceptually similar chunks (e.g. VkBuffer and VkImage creation code) end up near each other.

Phase 3: retrieval

A query is converted to a vector and matched against the nearest stored chunks. Asking about "pipeline barriers" retrieves the files in your engine that actually handle synchronization.

Phase 4: augmentation

The retrieved chunks get added to the prompt as context before the model generates a response:

Context: [Your engine's Buffer.cpp code]
Context: [The VMA Reference for allocation]
User Prompt: Create a new vertex buffer for our triangle.

Building a knowledge index

Many IDE plugins automate this, but it’s worth setting up something more transparent using tools already covered in the environment section.

Step 1: organize a docs directory

Index a specific, curated directory rather than your whole home folder:

/vulkan_project
  /docs
    /vma_library_reference.md
    /engine_architecture.md
    /naming_conventions.md
  /src
    (Your engine code)

Step 2: manual retrieval with Goose

Instead of relying on a hidden plugin, you can do RAG explicitly with Goose.

  1. Load context. Before asking for code, tell the agent what to read:

    Goose, read 'docs/naming_conventions.md' and
'src/core/Buffer.hpp'. We are going to implement
a new Index Buffer class.

  2. Ask, referencing what you loaded:

    Using our naming conventions and the RAII pattern
seen in 'Buffer.hpp', generate 'IndexBuffer.hpp'
and 'IndexBuffer.cpp'. Ensure it uses 'vmaCreateBuffer'
for allocation.

This works because you’re doing the retrieval step yourself, and MCP (via Goose) lets the agent read your actual files as ground truth rather than guessing at your conventions.

Step 3: vector indexing at scale

If your project grows to thousands of files, manual retrieval gets slow. At that point a local vector database (ChromaDB, Pinecone, via MCP) lets the AI search the whole engine in milliseconds even without knowing the exact filename.

Step 4: knowledge graphs for structure

Vector search is good at finding similar text; a knowledge graph is better at finding related structure — for instance, tracing which VkCommandPool was created by which VkDevice. Indexing your project as a graph lets you ask things like "show me all systems that depend on `GlobalDescriptorSet`" and get back not just the classes that use it, but the shaders and barriers logically linked to it.

Step 5: a function registry

A function registry is a lookup of capabilities the AI can call on, rather than reimplement. If you ask it to capture a frame, it can check whether your engine already has a RenderDoc::CaptureFrame() method and use that instead of writing a new capture system from scratch.

Tip: using a cloud model to prep the RAG source. You can use a stronger cloud model to do the initial curation — summarizing your renderer’s conventions into a single markdown file that a smaller local model can then retrieve from, getting some of the larger model’s understanding without the VRAM cost.

Small models with a good RAG index can compete with far larger ones

A model in the 7B–30B range, given a good RAG index of your codebase, can outperform a much larger general-purpose model on tasks specific to your engine — because it isn’t spending its limited attention trying to recall your class names from training; the index handles that, and the model can focus on reasoning about the logic in front of it.

Summary

Combining a base model’s pre-trained knowledge with MCP’s access to the current spec and RAG’s access to your own codebase gives you an assistant that’s aware of both the Vulkan spec and your specific engine.

  • MCP handles the spec.

  • RAG handles your project’s conventions.

The next chapter covers fine-tuning with LoRA — a further step for baking your engine’s conventions directly into a model’s weights, and deploying that to a shared team server.

Next: Fine-Tuning for Your Engine (LoRA)