Selecting the Base Model

The first question is whether the model fits on your hardware alongside your Vulkan application. Higher parameter counts generally mean better reasoning, but a 70B model isn’t worth it if it forces your application into slower system RAM.

Quantization (compression formats like GGUF or EXL2) makes larger models fit on consumer hardware. 4-bit (Q4) quantization is a common middle ground — it cuts memory footprint by roughly 75% with only a small drop in accuracy.

A reasonable test question: "Explain the synchronization requirements for a compute-to-graphics dependency where the compute shader writes to a storage buffer that is then used as a vertex buffer in the next pass."

A weaker model will often answer with a blanket vkDeviceWaitIdle or a generic barrier with no access masks specified. A stronger one will identify the need for a transition from VK_ACCESS_SHADER_WRITE_BIT to VK_ACCESS_VERTEX_ATTRIBUTE_READ_BIT and name the correct pipeline stages.

NB: This test, and any specific model recommendations, will age. Treat the criteria as durable and the named models as examples current at time of writing — expect to re-evaluate as new models ship. The model (and how much fine-tuning it gets) is really just another parameter you tune based on how much help a given task needs.

Vulkan engines have a lot of files. A model with a small context window (say, 4,000 tokens) will lose track of your memory allocator by the time it’s read through your pipeline setup. Look for at least 32k–128k tokens of context if you want the assistant to hold a renderer’s header files in view while working on a specific implementation.

Qwen 3-Coder (30B) is currently one of the stronger local, non-cloud coding models for C++.

Example: a bindless texture refactor. Moving an engine to bindless textures touches descriptor set layout bindings, descriptorIndexing features, and how GLSL access maps to SPIR-V. Given this task, Qwen 3-Coder can look at how your shaders use sampler2D textures[] and suggest the specific DESCRIPTOR_BINDING_PARTIALLY_BOUND_BIT flags needed to avoid validation errors, and it tends to catch synchronization issues that smaller models miss. That makes it a reasonable choice for architectural work and Vulkan 1.3/1.4 features — at the cost of roughly 20GB of VRAM at Q4_K_M, which realistically means a 24GB GPU or a local server.

For fast, inline feedback while typing, Llama 4 (17B) is a solid choice. It can predict the next several lines of a VkPipelineShaderStageCreateInfo array as you write it, correctly infer entry point names from project context, and set pSpecializationInfo to nullptr when it sees you aren’t using specialization constants.

At around 6GB of VRAM, it runs comfortably alongside a Vulkan application on mid-range hardware (12–16GB GPUs), which makes it a good fit for autocomplete, unit tests, and quick shader iteration.

Mistral-Nemo (12B), a joint NVIDIA/Mistral AI release, is tuned for efficient reasoning with a 128k context window.

Example: an engine-wide refactor. A global illumination system that touches the scene graph, several passes, and post-processing needs a model that can keep more than one file in view at a time. Because Mistral-Nemo can hold up to 128,000 tokens of context, it can read across a renderer’s header directory and give suggestions that stay consistent with the rest of the codebase — useful for large refactors and following class hierarchies that smaller models tend to lose track of. At roughly 8–10GB VRAM (Q4), it sits between the fast 17B models and the heavier 30B+ ones.

Cloud models like Claude 4.6 or GPT-5.3 handle abstraction well — you can talk to them roughly the way you’d talk to a colleague. If you say "I’m seeing Z-fighting on distant terrain, using a standard 24-bit depth buffer, what should I check first?", the model can infer you want the artifact fixed and suggest likely causes (depth precision, near/far plane ratio) without you spelling out your projection matrix setup.

Local models, even capable ones like Qwen 3-Coder (30B), generally need more specific instructions — they’re better treated as a skilled implementer working from a spec than as a conversational partner. A more effective local-model prompt: "I’m seeing Z-fighting. Check whether the near plane is too close (currently 0.01) or the far plane too distant (10000.0). Suggest a logarithmic depth buffer if the ratio exceeds 1:1,000,000, and provide the GLSL for the vertex shader update." Being explicit like this keeps a local model on track.

When planning with a model like Claude 4.6, instead of asking for code directly, ask it to produce an instruction document for a local assistant.

The cloud model does the translation from loosely specified intent to a precise spec, which saves you from writing that spec by hand; the local model then does the implementation without needing your proprietary code to leave the machine. This works best as a plan-then-implement split, and it’s really only worth the overhead for tasks big enough that writing the spec pays for itself — for something trivial, it may take the cloud model as long to write the spec as it would to just do the task itself.

NB: LLM_NOTES.md is a working document for the current session, not project documentation — it shouldn’t be checked into git.

This approach is most useful on longer tasks with multiple iterations, and it’s also a reasonable way to unstick a local model that’s gone in circles on a problem: a short cloud session can often get it back on track.