Vulkan vs OpenCL vs CPU: Which Backend and When

Understand GPU acceleration options and how TokForge auto-detects the fastest path for your device.

The Acceleration Paths

TokForge doesn't ask you to choose a backend. Instead, TokForge detects your chip and picks a path for each model automatically, using routes that have been tested on that GPU family. Here are the paths it can use:

• • CPU: Works on every phone, and on many phones it is the fastest path for chat
• • OpenCL: GPU compute, used mostly on Snapdragon phones
• • Vulkan: GPU compute, used mostly for GGUF models on supported phones with other GPUs, such as the Pixel 9 and the Xiaomi 14T Pro
• • NPU (Qualcomm QNN): the Snapdragon AI chip. In TokForge 1.3.6.1 it is used for image generation only, not for chat. In the beta (1.3.8 and newer), an optional speed boost also lets it read long chat prompts on Snapdragon 8 Gen 3 and 8 Elite Gen 5 phones, while the CPU writes the replyBeta: 1.3.8 and newer

You don't have to switch between them by hand, though you can. TokForge handles the choice, and the Runs on row in AutoForge shows where the current model runs.

New in 1.3.8 (beta):Beta: 1.3.8 and newer the per-chip rules on this page describe 1.3.6.1, the Google Play production version. In 1.3.8 they are starting points only. Each phone checks the GPU's answers once while idle, then times the CPU, the GPU and a mix of both for the model you loaded, and keeps whichever measured faster. Settings (Advanced mode) > Speed & engine > Performance > Measured on your phone shows what it found. On Snapdragon 8 Gen 3 and 8 Elite Gen 5, an optional NPU boost can read long prompts. Pixel 10 and Snapdragon 865 phones stay on the CPU, and Gemma 4 E4B stays on the CPU on every GPU.

Two Inference Backends

It's important to distinguish acceleration paths from inference backends. TokForge uses two backends on Android:

• MNN: runs on CPU, with OpenCL and Vulkan GPU paths depending on the phone
• GGUF/llama.cpp: runs on CPU, with OpenCL and Vulkan GPU paths, and uses ARM's KleidiAI CPU routines on supported chips

The model's format decides the backend: MNN bundles load in MNN, GGUF files load in llama.cpp. The acceleration path is chosen within that backend. On iPhone and iPad, TokForge runs models with llama.cpp on Metal; the rest of this guide is about Android.

GPU Path Comparison

Here's how the paths compare in real-world use:

Which Chipset Uses What

Device manufacturers use different SoCs (System-on-Chip). Here's how TokForge 1.3.6.1 handles each major category. In the beta (1.3.8 and newer) these rules are only starting points: your phone's own check and measurement pick the route (see how your phone picks the route).Beta: 1.3.8 and newer

Snapdragon phones

Qualcomm's chips, from flagships to mid-range. TokForge uses:

• CPU or OpenCL for MNN models, depending on the phone and model
• CPU for GGUF models, with Vulkan offered as an option on some recent flagships
• The NPU for image generation on supported chips (not for chat in 1.3.6.1; in the beta (1.3.8 and newer), an optional boost on Snapdragon 8 Gen 3 and 8 Elite Gen 5 lets it read long chat promptsBeta: 1.3.8 and newer)

Recent flagship phones with other GPUs

Flagship phones without a Snapdragon chip, such as the Xiaomi 14T Pro and the OnePlus Ace 5 Ultra:

• Vulkan for GGUF models; the largest models stay on CPU, and on some of these phones mixture-of-experts models do too
• CPU for MNN models, with Vulkan offered on some of these phones as an option for some models where it tested faster on long prompts

Pixel phones

• Pixel 9: Vulkan for GGUF models, CPU for MNN models
• Pixel 10 and older Pixel phones: CPU

Other and Mid-Range Phones

Mid-range Snapdragon phones, other mid-range and older chips, and similar:

• CPU for most models, with KleidiAI CPU routines on chips that support them
• OpenCL for MNN models on many Snapdragon chips
• On many of these phones the CPU is the faster path, which is why AutoForge measures instead of assuming

KleidiAI: ARM CPU Acceleration

Not all devices have strong GPU support. That's where KleidiAI comes in. It's ARM's CPU-side acceleration for mobile inference.

What Does KleidiAI Do?

KleidiAI uses the i8mm (integer 8-bit matrix multiply) instructions found on Armv9 CPU cores. Older cores such as the Cortex-A78 do not have them. When available, it provides:

• Faster prefill on supported chips compared to plain CPU code; gains vary by model and quantization
• GGUF models in Q4_0 format are marked as KleidiAI accelerated in the catalogue
• No GPU needed: it runs on the CPU path, so it helps on phones where no GPU path is available

When KleidiAI Is Used

KleidiAI is used automatically when:

• The phone's CPU cores support i8mm
• The model runs on the CPU path
• For GGUF models, the model is in Q4_0 format

In the GGUF backend, just load a Q4_0 quantized model and KleidiAI activates automatically on supporting chips.

How to Check Your Acceleration Path

Curious what your phone is running? AutoForge shows it:

1. Open TokForge → AutoForge (the Home screen tile, or Settings → Speed & engine)
2. On the Tune tab, What your phone can do lists the speed-up features TokForge detected
3. The Runs on row shows where the selected model runs
4. Run a speed test from the Results tab
5. Each result shows the model, the chip and the backend it ran on

Real-World Performance

Theory is fine, but what do real phones actually achieve? The live leaderboard has measured results by phone, model and backend.

Example: a recent flagship phone with Vulkan

GGUF models run on the Vulkan path here. For the actual numbers on hardware like yours, see the live leaderboard:

• Models within the GPU memory budget: TokForge runs these on the Vulkan path; whether Vulkan beats a well-threaded CPU run depends on the model, so AutoForge can measure both
• Very large and mixture-of-experts models: these stay on the CPU

Results vary widely by chipset; on many mid-range and budget devices the CPU path wins, which is why TokForge benchmarks instead of assuming.

Prefill vs Decode Speed

GPU paths can help prompt processing (prefill) more than writing the answer (decode):

• GPU Prefill: the whole input is processed in parallel, so long prompts benefit most
• GPU Decode: can be faster or slower than the CPU depending on the phone and model
• KleidiAI Prefill: faster than plain CPU code on supported chips

Does GGUF Work on Android?

Yes. GGUF is the model format for the llama.cpp engine, one of the two inference backends in TokForge, and it runs on every supported Android device on the CPU path, with OpenCL and Vulkan GPU paths where the phone supports them. The MNN engine uses its own bundle format instead; both live side by side in the app, so a GGUF file downloaded from Hugging Face works without conversion.

Don't Overthink It

Here's the reality: TokForge auto-selects for you. You don't need to:

• Memorize which path is best for your chipset
• Install special drivers
• Tweak low-level settings
• Recompile anything

Just open the app, load a model, and inference begins on the path TokForge picked for your phone. If you're curious about performance tuning, run AutoForge once. It benchmarks your phone and shows you the detailed results. That's it.

Most users never manually change backends. TokForge's automatic choice comes from routes tested on real phones in each GPU family.

Learn More

Interested in deeper optimization? Check out these related guides:

• AutoForge Guide: Automate device profiling and optimization
• TurboQuant Guide: The optional memory-saving setting for MNN models
• Full Documentation: Deep dives into MNN and llama.cpp backends