TurboQuant Explained: Leaner Context Memory on Your Phone

An optional Android setting that squeezes conversation memory so longer chats fit in the same RAM

What is TurboQuant?

TQ4 is an aggressive attention-cache (KV) quantization mode for models that run on the MNN engine in TokForge for Android. It is optional (under the default Automatic setting the app uses it only in a few specific cases), and it is not available on iPhone. It's designed for users who want to push longer conversations and bigger contexts onto their phones.

Think of TQ4 as a memory diet for your model's working state. It can trade some output quality for much smaller context memory. TQ4 stores the attention (KV) cache in a compact quantized format, meaningfully reducing the RAM it needs.

TQ4 suits users who prioritize long conversations and lean memory use, and who are willing to accept some risk to answer quality. The app itself marks the strongest memory saving as experimental.

What It Saves, and What It Costs

TurboQuant's measured win is memory, not raw speed: it compresses the attention (KV) cache, so long chats and large contexts fit in RAM that would otherwise overflow.

It is not a speed feature. On some phones and models it makes replies slower, and the strongest setting can be much slower. Turn it on when memory, not speed, is what runs out. For speeds on specific phones, see the phone-speed leaderboard.

How It Works (Simplified)

Normally the attention cache is kept at full quality. TokForge also offers a balanced mode that uses about half the memory and was near-lossless on the phones we tested.

TQ4 takes a different approach: it pushes quantization more aggressively, keeping the model's attention cache in an even more compact, highly specialized format. That shrinks the memory your conversation context occupies, but there's a tradeoff.

TQ4 achieves its memory savings by accepting lower numerical precision. On some models the difference is hard to spot. On others, especially small models, answers can get worse.

The technical benefit: TQ4 cuts context (KV) memory. The practical advice: if answers start to look wrong, switch back to a higher-quality setting.

When to Use TQ4

Best Use Cases

• Long conversations: Chats that would otherwise run out of memory
• Long prompts and documents: Content that fills a large context
• Brainstorming: Ideas and outlines where small wording differences don't matter
• Casual conversation: Everyday interaction without strict accuracy requirements

Less Ideal Use Cases

• Complex reasoning chains: Multi-step logic problems may lose precision
• Math problems: Numerical accuracy can be impacted
• Maximum-accuracy tasks: When precision is critical, leave memory saving off
• Speed-sensitive use: It can make replies slower on some phones

Rule of thumb: If long chats run out of memory and you can tolerate some loss of nuance, TQ4 helps. If you need speed or uncompromising accuracy, leave it off.

How to Enable TurboQuant

TQ4 is a setting, not a separate model. It works with the MNN models you already have, on Android:

1. Open AutoForge and pick an MNN model to tune
2. Under Memory saving (Advanced), choose Max saving
3. Tap Apply to save it

You can also set it in Settings → Speed & engine → Performance. Under Memory saving mode, the Save memory (robust) and Save memory (long context) options use TQ4, and Stretch memory for long chats uses it only for long conversations on supported models. To go back, choose Quality in AutoForge or Automatic (recommended) in Settings.

What Quality to Expect

The app describes the strongest saving honestly: it saves the most memory for very long chats, it is still experimental, and you should turn it off if answers look wrong. Quality can drop more on small models than on larger ones.

If you want a safer middle ground, the Balanced setting uses about half the memory and was near-lossless on the phones we tested. Try it first, and move to TQ4 only if you still run out of memory.

Which Models It Works With

TQ4 works with models that run on the MNN engine. There are no special TQ4 downloads: pick the model that fits your phone first, then decide whether you need the extra memory saving.

• MNN models only: GGUF models use llama.cpp, which has its own memory-saving options
• Safety limits still apply: on some models and phones TokForge keeps a safer mode even if you pick TQ4
• Android only: the iPhone app does not offer it

Not sure which model fits? The model size guide and the model pages list what runs on each amount of RAM.

Key Takeaways

• TQ4 is aggressive attention-cache quantization optimized for memory. It can trade some quality for a smaller context (KV) memory footprint.
• It is not a speed feature. It can make replies slower on some phones. The win is fitting longer chats and bigger contexts on the same phone.
• Quality can drop, especially on small models. If answers look wrong, switch back to Quality or Balanced.
• Optional. Under the default Automatic setting the app uses it only in a few specific cases. Turn it on yourself in AutoForge (Memory saving, Max saving) or in Settings. Android only, MNN models only.
• Pick your model first. TQ4 is a setting you add to an MNN model, not a model of its own.

Questions about TurboQuant? Check out the full documentation or dive into the other guides.