Documentation

TokForge runs open large language models directly on your phone. Chat, roleplay, and group chat; on-device image generation; text-to-speech and voice input; document memory and retrieval; and an optional local HTTP control API for automation. Inference runs on the device. After you download a model, the core chat experience works with no account and no server round-trip.

The app ships on two platforms from one shared inference core: Android (Google Play open testing) and iPhone and iPad (TestFlight). Most features exist on both; the differences are called out below.

Get the app

Android. Join the open testing track on Google Play and install like any other app. Updates arrive through Play. A signed APK is also handed out through the beta program on Discord if you prefer to sideload.

iPhone and iPad. Install Apple TestFlight, then open the TokForge invite link and tap Install. iOS is in beta on TestFlight; there is no App Store listing yet.

On first launch the app profiles your device and recommends a model to match its memory and chipset. Download the model, open a chat, and start. See the Quickstart for a screen-by-screen walkthrough.

What you need

• Android: Android 8.0 or newer (API 26), ARM64. 8 GB of RAM is comfortable; 4 GB runs the smallest models. Room for at least one model (a few hundred MB to several GB).
• iPhone and iPad: a recent iOS or iPadOS release on an A15 or newer chip. 6 GB devices run small models; 8 GB and 12 GB devices unlock larger ones.
• The app download itself is small (under 50 MB). Models are downloaded separately from inside the app and can be deleted anytime.

What you can do

Chat, roleplay, and group chat Android and iOS

• Chat with streaming tokens and a live tok/s counter, markdown and code rendering, and collapsible reasoning (<think>) blocks for models that support it.
• Roleplay with characters, and group chat with two or more characters taking turns. Each member can have its own read-aloud voice, mute state, and talkativeness.
• Regenerate replies, browse alternate responses, and edit messages in place.

Characters, personas, and lorebooks Android and iOS

• Character hub with built-in personalities plus import of your own cards (Character Card V2 PNG or JSON, and common flat card formats).
• chub.ai import: search and pull character cards from inside the app.
• Personas describe who you are to the model, applied per conversation.
• Lorebooks / world info inject the right background details only when their keywords come up.

Memory, RAG, and knowledge Android and iOS

• Persistent memory: the app extracts facts in the background so a character remembers you across conversations.
• Document RAG: attach PDFs, DOCX, EPUB, or text and the model answers grounded in them, using hybrid keyword plus semantic retrieval.
• Knowledge graph: extracted facts and their relationships, browsable and editable.
• Opt-in web search: off by default. When you turn it on, the model can pull in fresh results for a turn.

Images, voice, and vision Android and iOS

• On-device image generation from a chat prompt. Android uses Stable Diffusion routes (with optional NPU acceleration on supported hardware); iPhone uses Apple CoreML on the Neural Engine, plus a Metal path on 12 GB devices. Reference images, LoRAs, styles, and batches are supported where the device allows.
• Text-to-speech to hear replies aloud, with high-quality offline voices, plus voice cloning from a short sample.
• Voice input: speak instead of typing, transcribed on-device.
• Vision: send a picture and ask about it.

Engines and models Android and iOS

• Android engines: llama.cpp (GGUF), MNN (CPU and GPU), and a remote-API option to point at your own endpoint. The app picks a sensible default automatically.
• iPhone engines: llama.cpp with Metal (primary), MNN, MLX (the fastest big-model route on 12 GB devices), and a CoreML / Neural Engine lab route. You can also point at your own remote endpoint.
• Model downloader with Hugging Face search built in: find, download, and switch models from inside the app, with progress, resume, and disk-space checks.

ForgeLab, leaderboard, backup Android and iOS

• ForgeLab benchmarks a model on your device (prefill and decode tok/s, time-to-first-token, memory) and can auto-tune settings for your hardware.
• Leaderboard: opt in to submit a privacy-clean benchmark row and see how devices compare at leaderboard.tokforge.ai. Submissions carry no IPs and no chat content.
• Backup and restore your characters, conversations, and settings to a file.

Which model fits your device

TokForge recommends a model based on your device on first launch, so you do not have to choose blind. The tables below are the plain-English version of that guidance. Bigger models are smarter but need more memory and run slower. The app never silently overloads a device: if a model would run out of memory it refuses and offers a Load anyway option, and a model that only pages slowly from storage is labelled runs, slower rather than hidden.

Android, by RAM

Device RAM	Comfortable pick	Notes
4 GB	Qwen3-0.6B (about 0.35 GB)	Small and fast. Good for quick chat on entry devices.
6 to 8 GB	Qwen3-1.7B (about 1 GB) or a 4B (about 2.3 GB)	The everyday sweet spot. On-device image generation runs here (slower on 8 GB without GPU acceleration).
12 GB	An 8B (about 4.8 GB) or 9B (about 5.4 GB)	Noticeably stronger answers with headroom for longer chats.
16 GB and up	A 14B (about 8.5 GB)	Best quality for flagship phones. Larger models are available for the curious, slower.

iPhone and iPad, by RAM

Device RAM	Comfortable pick	Notes
6 GB (A15 / A16)	A 3B (fast) or Qwen3-0.6B	Tightest tier. Small models only. Image generation uses the CoreML / Neural Engine path.
8 GB (A17 class)	A 3 to 4B, or one 8 to 9B option	The sweet spot: small models are quick, and a select larger model runs.
12 GB (A19 class)	Up to a 9B dense model, or a 30B mixture-of-experts (paged)	Flagship tier. MLX gives the fastest big-model route; full image generation (LoRA, img2img) unlocks.

Rule of thumb: pick the model the app suggests first, then try the next size up if it stays responsive. A 4B or smaller model is the reliable everyday choice on almost any modern phone.

Dig deeper