AI generated
I’m currently going through Apple’s App Store approval for Inpherio Live, an app that turns an iPhone or iPad into a local network AI server with zero cloud dependencies. Most iOS LLM apps hit a hard RAM ceiling (iOS caps single-app memory at ~6GB on a 12GB iPhone). By modifying llama.cpp to stream Mixture-of-Experts (MoE) weights directly from flash storage while keeping active experts in RAM, it bypasses the memory limit entirely. Real-world benchmarks (recorded Nov 2026): * gpt-oss 120B (58.5 GB file) on a 16GB M1 iPad Pro: 4.5 tokens/s (from storage) * Qwen3 30B (17.3 GB file) on a 12GB iPhone 17 Pro Max: 10.9 tokens/s (from storage) * Qwen3 30B (17.3 GB file) on a 16GB M1 iPad Pro: 16.5 tokens/s (from storage) * Qwen2.5 1.5B on iPhone 17 Pro Max: 58.3 tokens/s (in memory) What it supports out of the box: * Drop-in API: Full OpenAI (/v1/chat/completions, /v1/responses, /v1/embeddings, /v1/audio/*) and Ollama (/api/chat) compatibility. Just change your base URL to [http://ipad.local:8080/v1](http://ipad.local:8080/v1). * Multimodal & Audio: Local Vision (Qwen2.5-VL, Gemma 3, SmolVLM2), Whisper STT, and Kokoro neural TTS. * Smart Memory Management: Auto-loads models on request, keeps up to 3 models "warm," and tracks real iOS memory limits so the OS doesn't kill the app. * 100% Private: No accounts, no telemetry, 0 bytes sent to the cloud. Streaming large MoE models from storage will be a single one-off purchase (Live Plus); everything else—including running in-memory models, embeddings, and audio—is completely free. Would love to know what models or workflows you’d test on this first once Apple clears it! Just interesting in hearing from the community. As soon as the app is approved ill update this post the website provides factul benchmarks as well as images of the process Russ