![\"Terminal](\"https://enigmus.cc/images/apple-core-ai-cover.webp\")
At WWDC 2026 Apple introduced Core AI, a new framework for running machine-learning models on Apple silicon — in Apple's words, "the inference framework powering on-device Apple Intelligence" — and, alongside it, an open-source repository that turns the framework into a practical local-LLM stack. For anyone shipping on-device language models on Apple platforms, this is the most consequential platform change since MLX appeared. Enigmus runs its inference on mlx-swift-lm directly today, and a transition to Core AI is under consideration — but not a quick one, for reasons covered below. This post is a technical first read: what the framework actually is, what is genuinely new, and where it beats or loses to building on MLX's LM layer directly.
\nCore AI is a general inference runtime, not an LLM framework. The press calls it the Core ML successor; Apple's own docs are more careful — Core ML is not deprecated, and the two are positioned side by side, with Core ML keeping classic model types and Core AI taking "the latest model architectures and inference techniques." It runs models across CPU, GPU, and Neural Engine, and its central idea is specialization: a .aimodel bundle is a device-independent source representation, exported from PyTorch via torch.export with first-class dynamic shapes. On first load it is compiled for the exact device in two phases — compute is segmented, planned, and optimized, then executable artifacts are generated for the compute units in use — and the result lands in an AIModelCache that can even be shared across apps in an app group. Ahead-of-time compilation via coreai-build shrinks, but does not remove, that first-load cost. There is a dedicated Core AI Debugger app plus Xcode instruments for profiling. Availability is 27.0 on every platform, and every entry is marked beta.
The LLM layer sits on top, in apple/coreai-models (BSD-3-Clause, first released June 2026). It contains model export recipes that convert Hugging Face checkpoints into .aimodel bundles, a Swift runtime with four engine strategies — a pipelined GPU engine, a sequential engine, a static-shape engine aimed at the Neural Engine, and a VLM variant — plus KV-cache management, samplers, tool-call parsing, and grammar-constrained generation built on XGrammar (the same constrained-decoding engine MLX adopted, so the two stacks converged there). Export recipes currently cover Gemma 3, GPT-OSS, Mistral, Mixtral, and the Qwen families for LLMs, with a longer tail of vision, audio, and diffusion models.
Three things stand out against the status quo of Core ML on one side and MLX on the other.
\nA working Neural Engine path for LLM decode. Core ML never handled autoregressive decoding well — static shapes, awkward KV-cache state, and limited ANE coverage of the decode loop. Core AI's static-shape engine targets the ANE directly, and the repository ships explicit transformer-on-ANE authoring rules: keep the whole model resident on the Neural Engine, per-head attention instead of fused SDPA, fp16 only, palettized weights. (Apple never states outright that LLM decode runs on the ANE — the claim is verified at the code level, not in marketing copy.) The one independent benchmark available (apple-silicon-llm-bench) measured Qwen3-0.6B decode on an iPhone 17 Pro at 49 tok/s on the ANE via Core AI against 39 tok/s for the legacy Core ML path — not fast in absolute terms, but running decode off the GPU entirely, which matters for sustained workloads and thermals.
\nA faster GPU engine for small models. The same benchmark measured Core AI's pipelined GPU engine at 181 tok/s steady-state on Qwen3-0.6B (iPhone 17 Pro) against 112 tok/s for MLX — roughly 1.6× once warm. Apple offers no explanation for the gap; the mechanism has to be read out of the engine's source, which describes a three-stage pipeline with rotated buffers so CPU and GPU work overlap, token sampling (argmax/top-k) executed directly on the GPU via MPSGraph compute shaders instead of round-tripping logits to the CPU, and a growing KV cache with pipelined expansion. The caveats are load-bearing: the first generation pays a cold-start cost (about 71 tok/s while kernels compile and the three-stage pipeline fills), and the advantage collapses as models grow. On an M4 Max the gap is ~2.47× at 0.6B but only ~1.05× at Qwen3-8B, where memory bandwidth rather than compute is the bottleneck — and on the Mac the same source places MLX, not Core AI, on the energy-per-token Pareto frontier. These are single-author numbers with documented session-to-session variance, so they should be read as directional. Apple itself publishes no throughput figures at all.
\nOne generation API over swappable engines. The Foundation Models framework shipped in OS 26, but OS 27 adds beta protocols — LanguageModel and LanguageModelExecutor — that let any third-party engine sit behind LanguageModelSession, the same API that drives Apple's built-in system model. coreai-models ships the reference implementation, and MLX ships its own bridge (MLXFoundationModels). An app can adopt LanguageModelSession as its only generation surface and choose the backend — Apple's system model, a Core AI .aimodel, or an arbitrary MLX checkpoint — behind one protocol, with structured output and tool calling handled by the framework.
Enigmus's current stack is MLX with mlx-swift-lm on top: the model registry, config decoding, weight loading, and the token loop, pinned to exact revisions. Measured against that, Core AI's trade-offs look like this.
\nIn Core AI's favor:
\nAgainst it:
\n.aimodel bundles through the repo's recipes, which cover a short list of architectures. mlx-swift-lm loads thousands of community-quantized checkpoints straight from Hugging Face as safetensors — no export step, no per-model recipe. For an app whose model picker is a catalog of community builds, this gap is structural.LanguageModel/LanguageModelExecutor protocols are beta, and coreai-models is not accepting pull requests — issues only. mlx-swift-lm's development happens in the open at roughly 150 commits per quarter.llm-benchmark tool but publishes no numbers; every figure above comes from one independent benchmarker.A detail worth spelling out: Apple ships Core AI LLM inference as command-line executables. coreai-models declares llm-runner and llm-benchmark as SwiftPM executable targets built on swift-argument-parser — no app bundle, no Xcode project, no entitlements dance. That makes a private, local LLM CLI on a Mac a short exercise:
# 1. Export a checkpoint to an .aimodel bundle (runs on the dev machine)\nuv run coreai.model.registry --list-models\nuv run coreai.llm.export Qwen/Qwen3-0.6B # macOS variant, 4-bit default\nuv run coreai.llm.export Qwen/Qwen3-0.6B --platform iOS # palettized 4-bit for iPhone\n\n# 2. Run it\nswift run -c release llm-runner \\\n --model path/to/exported_model_folder \\\n --prompt "Summarize the Bell inequality in three sentences."\n\n# 3. Measure it (512-token prompt, 1024 generated, 5 trials by default)\nswift run -c release llm-benchmark --model path/to/exported_model_folder\nllm-runner's flag set doubles as a map of what the stack can do: --json-schema for constrained decoding, --kv-cache-strategy, --warmup for pre-paying kernel compilation, --image for VLM input, and --inference-engine-variant to pick between the pipelined, sequential, and static-shape engines. It prints a performance summary and memory usage on exit.
A custom CLI is one SwiftPM executable that depends on CoreAILanguageModels and FoundationModels:
import FoundationModels\nimport CoreAILanguageModels\n\nlet model = try await CoreAILanguageModel(resourcesAt: modelURL)\nlet session = LanguageModelSession(model: model)\nlet response = try await session.respond(to: prompt)\nprint(response)\n(For Apple's built-in system model there is an even shorter path — macOS 27 preinstalls an fm command-line tool, with a companion Python SDK — but that drives the system Foundation Model only, not custom weights.)
The privacy read here is unusually auditable. The only network code in the entire coreai-models repository is the Hugging Face download inside the Python export tooling, which runs on the development machine; the Swift runtime contains no networking at all — it loads a local resources folder and generates. The OS-side framework is closed source, so Apple's "zero server dependencies" claim has to be taken on its word there, but the open-source layer can be checked directly.
Enigmus stays on mlx-swift-lm for now. The OS 27 floor alone settles the near term — a privacy app does not abandon its installed base for a beta OS. The catalog question matters just as much: the app's model picker is built on community MLX checkpoints fetched directly from Hugging Face, and Core AI has no equivalent supply today.
\nBut the transition is being considered seriously, and the architecture is being shaped for it. The practical reading of the Foundation Models protocols is that the generation layer of an app should become engine-agnostic: adopt LanguageModelSession semantics internally, keep mlx-swift-lm as the backend that ships, and leave the seam where a CoreAILanguageModel could slot in once OS 27 is the floor rather than the ceiling. In the meantime, the numbers worth trusting are the ones measured locally — llm-benchmark exists precisely so that the decision, when it comes, rests on device-measured tokens per second rather than anyone's blog post. Including this one.
![\"Enigmus](\"https://enigmus.cc/images/gemma4-catalogue-cover.webp\")
Enigmus 1.1.0 adds Google's Gemma 4 to the model picker across iPhone, iPad, and Mac. As with everything in the app, the new models run locally: the prompt, the generated tokens, and the chat history stay on the device, and the only network request is the one-time weight download. This post covers what changed under the hood — the runtime the models load into, the full Gemma 4 lineup now available, a compact text-only E2B checkpoint that needed a small configuration fix to run correctly, and three smaller refinements to rendering and downloads.
\nInference runs on Apple's MLX array framework, which executes the model on the Apple silicon GPU through Metal against the unified memory pool shared with the CPU. Because CPU and GPU address the same memory, weights are not copied across a bus at load time, and the practical ceiling on model size and context length is simply the machine's RAM.
\nThe model layer above MLX is mlx-swift-lm, the Swift port of mlx-lm. It owns the model registry that maps a checkpoint's model_type string to a concrete architecture, decodes each repository's config.json into a typed configuration, loads the safetensors weights, and drives the token-generation loop. Enigmus pins this stack — mlx-swift 0.31.4 and mlx-swift-lm 3.31.4 — so that every shipped model is validated against exact revisions rather than a moving dependency. Adding Gemma 4 to the catalog was possible without a dependency bump: mlx-swift-lm 3.31.4 already registers the gemma4 and gemma4_text model types.
Gemma 4 now spans the full size range in the installer — E2B, E4B, 12B, 26B, and 31B, as OptiQ 4-bit builds, with native context windows of 128K–256K tokens. These are the standard multimodal gemma4 checkpoints; the Swift port constructs only the causal-language-model decoder and does not load a vision tower, so they run as ordinary text models.
Model selection remains bounded by memory. The installer reads available RAM and free storage, lists each build with its on-disk size, marks the largest that fits as recommended, and disables anything that would exceed the memory or storage budget. A phone is pointed at an E2B or E4B build; a Mac with large unified memory can run the 26B or 31B builds. Weights are fetched from Hugging Face's CDN, the only optional network request in the app.
\nAlongside the standard OptiQ builds, 1.1.0 ships a smaller-footprint E2B option: Gemma4-E2B-IT-Text-int4, a natively text-only checkpoint quantized to affine 4-bit at group size 64. It is a ~2.7 GB download against roughly 5.2 GB for the standard E2B build — a meaningful saving on devices where the larger 4-bit weights do not fit. It is offered as the recommended model precisely in that gap: where ~2.7 GB fits but 4.0 GB does not.
Getting this checkpoint to run correctly required a configuration fix worth describing, because it is a good illustration of how quietly a positional-encoding bug can degrade output. Gemma 4's text decoder uses a partial rotary embedding: on its full-attention layers, only a fraction of each head's dimensions are rotated. The correct partial_rotary_factor is 0.25, so on the E2B's full-attention layers (7 of 35, with a 512-dimension global head) exactly 128 dimensions rotate.
This checkpoint declares its rope parameters nested under a text_config key. The pinned gemma4_text decoder reads rope parameters only from the top level of the config, and when it finds none it falls back to a default partial_rotary_factor of 1.0 — rotating all 512 dimensions. Every other shape field's default happened to match the real model, so the checkpoint loaded and produced fluent, coherent text. The damage was invisible at a glance and only surfaced on long-range retrieval: positional error under rotary embeddings grows with token distance, so short replies looked fine while facts that depended on attending across a long context drifted or dropped out.
Enigmus corrects this at download time with a small, idempotent config shim: for a gemma4_text checkpoint that has a nested text_config.rope_parameters but no top-level copy, it hoists those values verbatim to the top level before the model is loaded. The shim duplicates rather than overrides, so it resolves to the same correct model under the current decoder, under a future upstream fix to the default, and under a future decoder that reads the nested form directly — and it no-ops on any checkpoint that is already published with a flat config. A slow test loads the same weights twice, as published and with the values hoisted, and asserts the corrected build recalls a fact planted ~2,000 tokens back, outside the sliding-attention window, where only the full-attention layers can retrieve it.
One honest note on positioning: even with the rope fix, this post-training-quantized text checkpoint trades some knowledge recall against the quantization-aware-trained OptiQ builds. Side-by-side, the standard E2B build recalls specific named entities more reliably on hard prompts. The text-only checkpoint is therefore offered as the compact option for tighter memory, not as the quality pick — the standard OptiQ builds remain the default wherever they fit.
\nEquations now render as typeset math rather than raw LaTeX source — both inline and in display blocks, and progressively as a reply streams in, including math inside markdown tables and headings. For models that produce step-by-step derivations this makes the output legible as it arrives rather than after the fact.
\nEach assistant response now has a copy button that copies the raw markdown, with code blocks and equations preserved as source rather than as rendered output — so a code snippet pastes as code and an equation pastes as LaTeX.
\nModel downloads are faster and more reliable, and a paused or interrupted download can now be cleared with a "delete partial" action, which removes the incomplete weights and frees the storage rather than leaving a stalled transfer occupying disk.
\nEnigmus 1.1.0 is a free update for the existing app on iPhone (13 or newer), iPad (M1 or newer), and Mac (Apple silicon). The inference engine is identical across all three; the difference is only how much memory each has to spend on a model.
\n\n", "date_published": "2026-07-15T00:00:00.000Z", "image": "https://enigmus.cc/images/gemma4-catalogue-cover.webp", "tags": [ "AI", "privacy" ] }, { "id": "https://enigmus.cc/posts/enigmus-ai-1-0-launch-macos", "url": "https://enigmus.cc/posts/enigmus-ai-1-0-launch-macos", "title": "Enigmus 1.0 on the Mac: local LLMs on Apple silicon", "summary": "Enigmus 1.0 is now on the Mac App Store, alongside the iPhone and iPad build. The architecture is identical: language models run locally, and the prompt, the generated tokens, and the chat history…", "content_html": "![\"Enigmus](\"https://enigmus.cc/images/launch-mac-hero.webp\")
Enigmus 1.0 is now on the Mac App Store, alongside the iPhone and iPad build. The architecture is identical: language models run locally, and the prompt, the generated tokens, and the chat history stay on the machine. There is no account and no backend. Network access happens only while a model is being downloaded; inference itself runs with the machine offline.
\nInference uses Apple's MLX framework, which runs the model on the Apple silicon GPU through Metal against the unified memory pool shared with the CPU. The models are 4-bit quantized builds of the open-weight Qwen3 family. A Mac's larger unified memory raises the ceiling on both model size and context length compared to a phone. A small model is bundled in the app, so the first message works before any download. The sections below cover what the Mac build does, one screen at a time.
\nThe onboarding screen documents the runtime model up front rather than in a settings pane: conversations are stored on-device, there are no accounts, servers, or telemetry, and the weights are publicly released rather than a proprietary endpoint. These are properties of the design, not switches — with no outbound traffic during inference, there is no server-side log or profile to produce.
\n![\"Enigmus](\"https://enigmus.cc/images/launch-mac-welcome.webp\")
\n\nOn macOS the app is a standard window. Conversation history is held in a local SwiftData store and listed in a searchable sidebar; the active conversation fills the main pane, with keyboard and pointer input. Generation streams token by token from the local model, so throughput is bounded by the GPU rather than a network round-trip, and it proceeds with networking disabled. Output is rendered as markdown — headings, lists, and syntax-highlighted code blocks — and LaTeX is typeset rather than left as source; the screenshot below shows the model deriving Euler's formula with both inline and display equations. Reasoning-tuned models (Qwen3 4B and larger) emit a separate thinking trace, shown in a collapsible block ahead of the final answer.
\n![\"Enigmus](\"https://enigmus.cc/images/launch-mac-chat.webp\")
\n\nBoth feasibility and throughput are bounded by memory, so the installer is built around the machine's limits. It reads the available RAM and free storage, lists each model with its on-disk size, marks the largest build that fits the memory budget as recommended, and disables the ones that exceed available RAM or storage. In the screenshot, a 24 GB machine is pointed at an 8B 4-bit build and a 14B build is disabled for insufficient free storage. The catalog spans the bundled model through larger Qwen3 variants, including reasoning-tuned builds. Weights are fetched from Hugging Face's CDN, the only optional network request in the app.
\n![\"Model](\"https://enigmus.cc/images/launch-mac-models.webp\")
\n\nThe Mac build targets Apple silicon (M1 or newer) and runs the same inference engine as the iPhone and iPad release. It is free, with no account.
\nDownload Enigmus on the Mac App Store
\n", "date_published": "2026-06-25T00:00:00.000Z", "image": "https://enigmus.cc/images/launch-mac-hero.webp", "tags": [ "AI", "privacy" ] }, { "id": "https://enigmus.cc/posts/enigmus-1-0-launch", "url": "https://enigmus.cc/posts/enigmus-1-0-launch", "title": "Enigmus 1.0: local LLMs on iPhone and iPad", "summary": "Enigmus 1.0 is out on the App Store for iPhone and iPad. It runs large language models entirely on the device — the prompt, the generated tokens, and the chat history never leave the hardware. There…", "content_html": "![\"Enigmus](\"https://enigmus.cc/images/launch-hero.webp\")
Enigmus 1.0 is out on the App Store for iPhone and iPad. It runs large language models entirely on the device — the prompt, the generated tokens, and the chat history never leave the hardware. There is no account to create and no server to talk to. The only time the app touches the network is when a new model is downloaded; after that, it works in airplane mode.
\nInference runs on Apple's MLX framework, which executes on the Apple silicon GPU and the unified memory shared with the CPU. The models are 4-bit quantized builds of the open-weight Qwen3 family, which keeps memory use low enough that a phone can hold a useful model resident and still leave room for the rest of the system. A small model ships inside the app, so there is something to talk to within seconds of the first launch — no download gate before the first message. The sections below walk through what 1.0 actually does, one screen at a time.
\nThe first launch states the constraints up front rather than burying them in a settings screen: conversations stay on the device, there are no accounts or servers or telemetry, and the models are publicly released open weights rather than a proprietary endpoint. These aren't toggles that can be switched off later — they're a consequence of the architecture. With nothing leaving the device, there is no log to leak and no profile to build.
\n![\"Enigmus](\"https://enigmus.cc/images/launch-privacy.webp\")
\n\nThe chat itself is a normal multi-turn conversation — ask a question, follow up, change direction — except that each reply is generated locally and streams in token by token as the model produces it. Because nothing is round-tripping to a data center, the latency is whatever the device's GPU can manage, and it keeps working with Wi-Fi and cellular fully disabled. Past conversations are kept in a local SwiftData store, so history persists between sessions without ever being synced anywhere.
\n![\"A](\"https://enigmus.cc/images/launch-chat.webp\")
\n\nOn-device inference lives or dies by memory, so the model installer is built around it. It reads the available RAM and pairs each model in the list with its on-disk size, marks the largest one that comfortably fits as recommended, and greys out the ones that would not — a 30B model is simply unavailable on a phone that can't hold it. The catalog ranges from the tiny bundled model up through larger Qwen3 variants pulled from Hugging Face, including reasoning-tuned builds. Downloads come from Hugging Face's public CDN, the one piece of optional network traffic in the app.
\n![\"Model](\"https://enigmus.cc/images/launch-models.webp\")
\n\nDifferent models trade quality for speed and memory in ways that depend on the specific device, so 1.0 includes a benchmark that runs a model and reports the numbers that matter: load time, the memory the model occupies versus what the system has, and throughput split into prompt processing and generation-only tokens per second. It's a quick way to find out whether a given model is fast enough to be pleasant on a particular iPhone or iPad before settling on it for daily use.
\n![\"Benchmark](\"https://enigmus.cc/images/launch-benchmark.webp\")
\n\nReplies render as formatted markdown rather than a wall of plain text — headings, lists, and syntax-highlighted code blocks — and LaTeX is typeset properly instead of being left as raw \\( source. The screenshot below is the model working through Euler's formula, with both inline expressions and display equations laid out as actual math. Reasoning-capable models (Qwen3 4B and up) also show their thinking in a collapsible section before the final answer, which makes it easier to see how a conclusion was reached.
![\"Enigmus](\"https://enigmus.cc/images/launch-math.webp\")
\n\nEnigmus follows the system appearance, so the same conversation reflows into a dark theme without any fiddling. The layout also adapts beyond the phone: on iPad and Mac it spreads into a wider arrangement with keyboard and pointer support, and a handful of cosmetic settings — tint color, fonts, and haptics — are there for anyone who wants to adjust them.
\n![\"The](\"https://enigmus.cc/images/launch-dark-mode.webp\")
\n\nEnigmus 1.0 requires an iPhone 13 or newer (A15 Bionic or later) or an iPad with an M1 chip or newer. The Mac build is close behind. It's free, with no account.
\nDownload Enigmus on the App Store
\n", "date_published": "2026-05-28T00:00:00.000Z", "image": "https://enigmus.cc/images/launch-hero.webp", "tags": [ "AI", "privacy" ] }, { "id": "https://enigmus.cc/posts/privacy-in-ai-matters", "url": "https://enigmus.cc/posts/privacy-in-ai-matters", "title": "Why Privacy in AI Matters More Than Ever", "summary": "As AI becomes deeply integrated into daily life, a critical question emerges: what happens to all the data these systems consume? From voice assistants to recommendation algorithms, modern AI is…", "content_html": "![\"Why](\"https://enigmus.cc/images/ai-and-privacy.webp\")
As AI becomes deeply integrated into daily life, a critical question emerges: what happens to all the data these systems consume? From voice assistants to recommendation algorithms, modern AI is built on vast amounts of personal information. Understanding the privacy implications isn't optional anymore—it's essential.
\nPrivacy in AI isn't just about keeping secrets. It's about maintaining control over digital identity. AI systems process everything from browsing patterns to biometric data, creating detailed profiles that reveal far more than any single data point would suggest.
\nThe scale is staggering. A typical large language model trains on hundreds of billions of data points. Interactions with cloud-based AI services often become part of that training data—whether users realize it or not.
\nData Exposure: Centralized AI services create attractive targets. A single breach can expose millions of conversations, queries, and personal details simultaneously.
\nInference Attacks: Even anonymized data can be de-anonymized. AI itself can reconstruct personal information from seemingly harmless metadata.
\nModel Memorization: Large language models can inadvertently memorize and reproduce sensitive information from their training data, including private conversations and personal details.
\nBehavioral Profiling: Continuous interaction with AI services generates behavioral patterns that can predict decisions, preferences, and vulnerabilities.
\nThe architecture of AI deployment matters enormously for privacy. When AI runs locally on a device:
\nThis isn't about having something to hide. It's about the principle that thoughts, questions, and creative explorations belong to the individual.
\nAI capability and strong privacy aren't mutually exclusive. On-device processing through frameworks like Apple's MLX demonstrates that language models can run entirely locally, respecting the boundary between private digital life and external servers.
\nThe future of AI should enhance human capability without compromising human autonomy.
\n", "date_published": "2024-11-02T05:00:00.000Z", "image": "https://enigmus.cc/images/ai-and-privacy.webp", "tags": [ "privacy", "AI" ] } ] }