Meta's Muse Glimmer makes a 30-billion-parameter agent model fit inside a 24 GB hardware envelope. The released K-Quant-17GB weights leave room for a KV cache, a 1.8B-parameter perception encoder, and a DFlash speculative-decoding drafter. That is a meaningful systems milestone because it moves multimodal, tool-using agents from a cloud dependency into hardware that individuals and small teams can own.
Fitting is only the admission test. An always-on local agent also needs responsive generation, bounded context memory, durable state, recoverable tool calls, least-privilege access, and an auditable update path. The useful unit is therefore the entire runtime contract, not the model file.
Muse Glimmer makes that contract easier to examine because Meta published open weights, quantized variants, a model card, evaluation methodology, and concrete device measurements. The same evidence also shows where the launch claims stop and where deployers must run their own tests.
What Meta released
The Muse Glimmer model card describes a dense causal transformer with about 29.6 billion parameters, including a 1.8B ViT-G/14 perception encoder. It accepts text and images, produces text, supports a context length of 131,072 tokens or more, and uses grouped-query attention with 32 query heads and two KV heads. Its layer pattern repeats three local-attention layers followed by one global-attention layer.
Meta released the artifacts under Apache 2.0:
- Full-precision BF16 weights.
- Two approximately 4-bit quantized variants.
- A DFlash drafter head for speculative decoding.
- The perception encoder.
The model targets local agents, coding, schema-based tool calls, multimodal reasoning, synthetic data generation, and LLM-as-a-judge workloads. Meta says it was trained for multi-step reasoning, failure recovery, scaffold compatibility, and more than 100 languages.
These are model capabilities, not a complete agent product. The launch post says optimized integrations for llama.cpp, MLX, and ExecuTorch would arrive in the following days. On launch day, availability across every advertised runtime should therefore be treated as a moving integration status rather than a uniform turnkey experience.
The 24 GB envelope, line by line
At BF16, a 30B model needs more than 55 GB for weights alone. Meta's quantization changes the memory budget:
| Variant | Vendor-reported quality change | Target hardware | Deployment meaning |
|---|---|---|---|
| Full precision | Baseline | 64 GB VRAM | Fine-tuning and highest-fidelity reference |
| K-Quant-Dynamic | 0.2% degradation | 32 GB VRAM | More quality headroom with a larger device budget |
| K-Quant-17GB | 1.0% degradation | 24 GB VRAM | Leaves room for KV cache, vision encoder, and drafter |
Meta calculates degradation as an average over 15 benchmark accuracy metrics. It is a vendor aggregate, not a guarantee for every task. A one-percent average can hide larger changes in a narrow workflow, tool schema, language, image type, or long-context regime.
The 17 GB label also describes model weights, not peak process memory. Real peak usage depends on:
- Context length and KV-cache precision.
- Number and resolution of images.
- DFlash drafter memory.
- Runtime buffers and backend implementation.
- Batch size, parallel requests, and tool-scaffold overhead.
- Unified-memory pressure from other applications on a Mac.
The official GGUF repository makes the fixed part of the budget visible. The K-Quant-17GB model file is about 16.76 decimal GB, the quantized vision projector about 1.40 GB, and the DFlash drafter about 1.63 GB. Together they occupy about 19.79 GB before KV cache, runtime buffers, and the Agent host. The dynamic variant raises that fixed bundle to about 22.69 GB. This is why 24 GB should be read as a validated target configuration, not 24 GB of free context capacity.
The correct acceptance test is therefore peak resident memory at the target context and workload. A model loading successfully at 4K context does not prove that a 100K-token agent session, vision input, and speculative decoding will fit together.
Latency is part of the agent contract
An agent repeatedly alternates between model inference and tools. Slow decoding turns each plan, tool call, and recovery step into accumulated wait time. Meta pairs Muse Glimmer with a five-layer DFlash block-diffusion drafter that proposes blocks of 16 tokens, which the main model verifies in parallel.
The model card reports the following batch-one, greedy-decoding measurements:
| Device | Baseline | With DFlash | Reported speedup |
|---|---|---|---|
| Nvidia RTX 5090 | 74.9 tok/s | 233.4 tok/s | 3.1x |
| Apple M4 Max | 23.7 tok/s | 37.8 tok/s | 1.5x |
| Apple M5 Max | 26.6 tok/s | 50.2 tok/s | 1.8x |
These numbers establish that the architecture can be responsive under Meta's test setup. They do not specify prompt-processing latency, time to first token, long-context slowdown, thermal behavior, or performance across all runtime versions. Early community reports show the 24 GB fit is plausible on RTX 3090-class hardware, while DFlash speed has depended on implementation details and recent llama.cpp changes. Those reports are useful deployment signals, but they remain independent user tests rather than controlled vendor results.
The software path was still moving on August 11. Muse Glimmer support entered llama.cpp on August 10, while a DFlash GPU optimization remained a draft and a tool-call parsing fix remained open. Reproducible benchmarks should record the exact runtime commit and parser configuration. A model release and a production-ready integration can have different dates.
For an always-on agent, the useful latency metric is end-to-end task time at the 95th percentile. It should include prompt ingestion, tool execution, model retries, and recovery. Tokens per second alone measures only one segment of the loop.
Benchmark scores need a harness label
Meta reports strong results for its size class, including 75.5 on MCP Atlas, 74.6 on DeepSearch QA, 51.2 on SWE-Bench Pro, and 76.0 on SWE-Bench Verified. Muse Glimmer does not lead every row. Qwen3.6-27B scores higher on SkillsBench, OSWorld-Verified, SWE-Bench Verified, and Terminal-Bench 2.1 in Meta's table.
The accompanying evaluation methodology provides the context needed to interpret those numbers:
- Results combine internal reproductions, self-reported scores, and Artificial Analysis data depending on the benchmark.
- Agentic evaluations use particular tools, prompts, containers, judges, turn budgets, and scaffolds.
- Third-party setups may receive less tuning than their native environments.
- Several benchmarks use LLM judges, while others use deterministic grading.
- Scores are averaged over multiple runs, but the number of runs varies by benchmark.
A model score without its harness is an incomplete claim. Local deployment adds another harness: quantization, inference backend, context settings, tool definitions, and host operating system. The production question is whether the exact local stack completes your tasks within your memory, latency, safety, and recovery budgets.
Our AI agent benchmark gap framework develops this idea further: evaluation should freeze the task contract and attribute every result to the model plus harness combination.
Always-on requires durable state outside the context window
A 131K context window is a large working set. It is not durable memory.
When the process restarts, the model needs a source of truth for goals, approvals, tool results, file versions, and unfinished effects. Context compaction can preserve a summary while losing details required for audit or recovery. A local agent therefore needs explicit state layers:
- Ephemeral context: the current prompt, recent observations, and active plan.
- Durable session state: events, checkpoints, tool intents, results, and approval records.
- Long-term memory: user-approved facts and preferences with provenance, retention rules, and deletion controls.
- External system state: files, calendars, messages, repositories, and devices that may change independently.
The separation is operationally important. A retried tool call can create duplicate calendar events or messages even when the model reconstructs its conversation perfectly. Recovery needs idempotency keys, reconciliation, or a policy that pauses ambiguous actions for human review.
The same lifecycle distinctions are covered in AI agent sandbox persistence. Local execution changes where state lives. It does not remove the need to define what survives each lifecycle.
Local inference changes the trust boundary
Local execution can reduce the amount of raw personal context sent to a model provider. It also places a highly capable process next to the user's files, browser sessions, messages, calendar, and credentials.
Privacy and privilege must be evaluated separately:
- Data locality: which inputs remain on-device and which tools transmit data elsewhere?
- Credential reach: can the agent read tokens, cookies, SSH keys, or password stores?
- Action authority: which operations are read-only, reversible, or irreversible?
- Prompt-injection exposure: can an untrusted document or webpage redirect tool use?
- Auditability: can a user see what data was read and which external effects occurred?
- Update integrity: are model, quantization, runtime, and skill artifacts pinned and hash-verified?
Meta's own model card recommends system-level guardrails and human confirmation for irreversible actions. Its safety table also shows meaningful residual risk: the vendor reports a 28.4% attack success rate on Siren AgentDojo, compared with 25.6% for Gemma4-31B and 40.3% for Qwen3.6-27B in the same table. Whatever the exact benchmark transfer to production, the direction is clear. Locality cannot serve as the security control by itself.
Scaffold compatibility should also be treated as an interface contract. A tool schema that works in OpenClaw may differ in system prompt, error representation, approval flow, and observation formatting in another harness. Our guide to Agent Skills compatibility separates portable instructions from runtime-specific adapters.
A deployment acceptance matrix
Before granting Muse Glimmer persistent access to real tools, test the complete stack:
| Contract area | Minimum evidence | Example failure gate |
|---|---|---|
| Memory | Peak usage at target context, vision load, drafter, and concurrent apps | OOM, swap thrashing, or context reduction below the task need |
| Latency | P50 and P95 end-to-end task time, including tools | Tail latency breaks the interaction or automation deadline |
| Task quality | Repeated runs on frozen, representative tasks | Success rate misses the baseline or quantization loss concentrates in critical tasks |
| Tool reliability | Schema compliance, timeout recovery, idempotent retry | Duplicate or lost external effects |
| Permissions | Least-privilege scopes and confirmation for irreversible actions | Agent can access unrelated secrets or perform unapproved writes |
| Prompt injection | Untrusted web, email, and document challenge set | Content can override policy or exfiltrate data |
| State recovery | Restart and crash tests at every tool boundary | Session resumes with missing context or repeats an action |
| Updates | Pinned hashes, regression suite, rollback artifact | A model or runtime update silently changes behavior |
This matrix changes the purchase decision. The question is no longer whether a 30B model fits on a 24 GB card. The question is whether the whole local agent can pass a verifiable service contract on the hardware you actually own.
FAQ
Can Muse Glimmer really run in 24 GB?
Meta targets 24 GB VRAM for its K-Quant-17GB variant and budgets the remaining memory for KV cache, perception encoder, and DFlash drafter. Actual fit depends on context, cache precision, images, runtime buffers, and other workloads, so peak-memory testing remains necessary.
Does 24 GB mean system RAM or GPU memory?
The model card labels the target as 24 GB VRAM. Apple systems use unified memory, so available capacity is shared with the operating system and other applications. LM Studio separately lists 26 GB of minimum system memory for its package. These are different resource definitions, so compare VRAM, RAM, unified memory, and model file size separately.
Is Muse Glimmer fully offline?
The model can run without cloud inference or network access. An agent becomes fully offline only when its tools, memory, model files, telemetry, and update path also remain local.
Does local inference make an agent private and secure?
It improves data locality. Security still depends on tool permissions, credential isolation, prompt-injection defenses, logging, confirmation gates, and the behavior of any networked tools.
Which benchmark should determine deployment?
Use public benchmarks to select candidates. Make the final decision with repeated task evaluations on your exact quantization, backend, prompts, tools, hardware, and safety policy.
References
- Meta AI: Introducing Muse Glimmer
- Muse Glimmer 30B model card and weights
- Official Muse Glimmer GGUF artifacts
- Muse Glimmer evaluation methodology
- Meta AI Developer Center: Muse Glimmer documentation
- LM Studio Muse Glimmer package
- llama.cpp Muse Glimmer support
- DFlash: Block Diffusion for Speculative Decoding