# Thinking Machines Inkling: 975B Model, Benchmarks and Mac Support

- URL: https://ai-on-mac.com/articles/thinking-machines-inkling-en/
- Language: English (United States)
- Published: 2026-07-18
- Updated: 2026-11-02
- Category: Model News
- German version: https://ai-on-mac.com/artikel/thinking-machines-inkling/
- Publisher: Julian Dominic Altmann, AI on Mac / KI auf dem Mac (https://ai-on-mac.com)

> Architecture, 45-trillion-token training, benchmarks, Tinker pricing, open-weight licensing and local inference for Thinking Machines Inkling on Apple Silicon.

**Verdict:** Inkling is not a Mac model: 975B total parameters require data-center hardware, and even the 1-bit GGUF build at 270 GB is no path for ordinary Macs. On Apple Silicon, Inkling only makes sense via Tinker or remote inference – as a broad, adaptable foundation model, not a category leader.

- **Mac verdict:** Not local: BF16 needs about 2 TB VRAM, NVFP4 about 600 GB, and even the 270 GB GGUF is no consumer path
- **Parameters:** 975B total and 41B active per token; Inkling-Small: 276B and 12B
- **Context:** Weights up to 1M tokens, but only 64K or 256K on Tinker
- **Tinker pricing (checked Oct 10, 2026):** Inkling: $1.87 / $0.374 / $4.68 per 1M tokens for prefill, cache and sampling; re-check before use
- **License:** Apache 2.0 plus an additional Acceptable Use Policy – only partly open source

![Thinking Machines Inkling with 975 billion total parameters, 41 billion active parameters and multimodal inputs](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-hero-en-light.svg)

Thinking Machines Lab released its first fully trained general-purpose model on **July 15, 2026**. **Inkling** is a multimodal Mixture-of-Experts model with **975 billion total parameters** and **41 billion active parameters per token**. It natively processes text, images and audio, supports a stated context window of up to **one million tokens**, and is available as a complete weight download.

Thinking Machines explicitly states that Inkling is **not the strongest model overall**, whether open-weight or closed. The company presents it as a broad, customizable foundation that research teams and enterprises can adapt to their own data, tools and workflows.

Inkling brings together a large sparse MoE architecture, native multimodality, controllable reasoning effort, long context and direct integration with the **Tinker** fine-tuning platform. At the same time, its scale makes self-hosting impractical for ordinary Macs and workstations.

## Mac verdict: three realistic routes

Inkling is a data-center model, not a local Mac download. The Tinker prices in this article were last checked on October 10, 2026, so re-check before booking. As of that check, Thinking Machines advised against intensive production use of the serverless beta.

**The three realistic routes on a Mac:**

- **Use Tinker** – for sampling, fine-tuning (SFT, PEFT/LoRA, RL) and distillation, with 64K or 256K context instead of the weights' nominal 1M tokens.
- **Remote inference** – via the serverless beta or another hosted provider, with model ID, context limit and token costs checked first.
- **As a teacher model** – use Inkling's document understanding for knowledge distillation into a compact on-device model, instead of running the model itself locally.

## Update: Inkling-Small released July 30, 2026

Thinking Machines released **Inkling-Small on July 30, 2026**. The smaller sibling has **276 billion total parameters and 12 billion active parameters** and, like Inkling, accepts text, images and audio with a stated context window of up to **1 million tokens**. The company describes it as comparable in capability at lower cost and latency; it is not a dense 12B model and still requires data-center hardware for complete self-hosted inference.

| Specification | Inkling | Inkling-Small |
|---|---:|---:|
| Total / active parameters | 975B / 41B | 276B / 12B |
| Context | 1M; 64K/256K on Tinker | 1M; 64K/256K on Tinker |
| Official BF16 requirement | about 2 TB aggregate VRAM | about 600 GB aggregate VRAM |
| Official NVFP4 requirement | about 600 GB aggregate VRAM | about 180 GB aggregate VRAM |

For Inkling-Small, the official model card lists **one B300 GPU in W4A4 mode** or **two H200 GPUs in W4A16 mode** as reference configurations. That is a much lower threshold than Inkling, but it is not a verified Apple-Silicon solution: a nominal 180 GB unified-memory fit would not account for runtime overhead, KV cache or an officially documented Mac inference path. For ordinary Macs, use Tinker or remote inference.

**Inkling-Small Tinker rates checked October 10, 2026:** Both 64K and 256K cost **$0.58 per million prefill tokens, $0.116 per million cached prefill tokens and $1.44 per million sampled tokens**. Training costs **$1.73 at 64K** versus **$3.47 at 256K**, according to [Tinker's machine-readable rate sheet](https://tinker-docs.thinkingmachines.ai/tinker/models.json). These are time-limited 50%-discounted Tinker training/sampling rates with no guaranteed end date, not the separate serverless beta pricing below.

## Inkling at a glance

  ![Key technical specifications of Thinking Machines Inkling](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-specs-en-light.svg)

Inkling uses 66 decoder layers and a sparse MoE feed-forward backbone. Each token is routed to **six of 256 specialized experts**, while **two shared experts** are always active. This keeps per-token computation closer to a 41B model even though the checkpoint stores nearly one trillion parameters.

The distinction between **total parameters** and **active parameters** is critical. Sparse routing reduces the computation used for each token, but it does not eliminate the need to store the expert weights. This is why the official BF16 checkpoint still requires roughly two terabytes of aggregate GPU memory despite activating only 41B parameters at a time.

## Inkling architecture

  ![Simplified Inkling architecture with text, image and audio inputs, hybrid attention and Mixture of Experts](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-architecture-en-light.svg?v=20261103a)

### A 66-layer decoder-only transformer

Inkling is an autoregressive decoder-only transformer. Text, image and audio representations are projected into a shared hidden space and processed by the same decoder. Thinking Machines describes Inkling as **natively multimodal** because text, image and audio representations run through the same decoder.

Images are handled by a hierarchical patch encoder. Small local groups of patches are folded into the channel dimension and processed through a hierarchical MLP path. Audio is converted to a mel representation, discretized into bins and embedded as model tokens. Hugging Face notes that the image representation includes a temporal dimension for video, though native video capabilities have not been formally evaluated for this checkpoint.

### Hybrid attention at a 5:1 ratio

Thinking Machines interleaves **five sliding-window layers for every global-attention layer**. Local attention lowers the cost of long sequences, while periodic global layers allow information to move across the full context. The final layer also uses global attention.

### Relative positions instead of RoPE

Inkling uses relative position representations rather than the rotary embeddings common in modern LLMs. Position information is incorporated directly into the attention logits. Thinking Machines reports better extrapolation to long sequences in its experiments. The advertised context window is up to one million tokens.

### Short convolutions

Short one-dimensional convolutions are applied after key and value projections and on the attention and MLP residual branches. These local operations can capture short-range patterns efficiently, reducing the burden on attention and expert layers.

### Multi-token prediction

The release includes an MTP drafter. Extra layers propose several future tokens, which are then verified by the main model during speculative decoding. This can increase generation speed without changing the accepted output sequence, although the drafter consumes additional memory.

## Training: 45 trillion tokens and more than 30 million RL rollouts

  ![Inkling training pipeline covering pretraining, supervised fine-tuning, large-scale reinforcement learning and Tinker customization](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-training-en-light.svg)

Thinking Machines says Inkling was trained **from scratch** on **45 trillion tokens** spanning text, images, audio and video. The data came from publicly available sources, third-party agreements and synthetic or augmented datasets. The published documentation provides source categories and general cleaning practices, but not a complete dataset inventory or precise mixture ratios.

The optimizer setup combined **Muon** for large matrix weights with **Adam** for other parameters. Weight decay was coupled to the square of the learning rate to keep weight magnitudes stable over long training horizons.

Post-training covered mathematics, agentic coding, tool use, image and audio tasks, chat and safety. An initial supervised fine-tuning bootstrap used synthetic data generated by open-weight models including **Kimi K2.5**. Thinking Machines emphasizes that this bootstrap represented only a small share of the compute; most post-training compute went into large-scale reinforcement learning.

Thinking Machines reports **more than 30 million rollouts** across two long, stable runs in synthetic and human-created environments. Aggregate reasoning performance reportedly improved approximately log-linearly throughout the process. Training ran on NVIDIA GB300 NVL72 systems.

## Controllable reasoning effort

Inkling does not expose a single fixed reasoning mode. Developers can select levels such as `none`, `minimal`, `low`, `medium`, `high`, `xhigh` and `max`, while Tinker can control effort with a floating-point value. Higher effort allows longer reasoning traces but increases output tokens, cost and latency.

Thinking Machines compares the full performance-versus-token curve rather than only the maximum benchmark point. In its own Terminal-Bench 2.1 comparison, Inkling reaches roughly the performance of Nemotron 3 Ultra while generating about one third as many tokens. This is a vendor-reported result that needs independent replication. A workflow can therefore trade speed against quality by adjusting the reasoning budget.

## Benchmarks: task results and comparison models

  ![Selected reasoning, coding and instruction-following benchmarks for Inkling and open-weight comparison models](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-benchmarks-en-light.svg)

The results vary by task: Inkling scores **97.1% on AIME 2026**, **87.2% on GPQA Diamond** and **29.7% on Humanity’s Last Exam without tools**. Tool access raises its HLE score to 46.0%.

For agentic coding, Inkling reaches **77.6% on SWE-bench Verified**, **54.3% on SWE-bench Pro Public** and **63.8% on Terminal-Bench 2.1** with the best harness. The official July 30, 2026 release post reports that the sibling **Inkling-Small** surpasses Inkling on Humanity’s Last Exam (31.6% vs. 29.7%), SWE-bench Verified (80.2% vs. 77.6%) and Terminal-Bench 2.1 (64.7% vs. 63.8%), while Inkling still leads on knowledge coverage and factuality. The family as a whole trails Kimi K2.6, GLM 5.2 and the strongest closed models.

Inkling scores **79.8% on IFBench** and **88.7% on Global-MMLU-Lite**. These results show broad instruction following and knowledge coverage, but not category leadership.

These benchmarks are not fully controlled comparisons: Thinking Machines uses externally reported Artificial Analysis scores for several evaluations and internal harnesses for others. Inkling was tested at `effort = 0.99`, temperature 1.0, and coding evaluations allowed trajectories of up to 256,000 tokens. Different harnesses, token budgets and reasoning settings can materially affect results.

## Image and audio capabilities

  ![Official audio and vision benchmark results for Inkling compared with omni models and Gemini 3.1 Pro](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-multimodal-en-light.svg)

Inkling is one of the few very large open-weight models that handles text, images and audio in a shared model. It scores **91.4% on VoiceBench**, **77.2% on MMAU** and 56.6% on Audio MC. In the official comparison it substantially exceeds Qwen3-Omni and Nemotron-3 Nano-Omni on Audio MC, while remaining behind Gemini 3.1 Pro.

For visual reasoning, Inkling achieves **73.5% on MMMU Pro Standard 10**, **78.1% on CharXiv RQ** and **82.0% on CharXiv RQ with Python tools**. These results cover visual reasoning and chart understanding; the strongest closed models remain ahead.

Inkling produces text only. It can interpret images and audio, but it does not synthesize speech or visuals. While video is represented in the architecture, Hugging Face notes that default video capabilities remain unevaluated for this release.

## Open weights, Apache 2.0 and the additional AUP

The model card lists **Apache 2.0** as the license. That normally permits commercial use, modification, redistribution and integration. However, Thinking Machines also binds access and use of the model materials to a separate **Model Acceptable Use Policy** covering harmful, deceptive, surveillance-related and high-risk applications.

"Open source" describes this combination only partly: the weights are openly downloadable, but the additional policy imposes behavioral restrictions beyond the plain Apache license. Enterprises should review the complete terms, especially when distributing derivatives or operating a public service.

Training-data transparency is also limited. Thinking Machines discloses broad source categories, the possible presence of personal and copyrighted information, and general cleaning, deduplication and filtering practices. It does not publish a dataset-by-dataset inventory, exact mixture proportions or comprehensive opt-out details.

## Hardware requirements and self-hosting

  ![Hardware and memory requirements for Inkling BF16, NVFP4 and the 1-bit GGUF quantization](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-hardware-en-light.svg)

The official BF16 checkpoint is roughly **1.9 terabytes** on Hugging Face and requires at least **2 TB of aggregate VRAM**. Thinking Machines lists eight NVIDIA B300 GPUs or sixteen H200 GPUs as reference configurations.

The official NVFP4 checkpoint lowers the requirement to at least **600 GB of VRAM**. W4A4 can run on four B300 GPUs and requires SM100 or newer hardware; W4A16 is designed for eight H200 GPUs. This remains firmly in data-center territory.

Unsloth has also published GGUF quantizations. The aggressive **UD-IQ1_S build occupies 270 GB on disk**. It makes complete local loading theoretically possible on a single high-memory workstation, but it is not a normal consumer model. Runtime state, multimodal projectors, context and the KV cache require additional memory, and a dynamic 1-bit quantization should be expected to lose substantial quality relative to BF16 or NVFP4.

## Can Inkling run locally on a Mac?

  ![Assessment of which Mac memory tiers can run Inkling locally](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-mac-en-light.svg)

For typical Macs, the answer is unambiguous: **no**. Systems with 16, 32, 64, 96 or 128 GB of unified memory are far below the smallest known complete quantization. A 192 GB Mac Studio cannot hold the 270 GB GGUF entirely in unified memory either.

A 256 GB configuration would require heavy SSD offloading. With SSD offloading, some llama.cpp setups could load it, but it would be slow at this scale and leave little space for long-context KV cache. A Mac with 512 GB of unified memory could theoretically load the 1-bit quantization in full. That does not automatically make it attractive: a one-million-token context adds large cache requirements, memory bandwidth must serve almost a trillion stored parameters, and the 1-bit model sacrifices quality.

On Apple Silicon, the practical access paths are Tinker, a remote inference provider, or a teacher model used to distill a much smaller on-device model. Hugging Face explicitly suggests using Inkling’s document-understanding capabilities for knowledge distillation into compact models.

## Tinker usage, fine-tuning and pricing

  ![Tinker pricing for Inkling at 64K and 256K context](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-pricing-en-light.svg?v=20261010c)

*Tinker training and sampling rates in USD per million tokens, checked October 10, 2026. Source: official Tinker rate sheet; serverless inference is priced separately.*

Tinker exposes Inkling in 64K and 256K configurations. The released weights support up to one million tokens, while the managed service currently exposes 64,000 or 256,000 tokens.

**Tinker pricing checked October 10, 2026 (USD per million tokens):** The **64K and 256K Inkling configurations share the same prefill ($1.87), cached prefill ($0.374) and sampling ($4.68) rates**. Only **training costs more at 256K: $11.23 versus $5.61 at 64K**. Extended context therefore affects the training price, not these inference-token rates. These are **Tinker training/sampling** charges, **not** consumer subscription fees or the separate serverless beta prices below. The 50% promotion is time-limited, with no fixed end date stated. Primary source: [Tinker's machine-readable price list](https://tinker-docs.thinkingmachines.ai/tinker/models.json).

### Serverless inference has separate beta pricing

Tinker now lists **serverless inference (beta)** for Inkling and Inkling-Small at **256K context**. This is **separate from the training/sampling pricing above**. Published beta list rates per 1 million tokens:

| Serverless inference (beta) | Input/prefill | Cached prefill | Output/sample |
|---|---:|---:|---:|
| Inkling, 256K | $1.00 | $0.17 | $4.05 |
| Inkling-Small, 256K | $0.30 | $0.06 | $1.20 |

  ![Tinker serverless beta prices for Inkling and Inkling-Small at 256K context, distinct from training rates](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-serverless-en-light.svg)

**Do not treat beta as production-ready:** Thinking Machines explicitly advises against intensive production use until serverless inference exits beta and directs interested customers to a waitlist. Check account access, model IDs, throughput and pricing before integration. [Tinker pricing](https://tinker-docs.thinkingmachines.ai/tinker/models/models_and_pricing/).

Tinker supports supervised fine-tuning, PEFT/LoRA, reinforcement learning and distillation workflows. Thinking Machines demonstrated the customization loop by having Inkling write, run and evaluate its own fine-tuning job. The example shows an agent workflow, but it does not prove that autonomous self-improvement is reliable without human oversight.

## Getting started with Transformers

The following pattern mirrors the Hugging Face example and requires very large GPU infrastructure:

```python
from transformers import pipeline

model_id = "thinkingmachines/Inkling"
pipe = pipeline("any-to-any", model=model_id)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "diagram.png"},
            {"type": "text", "text": "Explain the diagram and identify possible errors."},
        ],
    }
]

result = pipe(
    messages,
    max_new_tokens=2_000,
    return_full_text=False,
    reasoning_effort="high",
)

print(result[0]["generated_text"])
```

Inkling can also be accessed through Tinker or compatible hosted inference providers. Local GGUF inference is possible with llama.cpp when sufficient memory is available:

```bash
llama-server -hf unsloth/inkling-GGUF:UD-IQ1_S
```

This starts an OpenAI-compatible local server. The model will not be practical on an ordinary Mac.

## Safety and known limitations

Thinking Machines reports 78.0% on FORTRESS Adversarial and 95.9% on FORTRESS Benign. The company says these results reflect rejection of harmful requests while avoiding excessive refusals on benign look-alikes. StrongREJECT is reported above 98%.

The model card still warns about standard foundation-model risks: hallucinations, imperfect instruction following, degradation in long multi-turn conversations, uneven performance across languages and domains, and biases inherited from web-derived and synthetic training data. Role-play and indirectly framed harmful prompts can occasionally bypass built-in safeguards.

For public deployment, Thinking Machines recommends defense in depth: input and output classifiers, moderation models such as Llama Guard, rate limits, monitoring and human review. Medical, legal, financial and other consequential decisions should not be automated without domain-specific validation and qualified oversight.

## Strengths, limitations and target users

  ![Strengths, limitations and target users for Thinking Machines Inkling](https://ai-on-mac.com/images/thinking-machines-inkling/inkling-summary-en-light.svg)

The differentiator is not the headline parameter count: **41B active parameters affect compute per token, but 975B total parameters still drive weight storage**. Tinker's 64K/256K service limits also differ from the checkpoint's advertised 1M maximum. Published benchmark numbers are not measurements on Apple Silicon.

## Decision: evaluate online or deploy locally?

| Objective | More realistic route | Verify first |
|---|---|---|
| Compare Inkling on your prompts | Use a clearly identified hosted provider | Model ID and revision, service context cap, full token charges, data processing |
| Fine-tune Inkling | Run Tinker training/PEFT | Training rates vs sampling rates, checkpoint storage, acceptable-use restrictions |
| Use a private LLM on a 16–128GB Mac | Select a smaller model with a supported Apple Silicon runtime | Download size **and** KV cache, macOS memory pressure, actual offline behavior |

**Do not plan a single-Mac Inkling deployment from its active-parameter count.** The released quantizations still require substantially more memory than these machines have. Model files fitting on an SSD does not establish GPU inference support. Start with the [open-weight Mac shortlist](https://ai-on-mac.com/articles/best-open-llm-2026/) and size the whole workload using the [unified memory guide](https://ai-on-mac.com/articles/unified-memory-mac-en/).

## Sources

1. [Thinking Machines Lab: Inkling – Our open-weights model](https://thinkingmachines.ai/news/introducing-inkling/)
2. [Thinking Machines Lab: Inkling Model Card](https://thinkingmachines.ai/model-card/inkling/)
3. [Thinking Machines Lab: Inkling product page](https://thinkingmachines.ai/inkling/)
4. [Thinking Machines Lab: Model Acceptable Use Policy](https://thinkingmachines.ai/model-acceptable-use-policy/)
5. [Thinking Machines Lab: Training Data Documentation](https://thinkingmachines.ai/training-data-documentation/)
6. [Hugging Face: Welcome Inkling by Thinking Machines](https://huggingface.co/blog/thinkingmachines-inkling)
7. [Hugging Face: thinkingmachines/Inkling](https://huggingface.co/thinkingmachines/Inkling)
8. [Hugging Face: thinkingmachines/Inkling-NVFP4](https://huggingface.co/thinkingmachines/Inkling-NVFP4)
9. [Tinker documentation: Using Inkling](https://tinker-docs.thinkingmachines.ai/cookbook/inkling/)
10. [Tinker documentation: Models and pricing](https://tinker-docs.thinkingmachines.ai/tinker/models/)
11. [Unsloth: inkling-GGUF](https://huggingface.co/unsloth/inkling-GGUF)
12. [Thinking Machines Lab: Introducing Inkling-Small](https://thinkingmachines.ai/news/inkling-small/)
13. [Thinking Machines Lab: Inkling-Small Model Card](https://thinkingmachines.ai/model-card/inkling-small/)

## FAQ

**Can Inkling run locally on a Mac?**

No. With 975 billion total parameters, self-hosting remains practically out of reach for normal Macs and workstations. According to the model card, even Inkling-Small needs data-center hardware; the practical routes are Tinker or remote inference.

**What is the difference between Inkling and Inkling-Small?**

Inkling has 975 billion total parameters with 41 billion active per token; Inkling-Small has 276 billion and 12 billion. Both process text, images and audio with a nominal context of up to 1 million tokens.

**Is Inkling open source?**

Partly. The model card names Apache 2.0, but Thinking Machines additionally ties use to a separate Acceptable Use Policy with conduct restrictions that go beyond the license.

**Is Inkling better than Kimi K2.6 or GLM 5.2?**

Overall, no. Thinking Machines itself states the model is not leading among open or closed models. Its strengths are breadth, adaptability via Tinker and multimodal input.

## Sources

1. https://thinkingmachines.ai/inkling/
2. https://thinkingmachines.ai/model-card/inkling/
3. https://thinkingmachines.ai/model-card/inkling-small/
4. https://thinkingmachines.ai/news/introducing-inkling/
5. https://thinkingmachines.ai/news/inkling-small/
6. https://thinkingmachines.ai/model-acceptable-use-policy/
7. https://huggingface.co/thinkingmachines/Inkling
8. https://huggingface.co/blog/thinkingmachines-inkling
9. https://tinker-docs.thinkingmachines.ai/cookbook/inkling/
10. https://tinker-docs.thinkingmachines.ai/tinker/models/models_and_pricing/
