# The Small Model That Knows One Job

Tobi Lutke shared a tiny Shopify model beating a frontier model at one internal job. I wanted to know what that result really says, and whether the same idea is useful for PLAYGRND.

Author: Hrvoje Pavlinovic

Published: 2026-09-03

Canonical: https://hrvoje.pavlinovic.com/blog/the-small-model-that-knows-one-job

Tags: Small Models, Fine-Tuning, Distillation, PLAYGRND, ML Infrastructure

Use this Markdown when pasting the article into Codex, Claude, ChatGPT, a gist, or a personal runbook.

I saw [a chart from Tobi Lutke](https://x.com/tobi/status/2094808564355191249) a few days ago and spent much longer looking at it than I expected.

Shopify fine-tuned a Qwen3.5 model with roughly 0.8 billion parameters for one internal buyer-profile task. On their judge it scored 84.6, while GPT-5.6-sol at xhigh reasoning scored 83.0. The system prompt went from 9,100 tokens to 1,100, and reported capacity increased from 2 million to 72 million profiles per day. They serve it on 100 H100 GPUs.

It is a great result and a slightly dangerous chart. Remove the task name and people will read it as proof that a tiny model is smarter than a frontier model. Put the task back and the result becomes more useful: Shopify trained a specialist that has seen enough versions of one job that it no longer needs a nine-thousand-token briefing before every request.

I have no way to verify the internal benchmark. The post does not include the judge, data split, output lengths or full cost. The throughput number also belongs to a fleet of 100 H100s, around 833 profiles per second across the fleet, rather than one magical 0.8B model running under somebody's desk.

Even with those caveats, I think the direction is important. We keep using models trained to answer almost anything for small pieces of product logic that barely change. At enough volume, teaching a smaller model the job starts to make sense.

## What Shopify probably built

The public chart does not describe the whole pipeline, so this part is my reading of it rather than a claim about Shopify's exact implementation.

You begin with real inputs and a precise definition of a good output. A strong model, deterministic system or human expert produces answers. Bad answers are rejected or corrected. The accepted pairs are then used to update a smaller model.

The transfer from the stronger teacher into those examples is usually called distillation. Fine-tuning is the part where the student model's weights are changed. In practice they live in the same workflow, which is why people often use the words interchangeably.

This is well-trodden ground. [Sequence-level distillation](https://arxiv.org/abs/1606.07947) trains the student on complete outputs produced by a teacher. [Distilling Step-by-Step](https://arxiv.org/abs/2305.02301) uses teacher explanations as extra supervision. [Self-Instruct](https://arxiv.org/abs/2212.10560) grows a small seed set with synthetic instructions and filters the results.

The last few words are where most of the actual work sits. A teacher can generate thousands of examples in an afternoon, including thousands of confident versions of the same mistake. If the student's own unreviewed answers go straight back into training, the system eventually becomes very good at agreeing with itself.

I would only feed reviewed failures back into the next version. Low confidence, failed validation, user corrections and a sample of normal successes enter a queue. A human or stronger model proposes the correction, the deterministic checks run again, and only then does it become training data. That is slower than the fully automatic flywheel in a slide deck, but I might trust it.

## The PLAYGRND version of this

PLAYGRND has a small problem that looks suitable for this kind of model.

Users ask football questions in Croatian, Dalmatian and English, often without diacritics and with very creative spelling. Sometimes the whole request is half a surname or something like "a di igra onaj mali". Before the backend can do anything, the assistant has to decide whether this is a player search, match question, table, comparison, organizer command, confirmation or a request it should refuse.

Today a general model produces a typed plan and the Go backend handles the important parts. It resolves the actual player and competition, checks permissions, validates the values and asks for confirmation before a write. I would keep that boundary. A smaller model may understand the sentence, but it does not get to decide that the first Ivan in the database is probably fine.

We already have close to 150 versioned text cases. Some come from real phrasing and some are deliberately awkward. We score the model's raw plan, then score the effective result after deterministic normalization, and keep separate numbers for cost, latency and security failures.

One hundred and fifty cases are not a training dataset. They are a decent exam. I would freeze part of them, add a later holdout from new traffic, and make sure neither set appears in the prompts used to generate synthetic data.

The new examples should concentrate on the annoying edges we have actually seen: names with missing diacritics, two similar players, commands from somebody without permission, prompt injection, unsupported questions, corrections in the next message and sentences where one changed word should produce a different plan.

I would use a frontier model to create variations and candidate plans, then run each plan through the same validator as production. Short reason codes like `ambiguous_entity` or `missing_scope` are enough. There is no reason to store pages of generated reasoning.

Before generating paraphrases, I would split the data by meaning, time and, where possible, user or competition. Otherwise twenty versions of the same sentence end up on both sides of the test and we congratulate ourselves because the model recognises its siblings.

I would leave match-sheet photos for later. They bring OCR, layout, rosters, scores and image quality into the same experiment. Text routing is less impressive in a demo and much easier to measure honestly.

## A first training run should be cheap

[Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) is where I would start. It is Apache 2.0, supports tool-style outputs and is small enough that I can make several bad decisions without an expensive invoice. The model card currently reports about 0.9 billion parameters despite the name.

The first run would use BF16 [LoRA](https://arxiv.org/abs/2106.09685). I would try ranks 16 and 32, two or three learning rates and one to three epochs, while masking prompt tokens from the loss. If adapting only the attention projections stops improving, I would include MLP layers. A 2K to 4K context is already more than this routing task deserves.

For this model, an 80GB GPU does not need [QLoRA](https://arxiv.org/abs/2305.14314). Quantized training becomes useful if I move to a larger student or decide to run everything on the Mac. Full fine-tuning can wait until LoRA shows that the examples and evaluation make sense.

The model only replaces the current path if it survives the boring tests: unseen cases, permission and privacy checks, realistic concurrency, quantization on the deployment hardware and a measured fallback rate. I would run it in shadow mode first and inspect disagreements before it answers anybody.

Current rental prices make the first attempt easy to justify:

GPUMemoryListed priceMy use

RTX 409024 GB$0.69/hour on RunpodCheapest LoRA test
A100 PCIe80 GB$1.39/hour on RunpodFirst proper experiment
H100 PCIe80 GB$2.89/hour on RunpodPay more when waiting becomes annoying

At [Runpod's listed rates](https://www.runpod.io/pricing), twenty hours cost $13.80 on a 4090, $27.80 on an A100 or $57.80 on an H100. Daytona lists an H100 at [$2.27 per hour](https://www.daytona.io/pricing), Lambda at [$3.29](https://lambda.ai/instances), and an AWS H100 Capacity Block in the US at [$4.326](https://aws.amazon.com/ec2/capacityblocks/pricing/).

I would rent one A100 and give the experiment a $30 starting budget. It has enough memory to compare LoRA and full tuning without turning memory optimisation into the project. The H100 would finish faster, but capacity is not the problem here.

I expect reviewing labels to cost more than the GPU. That is fine. The model run is easy to repeat; deciding whether "a di igra onaj mali" has enough context for a useful answer is the product work.

## Where the Mac Ultra fits

The Mac Ultra is more interesting as the machine that serves the accepted model than as something we must use for training.

The [base M3 Ultra](https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac-studio-the-most-powerful-mac-ever/) has 96GB of unified memory and 819GB/s memory bandwidth. A 0.9B model is roughly 1.8GB in BF16 and below 1GB at four-bit before runtime overhead, so memory is not close to a limit.

Apple has announced an [M5 Ultra](https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/) starting at $5,499, with up to 512GB and 1.2TB/s bandwidth. I would not wait for it to test a model of this size. An existing M3 Ultra is already overqualified.

Local training is possible too. Apple showed [Qwen3.5-9B at around 180 training tokens per second on one M3 Ultra](https://developer.apple.com/videos/play/wwdc2026/233/), and [MLX-LM supports LoRA](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/LORA.md). I would still do the first repeatable runs on rented NVIDIA hardware because most recipes and debugging tools assume CUDA, and I can destroy the box when I finish.

For serving, I would fuse the chosen adapter, quantize the result for MLX and put it behind a narrow internal endpoint. PLAYGRND sends only the routing input and expects a strict plan back. Timeouts, schema validation and the current provider remain in place as fallback.

A Mac Studio at home is obviously not a data centre. The internet can disappear, macOS can restart and somebody can pull the cable. At PLAYGRND's current scale I can accept that if the fallback works. If the traffic becomes large enough for the machine to matter, moving the same model to proper infrastructure is a good problem.

## Is it worth doing?

I think it is worth a small, time-boxed experiment. A week, a frozen test set and a few rented GPU hours should be enough to learn something useful.

If the small model cannot match the current effective accuracy, or if maintaining the data becomes more work than the saved inference cost, I keep the frontier API and move on. There is no prize for owning a model.

If it works, we get lower latency, predictable cost and a router that understands a narrow mix of Croatian football language without reading the full PLAYGRND manual on every request. More importantly, we get the correction loop that can be reused for other stable tasks later.

That is what I found interesting in Shopify's result. The 0.8B number is good for attention, but the useful part is that they identified one repeated job, measured it properly and built a cheaper specialist around their own data. PLAYGRND is much smaller, but the experiment itself scales down surprisingly well.