The checkpoint · Speed · Quality · What it fits on · Use it · The base model · Bench card

Ornith-1.5-122B-A10B-W4A16-Merlin

A 4-bit build of Ornith 1.5 122B, ornith-ai's fine-tune of Qwen's Qwen3.5-122B-A10B (a 122B mixture-of-experts model, 10B active per token, with vision), made to run at its full 262,144-token context on four 24 GB GPUs. It is 66 GiB instead of the original 233 GiB: the routed experts are INT4, the Gated DeltaNet projections INT8, and attention, the shared experts and the vision tower stay in BF16. It is served by the Merlin engine, a vLLM 0.32.0 fork tuned for Ampere and Ada, with the model's own MTP head for speculative decoding and the whole KV cache in VRAM. Measured on RTX 3090s, with and without P2P; the same profiles also match RTX 3090 Ti, RTX A5000 and other four-card sm_86 24 GB setups (untested), and no other GPUs have profiles for this checkpoint yet (see What it fits on).

_{numbers: Merlin 3.0.0 on 4× RTX 3090 w/o NVLINK @ 220W, P2P on, 2026-10-21, with the Ornith engine changes (uniform MoE verify on) before they merged into 3.0.0. KV pool: FP8, all in VRAM; three ~262K needle prompts sent at once, each needle found. Agents: eight 16K conversations, then eight simultaneous 512-token follow-ups, thinking off (the released 3.0.0 install, 2026-10-21). Decode: one stream, full thinking (the template's default), exactly 261,888 prompt tokens + 128 out, median of 3. Prefill: one cold 101,845-token prompt. Each figure is also in a table below.}

The checkpoint

Component	Format
Routed experts (256 per layer, 48 layers)	INT4 g128, symmetric round-to-nearest
MTP draft-head routed experts	INT4 g128, symmetric round-to-nearest
Gated DeltaNet QKV/Z and output projections	INT8 per channel
Attention (12 full-attention layers), shared experts, routers, DeltaNet a/b and conv, norms, embeddings, lm_head, MTP non-expert layers, vision tower	BF16, unchanged
KV cache	FP8 E4M3 with static scales calibrated over the 12 attention layers and the MTP layer (kv_scales.safetensors)

Files in this repository

Files	What they hold	Size
experts-int4-layerNN.safetensors (48)	Routed experts, INT4, one file per layer	55.7 GiB
model-000NN-of-00062.safetensors (62)	Attention, DeltaNet projections, shared experts, routers, norms, embeddings, lm_head, MTP non-expert layers, vision tower	9.5 GiB
mtp-routed-experts-int4.safetensors	MTP draft-head routed experts, INT4	1.2 GiB
kv_scales.safetensors, kv_scales_report.json	Calibrated static scales for the FP8 KV cache, and their report	4.9 KiB
model.safetensors.index.json	Which file holds each tensor	12.2 MiB
chat_template.jinja	ornith-ai's chat template, unchanged (byte-identical to their file); Merlin serves it as shipped	7.4 KiB
config.json, generation_config.json	Model settings (with the quantisation settings) and sampling defaults	5.6 KiB
tokenizer.json, vocab.json, merges.txt, tokenizer_config.json, processor configs	Tokenizer and image / video processor settings, byte-identical to ornith-ai's	21.8 MiB
ORNITH-MODEL-CARD.md, assets/ornith_logo.png, assets/ornith_122b_eval.png	ornith-ai's model card and its two images, unchanged	2.0 MiB
LICENSE-MIT, LICENSE-QWEN, NOTICE	The MIT text for ornith-ai's declared licence, the Apache-2.0 licence of the Qwen base model, and every change made here	15.2 KiB

_{About 66 GiB in all (130 files, listed with their SHA-256 in SHA256SUMS; this card and its images are not listed, since card edits change them). ornith-ai's card declares MIT and their repository ships no licence text, so LICENSE-MIT carries the standard MIT text; the Qwen base model's Apache-2.0 licence is in LICENSE-QWEN.}

Speed

Prompt	Decode, thinking on	Decode, thinking off	Prefill, cold	Time to first token
4K	144 tok/s	106 tok/s	4,791 tok/s	0.84 s
32K	127 tok/s	103 tok/s	4,590 tok/s	7.09 s
131K	106 tok/s	99 tok/s	2,664 tok/s	48.9 s
262K	100 tok/s	89 tok/s	1,704 tok/s	153.7 s

_{4× RTX 3090, one stream, median of 3. Prompt tokens: thinking on 4,046 / 32,566 / 130,386, thinking off 4,105 / 32,625 / 130,445, and exactly 261,888 at 262K for both; the prefill and time-to-first-token columns are the thinking-on sends. Thinking on: ≤ 256 tokens out (128 at 262K). Thinking off: the answer task, natural stop, ≤ 1,024 tokens out (128 at 262K). Prefill: prompt tokens / time to first token (the median below 262K; the cold send 1 at 262K). Below 262K every send was a cold prefill; at 262K sends 1 and 2 had nothing cached and send 3 reused 133,632 tokens. A separate 259,557-token ladder row decodes at 90 tok/s (thinking on).}

Cold prefill, one send	Prefill	Time to first token
10,157 tokens	5,148 tok/s	1.97 s
101,845 tokens	3,049 tok/s	33.4 s
261,888 tokens	1,704 tok/s	153.7 s

_{One cold request per prompt, one stream, thinking on, 8 tokens out; the 261,888-token row is the exact-262K prompt's cold send 1 (128 tokens out).}

Agents at once	Total tok/s
2	167
4	275
8	468

_{N 16K conversations, then N simultaneous 512-token follow-ups, thinking off, total tok/s while all N decode, 0 errors. Measured on the released Merlin 3.0.0 install, 2026-10-21, one fresh boot. Each overlap window was short (5.6, 7.0 and 7.8 s), so read these rows loosely.}

Prompts sent at once	Needles found	Errors
3 × ~262K (259,261–261,565 tokens)	3 / 3	0

_{Three needle prompts sent at once: the engine ran all three (running peak 3, KV pool use 100 %, 0 errors) and found every needle; first tokens came at 155, 305 and 454 s as each prefill finished, so no combined decode rate is given.}

How it was measured

• Host: 4× RTX 3090 (24 GB, sm_86) at 220 W, PCIe Gen4 with P2P, no NVLink, driver 610.43.02.
• Engine: the Merlin 3.0.0 release install with Ornith's engine changes added before they merged into 3.0.0 (the measured build also carried a prefill experiment that did not ship), one fresh boot. The release install served the same profile with the same 894,618-token pool on 2026-10-21.
• Layout: TP2 × PP2 + EP on four cards, Gated DeltaNet RecoverSSM, MTP with 3 draft tokens from the model's own head, uniform MoE verify on, FP8 KV, 1,024-token prefill chunks. Thinking in full: the template's only switch is enable_thinking.
• Decode: T=0, a needle prompt at depth; one discarded warm-up, then 3 sends per cell, median. tok/s = tokens after the first streamed event / time from first to last event.
• MTP acceptance (tokens per step, K=3, served sampling, thinking on): prose 2.39-2.48, code 2.56, GSM8K 2.58.
• Streams are client-timed. Recipe, settings and limits of the evidence: the engine's Ornith 1.5 bench card.

Quality

This checkpoint, measured on the Merlin engine:

Check	Result	Setting
GSM8K-200	190 / 200	5-shot, thinking off
Needles at 131K	10 / 10	single-occurrence needles, 130,663-token prompts, thinking off, cold cache
Needles at 262K	23 / 23	as above, 258,616-token prompts
Needle at exactly 261,888 tokens	3 / 3	thinking off
Agent-trace replay	52 / 52 tool calls valid	a recorded tool-use session replayed (27 requests); decode median 149 tok/s

_{Merlin 3.0.0 install with the Ornith engine changes before they merged, 2026-10-21; GSM8K on a separate boot of the same setup, and 190 / 200 again on the released 3.0.0 install (2026-10-21). These are checks of this checkpoint, not a matched BF16 comparison: no BF16 run of the same checks was made, so how much quality the 4-bit build keeps is not quantified here.}

The fine-tune, as reported by ornith-ai for the BF16 weights:

Area	Benchmark	Ornith-1.5-122B (BF16)
Coding	TerminalBench 2.1	81.2
Coding	SWE-bench Verified (non-hack)	84.5
Coding	SWE-bench Pro (no-hack)	76.9
Coding	NL2Repo	51.2
Reasoning	HLE (with tools)	48.0
Reasoning	SciCode	53.8
Agentic	BrowseComp	78.5
Agentic	MCP-Atlas	74.6
Agentic	Toolathlon-Verified	70.8
Agentic	ClawEval	79.4

_{A selection from ornith-ai's card, which has the full table, the harness settings and comparisons with other models. Not re-run by us or on this 4-bit build.}

What it fits on

Measured on	Context per request	KV pool	Sessions at once	Images
4× RTX 3090, 24 GB (🟢 Tested)	262,144 tokens	894,618 tokens, FP8, all in VRAM	3 × 262K by engine fit (6 × 131K by pool ratio, not run)	Up to 8 per prompt, 8 MP each

_{Measured on the reference host (2026-10-21): three ~262K needle prompts (259,261–261,565 tokens) sent at once, each needle found (whether the pool kept all three resident was not tested); an 8 MP (8,388,608 px) image with a 262K prompt left at least 1,058 MiB free on every card (checked with uniform MoE verify off; the speed figures run with it on).}

Other GPUs. The engine also ships profiles for these; the Status column says how far each has been run:

GPUs	Arch	Profile	Context fit	Status
4× RTX 3090 without P2P	Ampere, sm_86	ornith-quad-3090-nop2p	3 × 262K by engine fit (977,838-token pool; residency not run on the rented host)	🟢 Tested (rented host)
4× RTX 3090 Ti, RTX A5000 or other sm_86 24 GB	Ampere, sm_86	ornith-quad-3090	~3 × 262K (as the 3090)	⚫ Untested

_{merlin serve and ./setup.sh pick the profile from the GPUs they find. A ~ is the engine's estimate, not a measurement. Both profiles use four cards and an FP8 KV cache; the no-P2P row was measured on a rented host (4× RTX 3090, PCIe x8, 2026-10-21) with its own KV pin (977,838-token pool). There are no profiles yet for RTX 4090s, 32 GB and 48 GB cards, A100s, A30s or CMP 170HX. 🟢 measured by us · ⚫ a profile ships, not yet run by us (expected to work). Every profile: the engine's hardware matrix.}

Need	Details
Host RAM	Not measured for this checkpoint yet; the reference host has 96 GiB.
Disk	About 66 GiB for the checkpoint, plus the engine environment and caches
Engine host	Linux x86-64 (glibc 2.38 or newer), NVIDIA driver R580 or later (CUDA 13), Python 3.13, GCC/G++ 13 or newer, /dev/shm of at least 1 GB; GPUs headless with no other CUDA process. merlin doctor checks the host.

Use it

git clone https://github.com/halt95/merlin-engine.git && cd merlin-engine
./setup.sh --model ornith1.5

One command does the rest: after a small uv and Python bootstrap it checks the GPUs, driver, RAM and disk, shows the engine, package and checkpoint downloads with their sizes before fetching them, installs the prebuilt engine, downloads and verifies this checkpoint, and starts an OpenAI-compatible server on 127.0.0.1:8000. Enter takes the recommended answer at each question; --yes takes them all, --dry-run shows the plan. Run ./setup.sh again to start the server, and git pull && ./setup.sh to update the engine without downloading the model again. Requests use the model name ornith.

This checkpoint needs the Merlin engine; stock vLLM was not tested with it.

Manual install

git clone https://github.com/halt95/merlin-engine.git && cd merlin-engine
curl -LO https://github.com/halt95/merlin-engine/releases/download/merlin-v3.0.0/vllm-0.32.0+merlin.7002f56537-cp313-cp313-linux_x86_64.whl
MERLIN_WHEEL="$PWD/vllm-0.32.0+merlin.7002f56537-cp313-cp313-linux_x86_64.whl" \
  bash release/install-env.sh ./venv

venv/bin/hf download halt95/Ornith-1.5-122B-A10B-W4A16-Merlin --revision 7116c9e4c469286850f46230ce266edab8206c3f \
  --local-dir ~/models/ornith1.5
(cd ~/models/ornith1.5 && sha256sum -c SHA256SUMS)
venv/bin/python merlin_cli.py serve ~/models/ornith1.5

MERLIN_WHEEL installs the prebuilt engine (sm_80, sm_86, sm_89), checked against release/merlin-wheel.sha256; without it the installer compiles the engine instead, about 1.5 hours on 8 cores. merlin serve recognises the checkpoint, picks the largest tested layout for the GPUs it finds (or, with none, the largest untested one that fits, labelled as such) and serves on 127.0.0.1:8000. Add --dry-run to print the resolved command without starting anything. The first start compiles kernels.

Recommended settings (ornith-ai's, for this model):

Task	temperature	top_p	top_k	presence_penalty
General	1.0	0.95	20	1.5
Precise coding	0.6	0.95	20	0.0

_{ornith-ai's recommendation, with min_p=0.0 and repetition_penalty=1.0. Merlin's defaults are the checkpoint's generation_config.json (temperature 0.6, top_p 0.95, top_k 20) with presence and repetition penalties of 0 and 1.0. Reasoning tokens and the answer share the max_tokens budget.}

• Turn thinking off for a request with "chat_template_kwargs": {"enable_thinking": false}. With thinking on, a short max_tokens can end inside the reasoning with empty content.
• No effort levels: the template reads only enable_thinking, so reasoning_effort (the Qwen3.8 checkpoints' switch) has no effect, and thinking runs in full.
• Chat template: Merlin serves ornith-ai's template as shipped, with the qwen3 reasoning parser and the qwen3_coder tool-call parser. It rejects a system message that is not the first message: clients that send per-turn system notes get an error, and clients that edit the system prompt in place lose prefix-cache reuse.
• Up to 8 images per prompt, up to 8,388,608 px each. Video is off in Merlin's launcher.

The base model

Ornith 1.5 is ornith-ai's fine-tune of Qwen3.5-122B-A10B. The architecture, in short:

• Hybrid attention. Three Gated DeltaNet (linear-attention) layers for every full-attention layer: 36 and 12 of the 48 layers.
• Sparse experts. 256 experts per layer with 8 routed and 1 shared active per token: 10B of the 122B parameters do the work for each token.
• Multi-token prediction. One MTP layer, its own head (it differs from Qwen's); Merlin uses it for speculative decoding.
• Vision. A 27-layer vision transformer, unchanged from Qwen's in our checks: the fine-tune trained the language model and the MTP head.
• Thinking by default, with a non-thinking mode per request.

Property	Ornith-1.5-122B
Type	Causal language model with a vision encoder (images and video)
Parameters	122B total, 10B active (125.1B with the vision tower and MTP layer)
Layers	48: 12 × (3 × Gated DeltaNet → MoE, then 1 × full attention → MoE)
Experts	256 per layer; 8 routed + 1 shared active; expert width 1,024
Attention	32 query / 2 KV heads, head dim 256 · Gated DeltaNet: 64 V / 16 QK heads, head dim 128
Context	262,144 tokens native (ornith-ai: extensible with YaRN; this build serves 262,144)

_{Architecture from the checkpoint's config.json; the parameter count is the Hub's for ornith-ai's repository. Which weights the fine-tune changed: our reads of sampled tensors against Qwen3.5-122B-A10B.}

Other builds

Checkpoint	Base	GPUs	Decode at 262K
Ornith 1.5 (this card)	Qwen3.5-122B-A10B fine-tune (ornith-ai), 122B MoE	4× 3090	100 tok/s
Flash-Next	Qwen3.8-Flash-Next, 125B MoE	4× 3090	184 tok/s
Swift1.5	Flash-Next fine-tune	4× 3090	177 tok/s
Swift1.5-Huihui	Swift1.5, abliterated	4× 3090	189 tok/s
Qwen3.8-27B	Qwen3.8-27B, dense	2× or 4× 3090	142 / 215 tok/s
ThinkingCap	Qwen3.8-27B fine-tune	2× or 4× 3090	165 / 248 tok/s
Nemotron 3.5 Super VL	Nemotron 3.5 Super VL, 120B-A12B MoE	4× 3090	91 tok/s

_{One stream, thinking on, 261,888 prompt tokens + 128 out, median of 3; Merlin 3.0.0, 2026-10-20 bench cards. Effort as each bench card ran it: Flash-Next low; Swift1.5, Swift1.5-Huihui and ThinkingCap xhigh; Qwen3.8-27B medium. Merlin now serves medium on every Qwen3.8 checkpoint by default. Nemotron and Ornith 1.5 ran at full thinking: neither template reads reasoning_effort. Ornith 1.5 was measured on 2026-10-21 on the reference host with its engine changes (uniform MoE verify on) before they merged into 3.0.0.}

Good to know

• Every figure we measured for this checkpoint comes from one setup, 4× RTX 3090 at 220 W, PCIe Gen4 with P2P, except the no-P2P row under Other GPUs, measured on a rented host; the Other builds table names each build's card count. Hosts without P2P use the no-P2P profile (see What it fits on).
• Four cards are the minimum.
• Independent derivative: not made, reviewed or endorsed by ornith-ai or Qwen. ornith-ai's model card covers the model's training, intended use and evaluation; read it before use.

Changes

Date	Change to this repository
2026-10-21	The checkpoint as shipped: built from ornith-ai/Ornith-1.5-122B at revision dfa07bab, FP8 KV scales calibrated (kv_scales.safetensors)
2026-10-22	LICENSE-MIT added (the MIT text for ornith-ai's declared licence), then its wording corrected; SHA256SUMS updated

_{Built on the reference host; the Hub repository starts with this build. Card-only edits are not listed. Engine releases and their speed changes: the engine's CHANGELOG.}

Credits and licence

ornith-ai (the model; MIT, as their card declares, see LICENSE-MIT), Qwen (the base model, Qwen3.5-122B-A10B; Apache-2.0, see LICENSE-QWEN), the vLLM project (the engine Merlin is built on), aikitoria (peer-to-peer on consumer Ampere), Neiko2002, and CoolShades (the idle-power report behind the v2.5.2 engine fix). Upstream fixes are credited in the engine repository.