Palmyra-mini

Palmyra-mini: Small models, big throughput, powerful reasoning

Rakshith Vasudev

FIG. 01 Start from a small open base → Train three variants → Serve many requests at once → Score reasoning by sampling
Identifier
WRITER Research
Published
Sep 19, 2025
Authors
1
Type
Model
Area
Enterprise-optimized models

In one paragraph

A family of open-source small models (Apache-2.0) with a base model and two reasoning variants. They reach 93.8% on MATH500 and serve about 15.6K tokens per second on a single H200.

MATH500 Pass@1 (thinking-b)
93.80
MATH500 Pass@1 (thinking-b)
AIME24 Pass@1, avg of 64 (thinking-b)
59.42
AIME24 Pass@1, avg of 64 (thinking-b)
tokens/s on one H200 at 512-way concurrency
~15.6K
tokens/s on one H200 at 512-way concurrency
token context window
131K
token context window

FIG. 02: Mechanism

How it works

  1. Start from a small open base

    The models are fine-tuned from Qwen2.5-1.5B and released under Apache-2.0, with a 131,072-token context window.

  2. Train three variants

    A non-thinking base model handles general tasks. Two reasoning variants, thinking-a and thinking-b, are tuned to work through math and code problems step by step.

  3. Serve many requests at once

    On one H200, 512 concurrent requests with 2K-token inputs and 1.8K-token outputs complete 8.39 inferences per second, about 15.6K generated tokens per second.

  4. Score reasoning by sampling

    Each benchmark question is answered 64 times and Pass@1 is averaged over those samples. Thinking-b scores 93.80 on MATH500 and 59.42 on AIME24.

FIG. 03: Interactive figure

512 requests, one H200

Raise the number of concurrent streams and switch input length to see how total throughput and per-stream latency trade off.

Loading interactive demo…

FIG. 04: Notes

Summary

Palmyra-mini comes in three variants: a non-thinking base model and two reasoning variants (thinking-a and thinking-b). They are built for high-throughput, low-cost deployment where reasoning quality still matters. The models are fine-tuned from Qwen2.5-1.5B and released under Apache-2.0.

Results (thinking-b, Pass@1, average of 64)

  • MATH500: 93.80
  • AIME24: 59.42
  • AIME25: 49.68
  • MMLU-Pro: 55.49
  • GPQA: 42.00
  • LiveCodeBench: 34.50
  • HMMT25: 27.86
  • HLE: 5.22

Throughput (single H200, 512-way concurrency, 2K-token inputs, 1.8K-token outputs)

  • Requests: 8.39 inferences per second.
  • Tokens: about 15.6K tokens per second.
  • Inter-token latency: about 29 ms.
  • Time to first token: about 1.3 s.
  • End-to-end: about 60 s per request.

With 8K-token inputs the same GPU completes 5.45 inferences per second, about 9.8K tokens per second.

Reference