JSON to TOON: A Compact Format for LLM Prompts – and When to Use It

If you send the same JSON shape to an LLM over and over — agent tool calls, RAG context chunks, batch evaluations, fine-tuning samples — every repeated "name": and "price": and "in_stock": costs you tokens. Over millions of calls, those tokens add up to real money. TOON (Token-Oriented Object Notation) is an experimental format designed to attack exactly that problem: write each object key once, then list the values in compact rows below it.

One thing up front: TOON is not an industry standard, has no RFC, and is not a replacement for JSON in production APIs. Treat it as one tool in your prompt-optimization toolkit, alongside minification, schema simplification, and prompt-side dedup. This guide explains what TOON is, exactly how the savings are measured, where they show up, and where they don't.

What is TOON?

TOON stands for Token-Oriented Object Notation. It is a recent, community-driven serialization style that grew out of prompt-engineering practice. Its lineage is closest to YAML — indentation rather than braces, colons after keys — but it adds one distinctive idea: when an array contains many objects with the same shape, the keys are written once as a header and the values follow as compact comma-separated rows.

Here is the same data in JSON and in TOON, side by side:

// JSON
{
  "users": [
    {"id": 1, "name": "Alice", "age": 30},
    {"id": 2, "name": "Bob",   "age": 25}
  ],
  "settings": {"theme": "dark", "beta": true},
  "tags": ["alpha", "beta", "gamma"],
  "notes": ""
}

# TOON
users[2]{id,name,age}:
  1,Alice,30
  2,Bob,25
settings:
  theme: dark
  beta: true
tags: [alpha, beta, gamma]
notes: ""

The structural braces, the repeated quote characters around every key, and the repeated key names themselves are all gone from the users array. That is where most of the savings come from. Scalar arrays compact inline (tags: [alpha, beta, gamma]), nested objects use plain indentation, and empty strings stay explicit ("") so they can be distinguished from missing values.

Why TOON can save tokens (when it does)

To understand where savings come from, it helps to remember that LLM tokenizers do not count characters — they count tokens, which are sub-word units produced by a Byte-Pair Encoding (BPE) algorithm. A common English word like name is typically one token. The string "name": in JSON, however, is usually two or three tokens because the quote characters and the colon push it across BPE boundaries.

Multiply that across an array of 50 user objects with five fields each, and JSON's repeated key syntax has just spent roughly 50 × 5 × (2 to 3) = 500 to 750 tokens on key names alone — before any of your actual data. TOON collapses every one of those repetitions into one header row of about 10 tokens.

Important caveat: character count and token count are correlated but not identical. A 40 percent reduction in characters does not automatically mean a 40 percent reduction in tokens. TOON's comma-separated rows and bracket-and-brace tokens may tokenize differently on OpenAI's o200k_base, Anthropic's tokenizer, Gemini's tokenizer, and local model tokenizers. The two numbers tend to move in the same direction, but the ratio between them varies.

A measured comparison

Methodology: Character counts are measured directly from the JSON and TOON strings using string.length. JSON is compared in its minified form (the smallest fair baseline). Token estimates use OpenAI's o200k_base BPE tokenizer as a representative reference. Actual savings vary by tokenizer (OpenAI vs Anthropic vs Gemini vs local models), payload structure, and content. The first two rows below come from the Load Example payloads in the converter, so you can reproduce the numbers yourself.

The shape of the result is consistent: repetitive uniform arrays of objects benefit the most, nested non-repetitive configs benefit little. The 50-row case is the realistic "agent tool result" or "RAG context chunk" pattern, and it is where TOON earns its keep.

When TOON is the right choice — and when it isn't

Right choice:

• Arrays of uniform records inside LLM prompts — user lists, product catalogs, log lines, search results.
• RAG context windows where you are bumping against the model's input limit and need to fit more retrieved chunks per call.
• Agent tool inputs and outputs where the same JSON shape repeats across many tool invocations.
• Fine-tuning datasets with a repeated record schema, where every token saved per example multiplies across the whole training run.
• Batch evaluation prompts that include large fixed reference tables.

Wrong choice:

• Public REST APIs — clients expect JSON, and the broader tooling ecosystem (linters, validators, schema generators, IDE plugins) is built around it.
• Anything consumed by non-LLM code — databases, message queues, logging pipelines, telemetry systems.
• Small payloads — if your prompt only contains a few dozen tokens of structured data, the cognitive overhead of teaching your team a new format outweighs the savings.
• Payloads with no repetition — a single configuration object benefits very little, as the comparison table shows.
• Anywhere you need round-tripping through a stable, well-documented parser ecosystem — that ecosystem doesn't exist yet for TOON.

How to use the converter

The JSON to TOON Converter is the fastest way to experiment. Paste a JSON payload into the left panel and click Convert to TOON. The result appears on the right, with a savings indicator showing exact character counts and an estimated token-reduction range. If you do not have a payload handy, click Load Example to cycle through a built-in product catalog (the tabular case) and a nested service config (the non-tabular case) so you can see the difference between the two.

Once you have a TOON string, dropping it into a prompt is straightforward. With the OpenAI Node SDK it looks like this:

import OpenAI from 'openai';
const client = new OpenAI();

const toonContext = await fetch('/api/users')
    .then(r => r.json())
    .then(jsonToToon); // your conversion helper

const res = await client.chat.completions.create({
    model: 'gpt-4o',
    messages: [
        { role: 'system', content: 'You are a data summarizer.' },
        { role: 'user', content: `Summarize this user list:\n${toonContext}` }
    ]
});

If you want to validate the underlying JSON before converting, paste it into the JSON Validator first — the converter will reject invalid JSON, but the validator gives more precise error positions. For very large payloads, the JSON Minifier is also worth a look as a complementary technique: minifying alone often gets you 10–20 percent without changing the format at all.

Frequently Asked Questions