The JSON Killer? Introduction to Token-Oriented Object Notation (TOON)

⋅

How to slash your LLM API costs by ~50% without losing data fidelity.

Photo by Author (Google Gemini)

For the past decade, JSON has been the undisputed king of data interchange. But in the era of Generative AI, where every character counts towards a context window limit and every token cost money, JSON’s verbosity has become a liability. Enter TOON, a format purpose-built for the specific constraints of Large Language Models.

Why TOON?

The core friction point with JSON in AI is “structural redundancy.” If you send a list of 1,000 products to GPT-4, you are paying to transmit the strings “product_id”, “price”, and “description” 1,000 times.

TOON solves this by hybridizing the best parts of YAML and CSV:

  1. Header-Row Architecture: It defines keys once at the top of a list (like CSV) rather than in every object.
  2. Syntax Stripping: It removes “noise tokens” like curly braces {}, brackets [], and double quotes “” that LLMs do not strictly need to understand structure.
  3. Explicit Guardrails: Unlike YAML, TOON includes explicit array lengths (e.g., products[3]), which helps the LLM allocate attention and reduces “lost in the middle” hallucination errors.
# Standard JSON
[
  { "product_id": "SKU-992",
    "price": 1299.00,
    "description": "Pro Laptop 16GB RAM"
  },
  {
    "product_id": "SKU-884",
    "price": 49.99,
    "description": "Wireless Optical Mouse"
  },
  {
    "product_id": "SKU-771",
    "price": 15.50,
    "description": "USB-C braided cable"
  }
]
# JSON converted into TOON Format
products[3]{product_id,price,description}:
SKU-992,1299.00,Pro Laptop 16GB RAM
SKU-884,49.99,Wireless Optical Mouse
SKU-771,15.50,USB-C braided cable

Benefits: Speed, Cost, and Accuracy

  • Cost Reduction: For tabular data (lists of records), TOON creates a massive reduction in payload size. A batch processing job that costs $100/month in JSON could drop to ~$40/month simply by switching serialization formats.
  • Lower Latency: Fewer tokens mean faster “Time to First Token” (TTFT) and total generation time. The LLM has less text to parse and less text to generate.
  • Improved Reasoning: Benchmarks suggest LLMs actually perform better with TOON (73.9% accuracy) compared to JSON (69.7%) for data retrieval tasks. The cleaner signal-to-noise ratio helps the model focus on the values rather than the syntax.

Migration: Switching from JSON to TOON

Switching is relatively painless due to the emergence of libraries like python-toon. These libraries offer a familiar dump/load interface comparable to the standard json module.

Installation

pip install python-toon

Implementation Code

import toon
import json

# Your standard data payload
data = {
    "products": [
      { "product_id": "SKU-992",
        "price": 1299.00,
        "description": "Pro Laptop 16GB RAM"
      },
      {
        "product_id": "SKU-884",
        "price": 49.99,
        "description": "Wireless Optical Mouse"
      },
      {
        "product_id": "SKU-771",
        "price": 15.50,
        "description": "USB-C braided cable"
      }
    ]
}

# Convert to TOON string for the LLM Prompt
prompt_payload = toon.encode(data)

# Send to LLM (Mock)
print(f"Sending to LLM:n{prompt_payload}")

# Parse response back (if LLM replies in TOON)
response_data = toon.decode(prompt_payload)
print(f"nDecoded Response:n{json.dumps(response_data, indent=2)}") #indenting the response

Output:

C:Userssainathudata>uv run toon_test.py

Sending to LLM:
products[3,]{product_id,price,description}:
  SKU-992,1299.0,Pro Laptop 16GB RAM
  SKU-884,49.99,Wireless Optical Mouse
  SKU-771,15.5,USB-C braided cable

Decoded Response:
{
  "products": [
    {
      "product_id": "SKU-992",
      "price": 1299.0,
      "description": "Pro Laptop 16GB RAM"
    },
    {
      "product_id": "SKU-884",
      "price": 49.99,
      "description": "Wireless Optical Mouse"
    },
    {
      "product_id": "SKU-771",
      "price": 15.5,
      "description": "USB-C braided cable"
    }
  ]
}

The Challenges:

While TOON is powerful, it is not a universal replacement for JSON in all contexts.

  1. Nesting Complexity: TOON excels at “flat” lists of objects. If your data is deeply nested or highly irregular (every object has different keys), TOON loses its efficiency and reverts to a YAML-like structure, which saves fewer tokens.
  2. The “Universal” Barrier: JSON is supported by every language and database on Earth natively. TOON requires specific libraries. You should only use TOON at the “Last Mile” — converting your data just before sending it to the LLM and parsing it immediately upon return.
  3. LLM Training Bias: While current models understand TOON well, their base training data is overwhelmingly JSON. For extremely complex reasoning tasks, an LLM might still default to JSON patterns if not explicitly prompted to stick to TOON.

Strategic Workaround for Nesting Complexity:

Before diving into the workaround, let’s unpack the nesting problem to see why this strategy works.

The “Nesting Complexity” challenge arises because TOON achieves its massive compression by treating data as Tables (Rows & Columns), but complex JSON often acts like a Tree (Branches & Leaves).

When you force a Tree structure into a Table format, TOON loses its primary advantage the “Shared Header” and often becomes more verbose than JSON due to the overhead of repeatedly defining mini schemas for small, nested groups.

Why Efficiency Breaks

In a flat list, TOON declares keys once (id,name,email) and then streams values. This is highly efficient. However, if every row contains a sub-list (e.g., a User with multiple Addresses), TOON cannot use a single global header. It must either:

  1. Revert to YAML: Using indentation and keys for every item (losing the CSV-style compression).
  2. Repeat Headers: Declare a new address[N]{street,city} header for every single user, which adds massive token overhead.

Example:

# Nested JSON
[
  {
    "id": 101,
    "name": "Alice",
    "addresses": [
      { "city": "New York", "zip": "10001" },
      { "city": "Albany",   "zip": "12207" }
    ]
  },
  {
    "id": 102,
    "name": "Bob",
    "addresses": [
      { "city": "Austin", "zip": "73301" }
    ]
  }
]
# JSON converted to YAML
users[2]:
  id: 101
  name: Alice
  addresses[1]{city,zip}:
    New York,10001
    Albany,12207

  id: 102
  name: Bob
  addresses[2]{city,zip}:
    Austin,73301

Solution: The “Relational” Fix

To make TOON efficient for nested data, you should flatten your structures, similar to how you would normalize a SQL database. Instead of nesting Users inside addresses, send two separate TOON tables:

  1. users[2]{id, name}
  2. addresses[3]{city, zip, user_id}

This restores the “Single Header” advantage and can bring you back to the 40–60% token savings range.

### GENERATED TOON OUTPUT ###

users[2]{id,name}:
101,Alice
102,Bob

addresses[3]{city,zip,user_id}:
New York,10001,101
Albany,12207,101
Austin,73301,102
import json

# 1. The Input: Nested JSON (Inefficient for TOON)
nested_data = [
    {
        "id": 101, 
        "name": "Alice", 
        "addresses": [
            { "city": "New York", "zip": "10001" }, 
            { "city": "Albany", "zip": "12207" }
        ]
    },
    {
        "id": 102, 
        "name": "Bob", 
        "addresses": [
            { "city": "Austin", "zip": "73301" }
        ]
    }
]

def to_toon_table(table_name, data):
    """Converts a flat list of dicts to a TOON string."""
    if not data:
        return f"{table_name}[0]:"
    
    # Extract headers dynamically from the first item
    headers = list(data[0].keys())
    header_str = ",".join(headers)
    
    # Build the TOON Header: name[count]{keys}:
    toon_output = [f"{table_name}[{len(data)}]{{{header_str}}}:"]
    
    # Build the Rows
    for row in data:
        # Convert all values to string and join with comma
        values = [str(row[k]) for k in headers]
        toon_output.append(",".join(values))
        
    return "n".join(toon_output)

# 2. The Normalization Logic (Flattening)
users_flat = []
addresses_flat = []

for user in nested_data:
    # Create a copy of the user to avoid modifying the original input
    user_row = user.copy()
    
    # Pop the nested list (remove it from the parent)
    user_addresses = user_row.pop("addresses", [])
    
    # Add the clean parent record to the users list
    users_flat.append(user_row)
    
    # Process the children: Add the Foreign Key (user_id)
    for addr in user_addresses:
        addr["user_id"] = user_row["id"]  # <--- The Critical Link
        addresses_flat.append(addr)

# 3. Generate Output
print("### GENERATED TOON OUTPUT ###n")
print(to_toon_table("users", users_flat))
print("n" + to_toon_table("addresses", addresses_flat))

When to Avoid TOON

If your data resembles a Document Graph (e.g., HTML DOM trees, complex configuration files, or rich text with metadata), stick to JSON or YAML. TOON is strictly for Data Records.

Originally published at https://sainathudata.substack.com.

The JSON Killer? Introduction to Token-Oriented Object Notation (TOON) was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Liked Liked