⌘K
Noting Found

View all related articles

Most AI Token Savers Do Not Really Work, So I Built One That Measures Itself

AI Engineering 3 mins read

Table of Content

    Introduction

    If you use an AI coding agent like Claude Code or Cursor every day, you have seen your token usage climb. So a whole category of plugins showed up promising to cut it. Tools like Caveman and Ponytail went viral, some with claims like "65% fewer tokens".

    I tried them, looked closer at the numbers, and came away unconvinced. So I built a small tool called lean, with one rule for myself: never claim a saving I did not measure.

    This post is about what I learned, why most of these tools oversell, and what lean does differently. It is free and open source.

    The Problem With "65% Fewer Tokens"

    Here is the thing nobody puts on the badge. A coding agent's output is mostly code, diffs, and commands. Any honest compression tool has to leave that untouched, because changing a single character of your code is a bug.

    So what is left to compress? Only the prose. The "Sure, I'd be happy to help" and the three sentences of hedging around the actual answer. On a real coding session that prose is a small slice of the output.

    That leads to two problems:

    1. The big "65%" numbers are measured on long chat style answers, not real coding work. On actual tasks the saving drops to single digits.
    2. The plugin itself is not free. It injects instructions into the context on every turn. On a short reply or a code heavy turn there is nothing to compress, so you pay that overhead for almost no benefit. The tool can cost you more than it saves.

    I call that second one going net negative, and it is the trap most of these tools fall into.

    What lean Does Differently

    lean still gives you dense answers, but it is built around those two problems.

    It knows when to stay quiet

    lean has a gate. It only compresses when the reply is mostly prose and long enough to be worth it. On short answers, code heavy turns, or tool call turns, it does nothing and adds nothing. That is how it avoids going net negative.

    It never touches your code

    Commands, file paths, error strings, config values, and identifiers all stay byte for byte exact. Compression happens on the words around your code, never the code itself.

    It speaks Laravel

    lean keeps framework terms exact (Eloquent, Artisan, Blade, facades) and assumes you already know the framework, so it does not waste tokens explaining what a migration is. A bug answer comes back as Class::method, then the cause, then the fix.

    Here is the same N+1 question, before and after:

    1you: the /n-plus-one route uses Post::all() then reads $post->user in a map loop. why is it slow and how do i fix it?
    2
    3normal agent:
    4Sure! The reason this is slow is the classic N+1 problem. You are loading
    5all posts in one query, but then for each post you access the user
    6relationship, which fires a separate query per post. With 100 posts that
    7is 101 queries. You should eager load the relationship...
    8
    9lean:
    10N+1: `user` loaded once per post inside the map, 101 queries for 100 posts.
    11Eager load it:
    12Post::with('user')->get();

    Same fix. A third of the words. Nothing technical lost.

    It measures itself

    This is the part I actually care about. lean ships a small tool that reads your real session transcript and reports the actual tokens saved, straight from the numbers the API wrote down. No made up counter, no lifetime savings badge pulled from an estimate.

    1# real tokens for one session, plus how much of your output is even prose
    2node bin/lean-stats.mjs report ~/.claude/projects/<dir>/<session>.jsonl
    3 
    4# the honest A/B: same prompts with lean off, then on, then compare
    5node bin/lean-stats.mjs diff baseline.jsonl lean.jsonl

    If it cannot measure something, it says so instead of guessing.

    The Bigger Win Is On the Input Side

    When I shared this, a fair point came back: the real token spend is not the answer text, it is the agent's thinking and its tool calls. That is true, and it is worth being honest about.

    The answer text is the smallest lever. The bigger win is on the input side, the context that loads on every single session. Your CLAUDE.md and project notes get read every time. Compress them once and every session after loads lighter.

    lean does this too:

    1/lean-compress CLAUDE.md

    It rewrites the file densely, keeps every rule and fact, and backs up the original first. Then a verifier proves that no code block, path, URL, or command was changed:

    1node bin/lean-compress.mjs check CLAUDE.md.bak CLAUDE.md

    If anything technical went missing, the check fails and the rewrite is rejected.

    Install

    Claude Code (installs from the marketplace, run as two separate steps):

    1claude plugin marketplace add ousid/lean
    2claude plugin install lean@lean

    Any machine (auto-detects Claude Code, Cursor, Windsurf):

    1curl -fsSL https://raw.githubusercontent.com/ousid/lean/main/install.sh | bash

    You need Node 18 or newer for the stats tool. The skill itself is just a prompt, so it runs anywhere.

    Honest Limits

    Because the whole point is honesty, here are the limits up front:

    1. lean shrinks output tokens, plus input tokens for the files you choose to compress. It does not touch the model's reasoning tokens.
    2. On code heavy sessions the saving is small, and lean will show you exactly how small instead of hiding it.
    3. The gate depends on the model following the rule. It is a prompt, not a hard filter, which is exactly why the measurement tool exists, so you can check it kept its word.

    Conclusion

    Most AI token savers sell you a number they measured on the wrong workload. lean is my attempt at the opposite: a small tool that stays out of the way when it cannot help, keeps your code exact, understands Laravel, and proves its savings from your real session instead of a marketing table.

    It is open source and MIT licensed. You can find it here: github.com/ousid/lean.

    If you try it, run lean-stats on your own session and tell me what number you get. And if you have questions, drop them in the comments, I am happy to answer.

    Related Tags: laravel, php, ai, claude-code, cursor, tokens, ai-engineering, open-source

    Related Tags

    About the Author

    Oussama's Profile Picture
    Oussama
    Full Stack Web Developer | Technical Writer

    Oussama is an experienced full-stack web developer with a strong focus on Laravel. He's passionate about crafting web applications with Filament and the TALL Stack. With 8+ years of experience, and he's a dedicated open-source contributor.

    Comments

    Join our newsletter

    Subscribe to Our Newsletter and never miss our offers, latest news, Articles, etc.

    We care about the protection of your data. Read our Privacy Policy.