You can check Niro's help yourself, in four ways: read the line it prints at the end of each task, look at the record of which tools your agent asked, add up the last 7 and 30 days with niro impact, and run the same task with and without Niro. This page says what each number means, how it is worked out, and which numbers are estimates.

An everyday comparison: a taxi meter tells you the fare and the distance, and it is right about both. It does not know what the bus would have cost you. The end-of-task line is the same: its counts are measured, and its "instead of" figures are an estimate of the other route.

1. The end-of-task line

At the end of a task your agent calls a tool, mark_task_complete, and copies the line it gets back into its reply. It looks like this:

[NIRO-CLI] Listed the 4 module-detection paths that reach the changed premake reader · 2 calls instead of ~3 searches · ~1k tokens instead of ~5k · this session: ~39k tokens instead of ~96k

Example from a real session on Niro's own code. Your numbers will depend on your code and your tasks. It is copied exactly as the engine printed it. Of the 38 end-of-task lines with a token saving recorded on one developer's computer over two days, it ranks 29th by the size of that saving: higher than most, not the largest.

Part What it is Measured or estimated
"Listed the 4 module-detection paths ..." Your agent's own sentence about what Niro found, cut to 80 characters Written by the agent
"2 calls" How many Niro tool calls answered in this task Measured, counted by the engine
"instead of ~3 searches" How many extra searches and file reads the agent thinks it would have needed Estimated by the agent: it types this number itself
"~1k tokens" The size of every answer Niro returned in this task Measured characters, turned into tokens at 2.09 characters a token
"instead of ~5k" What finding the same things by hand would have cost Estimated by the engine, as below
"this session: ..." The same two token figures, added up over the session so far Same as above; shown from the second priced task

Where the 2.09 comes from. It was measured against the model's own token counts, in Keycloak A/B transcripts, on results of more than 2,000 characters each. Over 69 Niro answers the median was 2.09 characters a token (middle half 1.83 to 2.25). Over 367 shell-tool results it was 2.10. So one rate fits both sides of the line.

How "instead of ~Y" is estimated

The engine prices the by-hand route the way agents with only shell tools were measured to work on Keycloak: they search for the name they were asked about, then read about 40 lines around each place that matters, and almost never open a whole file. So for each task it counts:

Only what you were shown is priced. A page of an answer that was cut off adds nothing, and a short answer= form is priced for what it showed.

Honest limits of the estimate:

How the two token figures are made

What the answers cost. The engine adds up the characters of every Niro answer in the task, divides by the characters-a-token rate above, and rounds the result for the line.

The by-hand estimate. The engine looks only at the answers that showed you places (file:line). It prices one read of the lines around each place, counting nearby places in one file once, from each file's real size and line count. It adds one search for each name the question asked about, and one for what a name search cannot reach. It turns those characters into tokens at the same rate. An answer that showed no places, such as a count or a list of names, adds nothing.

The calls and the agent's guess. The line shows how many calls answered. It shows the agent's guess of searches only when the guess is more than the calls made.

How the searches figure is kept. The agent's number is stored with the task in the engine's record of the session (niro journal). It is not added up: the session total in the line covers tokens only.

How the session total is kept. Each task adds its two token figures to the session's running totals, in memory. A place or a search already priced earlier in the session is not priced again, because whoever made that read by hand would still have it. The total is shown from the second priced task on.

When the line is shown, and when it is not

What you see Why
The line Niro answered at least one call in this task
"N calls" with no "instead of" The agent's guess was not more than the calls made, so no saving is claimed
No token pair The by-hand estimate was not more than what Niro returned, or the two round to the same figure
No session total It is the first priced task of the session, or the session grew past 20,000 files or 100,000 places, after which a total would be incomplete
"Not consulted: this turn used file reads instead" The graph was there, and the agent searched the files itself
"Nothing to report: ..." No Niro call answered in this task, so the agent should print nothing
Nothing at all The line is switched off (niro mcp receipt --off), or the agent did not call mark_task_complete

In Claude Code, a forgotten line is asked for once. A Stop hook (a small command Claude Code runs when the agent is about to stop) checks the reply. If Niro answered and the line is missing, it asks the agent, once, to finish. It never prints anything itself, and it lets the agent stop if anything goes wrong.

The line is not saved. It is shown once, in your agent's reply. See FAQ: feedback and receipts.

2. The record of what your agent asked

niro journal lists the sessions the engine recorded and not yet sent, with each tool call, whether it worked, and how long it took:

niro journal --limit 3

9 session(s) waiting, 31 tool call(s) in total
  adb7d35bb51e  5 call(s)  started 2026-11-10T19:45:35+00:00  ended 2026-11-10T19:45:53+00:00
      get_impact_analysis          ok        122ms  {"http_path": "/api/shipments/{id}", ...
      resolve_name                 ok         19ms  {"name": "ShippingMain.shipment"}
      ...
      find_api_endpoints           ok         17ms  {"limit": 20, "offset": 0, "path_pattern": "shipments"}
      ...

This is the place to see which tools your agent really uses, which fail, and which are slow. The record is sent to Niro (by the background watcher, or niro journal-send), and a sent session leaves this list. What it holds, and what it does not, is in What leaves your machine.

The lines themselves are not kept, but their figures are. Each task's counts, tokens and the agent's sentence stay on this computer after the session is sent, and niro impact adds them up (next section).

3. What Niro did for you over time: niro impact

niro impact adds up the tasks on this computer for the last 7 and the last 30 days. It reads only the engine's record on this computer: nothing is sent, and no model is called.

Each window prints one figure per line, then the projects, then the good moments:

Line What it is Measured or estimated
The record on this computer starts on ... Shown only when the record starts after the window's first day, so "last 30 days" never quietly means fewer days. It sits directly under the window's heading Measured
Tasks that used Niro: "N of M" Tasks with at least one Niro answer, of all the tasks in the window Measured
Number of calls Niro answered The Niro answers in those tasks Measured
Tokens used: "~X tokens instead of ~Y (estimated)" X is what the answers cost; Y is the by-hand estimate of section 1, added up over the tasks that had both X measured, Y estimated
Searches and file reads avoided: "~N" The count the agents typed in mark_task_complete, added up. The kinds of call they named are kept in --json Estimated by the agents; nothing checks them
Look-ups avoided: "~N" The searches and file reads the graph answers stood for, less the Niro calls: the size of the work. See below Counted
Round trips avoided: "at least ~M" The searches your agent would have had to wait for one after another, plus reading the files, less the Niro calls: a floor on the waiting. See below Counted
Answers by kind The answers, grouped by the kind of question: impact check, caller lookup, trace of what a method calls, route and handler lookup, event trace, database lookup, type or name lookup, exception trace, text search, source read Measured
Median answer time The middle of the answer times recorded in the window Measured
By project The tasks, answers and tokens per project, by the project's name As above
Good moments The tasks with the best measured saving, tokens first and then calls, in the agents' own words, largest saving first: 3 for a window of up to 7 days, 10 for a longer one. A task rated not helpful or not used is never one Chosen by numbers, never by the words
Some figures cover only tasks recorded since ... Shown at the end of a window when a row covers fewer of its tasks, because some were recorded by an earlier version. The row itself shows its number as it is Measured
Graph kept current Per project, the files the graph holds and when it was last updated. Per window, the graph updates (full and incremental), the files they rebuilt, and the median update time Measured

An example. This is niro impact exactly as it printed after five real tasks on two projects on one computer (four on Keycloak, one on a small Java project), unedited:

What Niro did for you, from the record on this computer. Nothing here is sent anywhere.

Last 7 days (since 2026-11-04):
  The record on this computer starts on 2026-11-11.
  Tasks that used Niro: 5 of 5
  Number of calls Niro answered: 7
  Tokens used: ~12k tokens instead of ~60k (estimated)
  Searches and file reads avoided: ~29
  Look-ups avoided: ~238
  Round trips avoided: at least ~5
  Answers by kind: 3 caller lookups, 2 route and handler lookups, 1 impact check before a change, 1 text search.
  Median answer time: 70 ms

  By project:
    keycloak: 4 task(s) with Niro, 5 answer(s), ~8k tokens instead of ~56k (estimated)
    java-mono: 1 task(s) with Niro, 2 answer(s)

  Good moments, in the agents' own words, largest saving first:
    2026-11-11  keycloak: Counted the callers of the error helper across 45 files (~5k tokens instead of ~47k, 2 call(s) instead of ~10 searches)
    2026-11-11  keycloak: Listed the 9 direct callers of the lookup, 2 of them in tests (~1k tokens instead of ~7k, 1 call(s) instead of ~6 searches)
    2026-11-11  keycloak: Found where the token lifespan constant is read (1 call(s) instead of ~3 searches)

Last 30 days (since 2026-10-12):
  The record on this computer starts on 2026-11-11.
  Tasks that used Niro: 5 of 5
  Number of calls Niro answered: 7
  Tokens used: ~12k tokens instead of ~60k (estimated)
  Searches and file reads avoided: ~29
  Look-ups avoided: ~238
  Round trips avoided: at least ~5
  Answers by kind: 3 caller lookups, 2 route and handler lookups, 1 impact check before a change, 1 text search.
  Median answer time: 70 ms

  By project:
    keycloak: 4 task(s) with Niro, 5 answer(s), ~8k tokens instead of ~56k (estimated)
    java-mono: 1 task(s) with Niro, 2 answer(s)

  Good moments, in the agents' own words, largest saving first:
    2026-11-11  keycloak: Counted the callers of the error helper across 45 files (~5k tokens instead of ~47k, 2 call(s) instead of ~10 searches)
    2026-11-11  keycloak: Listed the 9 direct callers of the lookup, 2 of them in tests (~1k tokens instead of ~7k, 1 call(s) instead of ~6 searches)
    2026-11-11  keycloak: Found where the token lifespan constant is read (1 call(s) instead of ~3 searches)
    2026-11-11  java-mono: Listed the routes each service serves and which service calls which (2 call(s) instead of ~8 searches)

Graph kept current, per project:
  java-mono: 155 files indexed in 1 folder(s), last updated 2026-11-11 10:43 UTC.
  keycloak: 9,099 files indexed in 1 folder(s), last updated 2026-11-10 19:27 UTC.
  typescript-mono: 366 files indexed in 1 folder(s), last updated 2026-11-10 19:28 UTC.
  Graph updates:
    Last 7 days: 1 graph update (1 full, 0 incremental), 155 files rebuilt, median update time 3.6 s.
    Last 30 days: 1 graph update (1 full, 0 incremental), 155 files rebuilt, median update time 3.6 s.
    Updates are recorded from 2026-11-11.

How "look-ups avoided" is counted: the size of the work. For each Niro answer that walks the graph (a call chain, the callers of a method, a blast radius or impact check, a route, handler, event or database lookup), the engine counts one search for each method the answer expanded or found, and one file read for each different file it covered. Each method and each file counts once in a task: when a second answer covers a file the first already covered, the file is not counted again, and asking the same question twice counts once. It then takes away the Niro calls themselves, because each of those is a call too, and never counts below 0 for one task. A text search or a source read adds nothing, because by hand it would be the same single call. The count is the work the answers stand for, not a claim that your agent would have made every one of those calls: an agent working by hand often stops early.

A page counts only what it shows, in both rows. When an answer is cut by limit, is a later page asked with offset or from_line, or is too long for one reply and shown a window of lines at a time, the engine counts only the files the page shows and the methods whose names it prints, the same way the end-of-task line prices only what was shown. For example, the callers of a method with limit: 5 count the files of those 5 callers, not the files of every caller. An answer shortened with answer=count is not a page: it reports on everything the full answer covers, so it counts all of it.

How "round trips avoided" is counted: a floor on the waiting. A round trip is one call your agent makes and then waits for. Working by hand, an agent can put the names of one level into one search (all the callers of a method, say), but it cannot search the next level before that search comes back, because it does not know the names yet. So a walk three levels deep takes at least three searches one after another, then one more round to read the files. For each Niro answer that added at least one method or file new to the task, the engine counts the deepest level the answer walked, plus 1 for reading the files, minus 1 for the Niro call itself. An answer that adds nothing new to the task counts -1: the Niro call was a round trip of its own, and it stood in for none. A list of a method's direct callers counts 1; a blast radius three levels deep counts 3; asking the same question again in the task counts -1; a later page of a list already paged in the task counts -1 too, or only the levels deeper than any page before it, since by hand the names of one level go into one search; a text search or a source read counts 0. Never below 0 for one task. It says "at least" because an agent working by hand often needs more than one search for a level, so the real number of waits is this or more. In the example above, the 5 is 1 (a list of direct callers) + 2 (a blast radius three levels deep, 3, then the callers of the same method asked again in the task, -1) + 1 (a list of callers that found none) + 1 (a list of routes); the text search counts 0.

The agents' words are checked again before they are shown. They were masked when they were stored; niro impact masks them again, and replaces anything that looks like a file path, a piece of code or a network address with [path], [code] or [address]. Only the project's name is shown.

Options: --days N shows one window of N days (1 to 365) instead of both; --project <name> shows one project; --json prints the same report as JSON.

How far back it goes. The record keeps the newest 20,000 tasks, the newest 100,000 answers and the newest 1,000 graph updates. When the record starts after a window's first day, the report says on which day it starts, directly under the window's heading. Tasks recorded by an earlier version count in the numbers, with no words. When the look-ups, round trips or answers-by-kind rows cover fewer of the window's tasks because of them, each row shows its number as it is, and one line at the end of the window says "Some figures cover only tasks recorded since" the day they start. A graph update is recorded only when a build changed the graph: a build that failed, was refused, or found nothing changed is not one.

On a team server, this release shows only what was recorded on your own computer. Answers from a teammate's shared server are recorded on that server.

niro ab, an account-wide on and off test, is hosted Niro only. See Features of hosted Niro.

4. Judge it yourself: a small A/B

The most honest test is to run the same task twice, once with Niro and once without, and compare.

How

  1. Pick 3 to 5 real tasks from your own work, with an answer you can check: "list every caller of X", "which tests reach Y", "who writes table Z".
  2. Write down the right answer first, by hand, so neither run grades itself.
  3. Run each task twice in a fresh session: once as you normally work, and once with Niro switched off for that session. In Claude Code, start the second run with claude --strict-mcp-config --mcp-config '{"mcpServers":{}}', which starts it with no MCP servers.
  4. Keep everything else the same: the same model, the same prompt, the same agent version.

Which numbers to compare

Number Where to read it What it tells you
Correctness: items found, items invented Your answer key Whether the answer was complete and true. This matters most
Cost, and tokens: cache writes, cache reads, output Claude Code's /cost, or the result event of claude -p --output-format stream-json What the run really cost
Tool calls The transcript How much searching and reading it did
Wall time A clock, or the result event's duration_ms How long you waited

Report all of them together. Cost alone hides why it moved: in long sessions, cache reads (the conversation re-read on every turn) are most of the tokens.

How we run ours, and what to keep in mind when reading the results, is in Tokens, speed and accuracy.

Check that an answer was complete

A list of callers can be checked against a text search. On Keycloak, ErrorResponse.error has two overloads (two methods with the same name and different parameters), and a grep for the name finds the calls to both. So ask for both:

niro ask get_reverse_call_chain "method_fqn=org.keycloak.services.ErrorResponse.error(String, Response.Status)" answer=locations limit=300
# 100 callers, in 45 files

niro ask get_reverse_call_chain "method_fqn=org.keycloak.services.ErrorResponse.error(String, Object[], Response.Status)" answer=locations limit=300
# 3 callers, in 3 files

grep -rln --include=*.java "ErrorResponse.error(" .
# 47 files

Keep the file part of each caller line, put the files of both answers in one sorted list, and compare it with grep's sorted list (comm -3 a.txt b.txt prints lines in only one of them). Here both lists had the same 47 files. A file only grep finds is a caller Niro missed, or a mention that is not a call; open it and see which.

It does not always agree, and that is the point of checking. For JpaUserProvider.getUserByEmail, Niro answered "0 callers" while a grep for getUserByEmail( found calls in many files. The callers use a child interface's name; asking UserProvider.getUserByEmail gives 17. That is a known problem, listed in FAQ: wrong or missing answers.

For a class or interface, compare get_class_structure's subtypes with grep -rln "implements <Name>". A grep cannot see a class that gets the interface through its parent class, so Niro listing more is expected; Niro listing fewer is worth reporting.

5. Report a wrong or missing answer

Send the question, the answer, and what you expected (with the grep that showed it) to help@niro.ai. The known gaps are in FAQ: wrong or missing answers, and the feedback FAQ is FAQ: feedback and receipts.

Tokens, speed and accuracy, Get shorter answers.