Forty Files, Two Answers: Your Benchmark Measured the Shape of the Input

I ran the same command on the same number of files, in the same repository, on the same machine. The first run finished in 861 milliseconds. The second took 6.4 seconds — and lost to the tool it was built to replace.

Nothing about the command changed between those two runs. Only the shape of what I asked it for.

The command is git last-modified. It landed in Git 2.52.0 and is still marked EXPERIMENTAL in its own manual page, with real bug fixes arriving as late as 2.54. It does one walk through a repository's history and answers, for every path you hand it, which commit touched that path most recently. The thing it replaces is the loop every repository browser has always written: run git log -1 -- path once per file, forty times for a forty-row listing. That loop re-reads history from the newest commit on every call, which is exactly the waste the new command was designed to remove.

Spreading one history walk across many paths is a good idea. Whether it is a good idea for your paths is a separate question, and it is the one nobody benchmarks.

The setup

I tested against git.git — Git's own repository — cloned with --filter=blob:none. That flag fetches full commit and tree history but skips file contents, which is fine because last-modified never needs the bytes. The clone has 82,542 commits and 4,857 tracked files, on a 4-core virtual machine. I built Git 2.56.0 (released 27 September 2026) from source, because no package repository I could reach ships it yet. Three runs per method, fastest reported.

One term to fix before we go further: a pathspec is just the list of paths you give the command — Git's own documentation uses the word the same way. Here are the forty files I care about. Everything below is about how those forty files are distributed.

Round one: the case the command was designed for

The realistic use is a repository browser listing one directory. Here are these forty files, show me who touched each one last.

The result was clean: 861 milliseconds for git last-modified against 2,981 milliseconds for the forty-call loop. About 3.5x faster. I spot-checked the commits for five of the files against git log -1 and every hash matched, so it was not fast because it was wrong.

The reason it wins is history overlap. Forty files sitting in one directory tend to be touched by the same events: the same release-prep commits, the same sweeping refactor, the same rename. One walk over history answers all forty questions at once because the answers live in the same stretch of the log. The fixed cost of the walk gets spread across the whole list.

Round two: same count, different shape

So I built a second list. Same size, forty files, but picked at random from anywhere in those 4,857 tracked files — different directories, mostly unrelated histories. Then I ran the identical comparison.

6,402 milliseconds for git last-modified. 5,793 milliseconds for the loop. The new command was now the slower option, on the same number of files, in the same repository, on the same machine.

I pushed further to check it was not noise. At 50 scattered files: 7.48 seconds against 6.59. At 100: 25.7 seconds against 12.2. The loop's cost roughly doubles across that step, which is what you would expect — twice the files, twice the walks. last-modified more than tripled. I killed a 500-file run of the same shape after it had burned over four CPU-minutes without finishing. That is not a timing fluke; that is a cost curve bending the wrong way.

The bookkeeping the command does per path appears to compound badly once the paths stop sharing history. It is still doing the right thing conceptually — one walk, many answers — but when the answers are scattered across 82,000 commits, the efficient path through history stops being efficient.

The single-file case, which is not a bug

Hand it exactly one path and there is nothing to spread the cost across. You are just measuring setup. On a single file, last-modified ran 684 milliseconds at best (1,582 on a slow run) against 431 milliseconds (744 worst) for git log -1. Five runs each; the new command never once won. The wider spread is its own small warning: the setup cost is not just slower, it is less predictable.

None of this contradicts the command's purpose. It exists to amortize one walk across many paths. Give it one path and you have removed the amortization and kept the overhead.

The unstated assumption in every benchmark

Here is the part worth carrying out of this article. 'Forty files' is not a benchmark description — it is half of one. Forty files in one directory and forty files scattered across the tree are different benchmarks that happen to share a number.

You see the same blind spot everywhere. A database index benchmark that silently uses uniformly distributed rows, then falls apart on the skew your production data actually has. A cache comparison run warm that reports numbers you will never see cold. A vector search evaluation on randomly generated embeddings, which cluster nothing like the embeddings your model actually produces. A test harness where the reference documents were never committed, so every question was asked against a blank context and every answer still scored fine. In each case the harness returned a number, the number was internally consistent, and the number answered a narrower question than its name.

Whoever reported roughly 6x faster for this command was not lying. They measured a directory listing — the case their users actually have. The claim was true for that shape and got repeated as if shape did not matter. That is how a good measurement becomes bad advice.

A decision rule you can apply

• You are listing a directory. Use git last-modified. This is the case it was built for, and the 3.5x I measured is real.
• You are pulling files together from unrelated parts of the repo — a dashboard, a report, every file this team owns. Use the per-file loop. It beat the new command in every scattered test I ran.
• You need one file. Use git log -1. There is nothing to amortize.

The underlying question is: do these paths share history? If you selected them by directory, by release cadence, or by a refactor's blast radius, they probably do and the single walk pays off. If you selected them by an attribute — every file over 2,000 lines, every file matching a pattern, every file a team owns — they probably do not, and the per-file loop will beat it.

The bug that tells you the interface is young

While testing, I passed a directory and one of its own children as two separate pathspecs. In one order, the parent came back as a single aggregate row. In the other order, the parent silently expanded into all 18 of its immediate children and its own row vanished. The manual says the default behaviour (--max-depth=0) shows matching paths without descending into subtrees, and I had passed that flag explicitly. It still descended.

That matters for two reasons. First, if you are feeding this into a repository browser, an argument order that quietly changes the number of rows is the kind of thing that breaks a table without an error message. Second, it is a fair reminder of what EXPERIMENTAL means on a command that shipped in 2.52 and took fixes through 2.54. Pin the version. Assert on row count in your tests. Compare a sample against git log -1 in CI, not just once by hand.

How to benchmark something other people will re-run

1. Publish the shape, not just the size. Forty files from one directory and forty files sampled across the tree are different claims. Say which one you measured.
2. Run a second distribution that should agree. If your claim is about the tool, changing which forty files you picked should not flip the winner. When it does, your claim was about the shape.
3. Report the minimum and the spread. My single-file runs ranged from 684ms to 1.58s. A single number would have hidden that the setup cost itself is unstable.
4. Verify correctness on a sample. A fast wrong answer is the easiest benchmark in the world to win.
5. Say how you built the thing. Version 2.56.0, compiled from source, no distribution package, tells a reader how much to trust the comparison.
6. Report the runs you killed. I stopped a 500-file scattered run after four CPU-minutes. That is data, not a gap in the results.

The honest limits

This is one repository, one machine, one version. git.git is enormous and unusually lopsided — hand-written C, documentation, tests, contrib, all with very different commit rhythms. A small application repo may never produce a scattered set large enough for the crossover to matter, and the per-file loop there is cheap enough that nobody cares. And the command is explicitly experimental: 2.57 may well fix the compounding cost and the pathspec ordering, at which point the rule above needs re-measuring rather than re-quoting.

That is the point, though. The tool is not the problem. The habit of adopting a speed number without checking which shape it was measured on is the problem — and it costs you more than 5.5 seconds on a file listing.

Key Takeaways

• Same command, same 40 files, same machine: 861ms in one directory, 6.4s scattered. The input shape decided the winner, not the tool.
• git last-modified is the right tool for directory listings and the wrong tool for cross-cutting file sets. Use git log -1 for a single file.
• 'Forty files' is half a benchmark. State the distribution or your numbers do not transfer.
• Run a second distribution that should agree. If the winner flips, you measured your fixture, not the software.
• Experimental means argument order can change your output shape. Pin versions and assert on row counts in tests.
• Report spread and killed runs; a range of 684ms to 1.58s says something a single number cannot.