Skip to content

add test case for fread N=cols rnorm - #7907

Draft
tdhock wants to merge 8 commits into
masterfrom
atime-fread-cols
Draft

tdhock wants to merge 8 commits into
masterfrom
atime-fread-cols

Conversation

@tdhock

@tdhock tdhock commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Hi all!
Bad news: there seems to be a performance regression in fread, some time in the last three years. (I will soon run git bisect to find the PR responsible!)

I regularly present these slides https://docs.google.com/presentation/d/1mHTFR6Eg7OdKi6yJcAvMk5_B8hjtMmsczs8Ewxt2xT8/
which contain these benchmarks https://tdhock.github.io/blog/2023/dt-atime-figures/

So to verify if those conclusions are still true, 3 years later, I ran the benchmarks again, with upgraded software versions, https://tdhock.github.io/blog/2026/dt-atime-update/

The first fwrite benchmark looks ok, but the second one, about fread on a CSV with real numbers and a varying number of columns, looks like there is a significant difference (on two of my laptops) so here I submit a PR with a new performance test for this case.

Locally I run the new test via

tinfo <- atime::atime_pkg_test_info("~/R/data.table")
tcall <- tinfo$test.call[["fread N=cols regression"]]
tres <- eval(tcall)
tref <- atime::references_best(tres)
tpred <- predict(tref)
plot(tpred)

and I get this result: (on a 2025 ubuntu laptop)
image
we see that HEAD is about the same as Slow, both much slower (almost 2x) than Fast.
(please ignore the CRAN curve, I’m not sure why it is so slow, but that only happens on one of my two laptops, so I suspect that is not a real issue)

@tdhock tdhock self-assigned this Oct 8, 2026
@tdhock tdhock added atime Requests related to adding/improving/monitoring performance regression tests via atime. fread labels Oct 8, 2026
@tdhock tdhock added this to the 1.19.0 milestone Oct 8, 2026
@tdhock

tdhock commented Oct 9, 2026

Copy link
Copy Markdown
Member Author

Using the same test code in the OP, I get this consistent result on an old 2010 Ubuntu laptop
image
we see that HEAD and master are much closer to Slow than Fast.

@tdhock

tdhock commented Oct 9, 2026 •

Copy link
Copy Markdown
Member Author

git bisect on Fast and Slow gave a commit with intermediate performance (about halfway between them).
image
This suggests there is more than one commit responsible.

Bisecting 1685a3b and 1.18.6 yielded #7370 involving fread by @ben-schwen , can you please take a look and see if the performance can be improved?

df7fa8071c6b818d24e47cc1a84a6dc0491550d6 is the first bad commit
commit df7fa8071c6b818d24e47cc1a84a6dc0491550d6
Author: Benjamin Schwendinger <52290390+ben-schwen@users.noreply.github.com>
Date:   Mon Nov 3 22:03:18 2025 +0100
…
 NEWS.md               |  2 ++
 inst/tests/tests.Rraw |  3 +++
 src/fread.c           | 27 +++++++++++++++++++++------
 3 files changed, 26 insertions(+), 6 deletions(-)
bisect found first bad commit

bisecting 1.14.8 and 67db7f7 yielded #7375 by @ben-schwen and @MichaelChirico — can you please check and see if performance can be improved there too?

59f966cf6839a0494e42ed1ab98616eea03dcb36 is the first bad commit
commit 59f966cf6839a0494e42ed1ab98616eea03dcb36
Author: Benjamin Schwendinger <52290390+ben-schwen@users.noreply.github.com>
Date:   Mon Oct 20 19:20:52 2025 +0200
…
    Co-authored-by: Michael Chirico <chiricom@google.com>
 NEWS.md               |   1 +
 R/fread.R             |   7 ++-
 inst/tests/tests.Rraw | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++
 man/fread.Rd          |   3 +-
 src/data.table.h      |   2 +-
 src/fread.c           | 125 ++++++++++++++++++++++++++++++++++++++++++++-----
 src/fread.h           |   4 ++
 src/freadR.c          |   3 ++
 8 files changed, 257 insertions(+), 15 deletions(-)
bisect found first bad commit

To replicate my git bisect analysis, the source code I used for computing and visualizing bisect is in .ci/atime/bisect-*R and the details can be seen on this interactive data viz, https://tdhock.github.io/2026-10-09-fread-git-bisect/
image

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

atime Requests related to adding/improving/monitoring performance regression tests via atime. fread

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant