The Kaitchup – AI on a Budget

The Kaitchup – AI on a Budget

Share this post

The Kaitchup – AI on a Budget
The Kaitchup – AI on a Budget
Yes, We Need Statistical Significance Testing
Copy link
Facebook
Email
Notes
More

Yes, We Need Statistical Significance Testing

A rule of thumb may yield correct results but can’t be scientifically credible

Benjamin Marie's avatar
Benjamin Marie
Oct 27, 2022
∙ Paid
2

Share this post

The Kaitchup – AI on a Budget
The Kaitchup – AI on a Budget
Yes, We Need Statistical Significance Testing
Copy link
Facebook
Email
Notes
More
Share

A rule of thumb may yield correct results but can’t be scientifically credible.

Illustration by the author.

Take any research paper or blog post presenting a new method for AI, you’ll very probably find a statement similar to this:

[…] a significant improvement over previous work.

If this is a method applied to language generation tasks (automatic summariza…

Keep reading with a 7-day free trial

Subscribe to The Kaitchup – AI on a Budget to keep reading this post and get 7 days of free access to the full post archives.

Already a paid subscriber? Sign in
© 2025 The Kaitchup
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More