Vitest is not ready to replace Jest (and may never be)
I recently had an opportunity to try to improve the build-times for some client software. After reviewing the claims for why Vitest was superior to Jest (the client software’s current test-runner), I was convinced that switching to Vitest would be the right move. So here’s what happened…
Software size
The software consists of several ESM packages in a RushJS monorepo. The code is written in TypeScript. The most important package contains a React/NextJS website, which depends on a web-components (React) package and some smaller common libraries.
It took approximately 10 hours to convert enough of the code to fully-convert the web-package from Jest to Vitest.
Results
Test time with coverage in Jest: ~58sec
Test time with coverage in Vitest: ~92sec
The Vitest figure was the fastest/best number I could come up with after trying every combination of Vitest config that could possibly affect the results. That’s when I started to research as to whether I was the only person in the world seeing these kinds of numbers. Turns out, I’m not.
A known problem
This GitHub issue gives a pretty good summary of the exact issue I’m seeing. To summarise, Vitest is faster only if you turn off test isolation. Yep, that’s the current recommended solution from one of the developers.
It’s like an aircraft manufacturer saying that their planes can go faster if they don’t have to share airports with other planes. Obviously, this is of no use in the real world.
Thankfully, I’m not the only person to discover that Vitest is not fast.
Look, I understand that writing software for free and then receiving complaints about performance of said-software is annoying. And some issues are hard to fix. Users should not expect the software to perform perfectly.
However, the producers-of-the-software should NOT be making false claims (ie “It’s fast”) without real-world benchmarks to support those claims.
Summary
Use Vitest at your own peril. It cost me 10 hours to find this out. Hopefully this 3-minute article will save you 10 hours to discover that Vitest is slow for real-world use cases.
I would have loved for this article to be a Vitest success-story, obviously. But unfortunately for me and the Vitest team, it is a product-warning article. If the situation were to change, my git-branch is still available to re-test any new version of Vitest that solves these performance issues.
