Inconsistent coverage results when using additional vitest projects for unit tests #31962

lucasmuna · 2025-07-05T20:57:47Z

lucasmuna
Jul 5, 2025

Summary

Hello,

While adding unit tests as a new vitest project called unit I saw that my coverage actually got worse than just running the "standard" storybook project.

So, if I run npx vitest --project storybook --coverage --run I get the expected coverage report (without unit tests), but, if I run all tests with npx vitest --coverage --run I get worse coverage results. By worse I mean, code that was marked as covered becomes marked as uncovered.

My first guess is that I messed up with the configuration, but I failed to find something. My second guess is that each project report is somewhat different, as storybook uses browser/playwright env and unit uses jsdom, but could not find anything as well.

As additional info, if I add the unit tests file patterns to .storybook/main.ts config stories, then I have the expected coverage result, but, I cannot run storybook anymore because it throws Error: Invariant failed: No matching indexer found for. The error is expected because these unit tests are not stories, but the coverage behavior is interesting.

Any ideas are welcomed.

Additional information

My vitest.config.ts:

export default defineConfig({
  test: {
    coverage: {
      provider: "v8",
      reporter: ["text", "html", "json"],
      exclude: [
        ...coverageConfigDefaults.exclude,
        "./postcss.config.mjs",
        "./next.config.ts",
      ]
    },
    projects: [
      {
        extends: true,
        plugins: [
          storybookTest({ configDir: path.join(dirname, ".storybook") }),
        ],
        test: {
          name: "storybook",
          browser: {
            enabled: true,
            headless: true,
            provider: "playwright",
            instances: [{ browser: "chromium" }],
          },
          setupFiles: [".storybook/vitest.setup.ts"],
        },
      },
      {
        extends: true,
        plugins: [tsconfigPaths(), react()],
        test: {
          name: "unit",
          include: ["**/test/**/*.test.{ts,tsx}"],
          environment: "jsdom",
        },
      },
    ],
  },
})

npx vitest --project storybook --coverage --run

npx vitest --coverage --run (so storybook and unit projects)

npx vitest --project storybook --coverage --run with unit test file patterns under .storybook/main.ts config stories

This is actually the expected behavior when running test/coverage with both projects

Create a reproduction

No response

@lucasmuna · 2025-07-05T20:59:42Z

dosubot[bot]
bot Jul 5, 2025

Hi @lucasmuna! I'm Dosu and I’m helping the storybook team.

This kind of inconsistent coverage is a known challenge when running multiple Vitest projects with different environments (like browser/playwright for Storybook and jsdom for unit tests). The main reasons are:

Coverage tools (like v8) instrument code differently depending on the environment, so coverage data from browser-based and jsdom-based tests may not merge cleanly, leading to files being marked as uncovered when running both projects together see discussion.
Vitest and Storybook may not always merge Vite configs and plugins as expected, especially in complex setups. Manual merging in the viteFinal hook is often needed to ensure all plugins and aliases are respected see issue.
Running both projects together can cause plugin duplication or config conflicts, which may affect coverage instrumentation and reporting see issue.

Best practice is to run coverage for each project independently (e.g., vitest --project=storybook --coverage --run and vitest --project=unit --coverage --run) and, if you need a combined report, merge the resulting coverage outputs using a tool like nyc or v8-to-istanbul see docs. This avoids environment conflicts and ensures accurate results.

Also, make sure your Vite config is merged correctly for both projects, and that global styles, assets, and setup files are loaded identically in both environments to minimize discrepancies see discussion.

Recent Storybook releases have improved config merging for addon-vitest, but some setups still require manual intervention, especially with custom plugins see issue. If you continue to see issues, consider isolating your test configs further (e.g., separate vitest.config.ts files) and double-checking your coverage exclude/include patterns.

To reply, just mention @dosu.

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

1 reply

lucasmuna Jul 6, 2025
Author

@dosu

I followed your suggestion, but I'm still facing the same issue.

What I'm doing now is to run both projects independently and then I merge they coverage results and finally I report it using html reporter. For both post processing merging and reporter steps I'm using nyc.

Here is the script:

npx vitest --project storybook --run --coverage
cp coverage/coverage-final.json coverage-outputs/coverage-storybook.json

npx vitest --project unit --run --coverage
cp coverage/coverage-final.json coverage-outputs/coverage-unit.json

npx nyc merge coverage-outputs/ .nyc_output/out.json

npx nyc report --reporter=html --reporter=text --report-dir=$(pwd)/coverage-merged

The results is the same as running both projects together:

lucasmuna · 2025-07-06T13:29:16Z

lucasmuna
Jul 6, 2025
Author

I tried to avoid nyc and used lcov to merge data and genhtml to generate a html. Interestingly the same behavior happens, but at least I was able to find some more info. Check the coverage for the same code snippet under different conditions:

Coverage from storybook project only:

Coverage from unit project only:

Merged coverage (using lcov and genhtml) from storybook project only:

It is evident that the line (41) that turned into uncovered, was actually not covered but skipped before. Any ideas?

1 reply

dosubot[bot] bot Jul 6, 2025

This happens because coverage tools like Vitest, v8, nyc, lcov, and genhtml handle code instrumentation and merging independently for each environment. When you merge coverage from different environments (like browser/Playwright for Storybook and jsdom for unit tests), the tools may interpret "skipped" lines differently—what's skipped in one environment can be marked as "uncovered" in the merged report if another environment expects that line to be covered but doesn't execute it. This is a known limitation of the coverage toolchain and not something Storybook or its plugins control or interpret—Storybook just passes through the config and test files to the coverage tools, which do all the instrumentation and merging logic themselves (source).

To minimize these inconsistencies, try to keep your test environments and configs as similar as possible, but some differences are inevitable when mixing browser and jsdom. If you need a truly accurate combined report, you may need to merge only compatible coverage outputs or accept this as a current tooling limitation. If this behavior is blocking, it may be worth checking for or filing an issue upstream with Vitest or the coverage provider you use.

To reply, just mention @dosu.

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

ethriel3695 · 2025-07-08T23:45:18Z

ethriel3695
Jul 8, 2025
Collaborator

Hi @lucasmuna! My name is Reuben, and I'm one of the Solutions Engineers at Chromatic and Storybook.
Let's figure this out for you. What version of Storybook are you using? It would also be helpful to know what Framework you are using. If the Storybook is public, a link would be even better.

2 replies

lucasmuna Jul 9, 2025
Author

Hello @ethriel3695 , thanks for replying.

I'm using latest, so 9.0.16, with nextjs.

I was able to create a minimal reproducible example, see https://github.com/lucasmuna/vitest-storybook-coverage-demo

ethriel3695 Jul 10, 2025
Collaborator

This is perfect @lucasmuna, thank you! I'll take a look and get back to you.

valentinpalkovic · 2025-07-11T06:29:12Z

valentinpalkovic
Jul 11, 2025
Maintainer

Hi @lucasmuna

Do you encounter the same inconsistencies if you just run vitest for all projects at once and uses its coverage reporting?

3 replies

valentinpalkovic Jul 11, 2025
Maintainer

Tested, and indeed, it has something to do with @storybook/nextjs-vite. @storybook/react-vite or @storybook/nextjs don't show this behaviour

lucasmuna Jul 11, 2025
Author

Hello @valentinpalkovic , yes, running npx vitest --run --coverage has the same outcome as running both projects individually and then merging them with, e.g., nyc.

lucasmuna Jul 11, 2025
Author

I also tried to change @storybook/nextjs-vite to @storybook/react-vite. Hopefully I did it correctly as I had to improve vitest.config.ts to be able to resolve some paths with that config.

Anyway, as you mentioned, measuring coverage with @storybook/react-vite ended up having almost the same output as adding the unit tests path pattern to .storybook/main.ts (but this crashes storybook itself, as expected cause the tests are not stories). The only difference was that some imports got "branch uncovered" (somehow).

Uh oh!

Inconsistent coverage results when using additional vitest projects for unit tests #31962

Uh oh!

Uh oh!

lucasmuna Jul 5, 2025

Summary

Additional information

Create a reproduction

Replies: 4 comments · 7 replies

Uh oh!

dosubot[bot] bot Jul 5, 2025

Uh oh!

lucasmuna Jul 6, 2025 Author

Uh oh!

Uh oh!

lucasmuna Jul 6, 2025 Author

Uh oh!

dosubot[bot] bot Jul 6, 2025

Uh oh!

ethriel3695 Jul 8, 2025 Collaborator

Uh oh!

lucasmuna Jul 9, 2025 Author

Uh oh!

ethriel3695 Jul 10, 2025 Collaborator

Uh oh!

valentinpalkovic Jul 11, 2025 Maintainer

Uh oh!

valentinpalkovic Jul 11, 2025 Maintainer

Uh oh!

lucasmuna Jul 11, 2025 Author

Uh oh!

lucasmuna Jul 11, 2025 Author

lucasmuna
Jul 5, 2025

Replies: 4 comments 7 replies

dosubot[bot]
bot Jul 5, 2025

lucasmuna Jul 6, 2025
Author

lucasmuna
Jul 6, 2025
Author

ethriel3695
Jul 8, 2025
Collaborator

lucasmuna Jul 9, 2025
Author

ethriel3695 Jul 10, 2025
Collaborator

valentinpalkovic
Jul 11, 2025
Maintainer

valentinpalkovic Jul 11, 2025
Maintainer

lucasmuna Jul 11, 2025
Author

lucasmuna Jul 11, 2025
Author