CI/CD Test Optimization: Sharding and Parallel Testing

A suite that took four minutes a year ago and takes eighteen minutes now usually didn't get eighteen minutes' worth slower evenly. A handful of tests account for most of the growth, and adding more parallel runners without finding them just means more runners sitting idle waiting for the same three slow tests to finish.

Find the bottleneck before you scale runners

Pull the per-test duration data your CI already logs and look at the distribution, not the average. It's common to find that 10-15 tests account for a large share of total runtime, usually because of unnecessary setup and teardown, unmocked network calls, or a wait condition that's really a fixed sleep in disguise (the same root cause behind a lot of flaky tests). Fixing those tests directly often does more than doubling the runner count.

Shard by duration, not by file count or test count

Splitting a suite evenly across shards by number of tests looks balanced and usually isn't, because it assumes every test takes the same amount of time. One shard ends up with the five slowest tests in the suite and finishes last every single run, which means your total pipeline time is bounded by that one shard no matter how many others you add.

Balance shards by historical duration instead: feed the last few runs' per-test timings into the shard assignment so each shard's total expected runtime is roughly equal. Most CI providers and test runners support this directly; it's a configuration change, not a rewrite.

How to run tests in parallel in CI/CD

Most CI providers run parallel jobs from a matrix, and most test runners can run one slice of the suite per job. With GitHub Actions and Playwright:

strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npx playwright test --shard=${{ matrix.shard }}/4

Each of the four jobs runs a quarter of the tests at the same time. Jest has the same --shard flag. Cypress parallelizes with cypress run --record --parallel through Cypress Cloud or a compatible service. pytest spreads tests across the CPU cores of one machine with pytest -n auto from pytest-xdist.

Built-in sharding splits by number of tests or files, not by duration. If one shard keeps finishing last, switch to duration-based splitting: CircleCI does it with circleci tests split --split-by=timings, and tools like Knapsack Pro add it to other CI providers.

Parallelization has a shared-state tax

Tests that pass reliably in sequence and fail intermittently in parallel are almost always fighting over shared state: the same seeded test account, the same database rows, the same external sandbox rate limit. The fix isn't slower parallelization, it's fixture isolation, a fresh account or dataset per worker instead of a shared one everyone mutates.

Skipping this step is how teams end up with a suite that's fast on paper and untrustworthy in practice, because a chunk of its failures are actually contention, not real bugs, and the team starts re-running red pipelines out of habit.

Quarantine lanes, with an owner and a deadline

A flaky test shouldn't block every merge, and it shouldn't be silently skipped forever either. A separate CI lane for quarantined tests keeps them running and visible without gating deploys on them, but only if someone owns clearing that lane. Quarantine with no deadline is just a slower way of deleting the test.

What not to parallelize blindly

Some flows are sequential by nature: a payment that triggers a webhook that triggers a confirmation email. Running those in true parallel with shared infrastructure invites exactly the race conditions the flow exists to prevent in production. For flows like that, isolate them into their own lane rather than forcing them into the same sharding strategy as your stateless UI tests.

If your pipeline is the thing slowing down releases, that's usually a sharding and fixture problem before it's an infrastructure budget problem, and it's the first thing we look at in a test automation audit. Running less on each PR is the other half of the fix, covered in regression testing strategy.