32Flakiness: Local vs CI — diagnostic playbook

32.1 The 10 root causes (memorise the list)

1. Timing / race conditions — Faster local machine; slow CI exposes races. Fix: web-first assertions, no waitForTimeout.
2. OS/font/rendering differences — Pixel-perfect snapshot fails on Linux CI because baseline was generated on macOS.
3. Browser/OS resource limits — CI worker has less RAM/CPU; Chromium tab crashes mid-test.
4. Network differences — CI may not reach localhost-only services; DNS resolution differs; corporate proxies.
5. Env vars missing/different — Local has .env, CI uses secrets; one wasn't migrated.
6. Data state — Local DB has data your test assumes; CI's fresh DB doesn't. (Module 31)
7. Auth tokens expired or wrong — Local cached token still works; CI's fresh fetch hits a different IDP.
8. Parallelism — Local single-worker; CI runs 4× in parallel, exposing data collisions.
9. Time zone / locale — Local in IST; CI in UTC; date assertions drift by 5.5 hours.
10. Animations / fonts / lazy resources — Local browser cached them; CI's fresh browser hasn't.

32.2 Triage playbook (the interview answer)

1. Pull the trace from the failing CI run. npx playwright show-trace ci-trace.zip.
2. Read the failure step — exact action, locator, timing, error text.
3. Compare network / console with a green local run — what's different?
4. Check env var diff: echo $BASE_URL locally vs CI; verify all required vars are set.
5. Reproduce CI conditions locally:
   • Use the Playwright Docker image: docker run -it --rm -v $PWD:/work mcr.microsoft.com/playwright:v1.50.0-jammy
   • Set CI=true environment variable
   • Run with the CI's worker count: --workers=4
   • Throttle CPU/network if the CI worker is slower
6. Run repeated — --repeat-each=50. If 1/50 fails locally, you've reproduced the race.
7. Bisect changes if it's a recent regression: git bisect between last known green and current red.
8. Fix the root cause — not the symptom. Add a wait? No — find the missing assertion or fix the locator.
9. Lift quarantine once --repeat-each=100 passes on CI for two consecutive runs.

32.3 Quick deterministic-by-design patterns

32.4 Specific recipes for the top causes

(a) Timing race

// BAD
await page.click('#load-orders');
await page.waitForTimeout(2000);
expect(await page.textContent('#count')).toBe('3');

// GOOD
await page.click('#load-orders');
await expect(page.locator('#count')).toHaveText('3');   // auto-retries

(b) OS rendering on visual tests

# Generate baselines on the same OS image CI uses
docker run --rm -it -v $PWD:/work -w /work mcr.microsoft.com/playwright:v1.50.0-jammy \
  npx playwright test --update-snapshots

(c) Resource constraint

// playwright.config.ts — limit workers on CI if RAM is tight
export default defineConfig({
  workers: process.env.CI ? 2 : 4,
  /* or scale to CPU count: workers: '50%' */
});

(d) Timezone

use: {
  timezoneId: 'UTC',     // every project uses UTC
  locale: 'en-US',
}

(e) Parallel data collision

// Use a unique email per test
const email = `qa+${Date.now()}-${faker.string.alphanumeric(6)}@test.com`;

32.5 The classic interview trap question