Diagnosing Flaky Locators in CI
A locator that passes locally and fails in the pipeline is almost never random. The six causes, how to tell them apart from the failure message alone, and the XPath changes that fix each.
Start from the message
| Message | Most likely cause |
|---|---|
NoSuchElementException / Timeout waiting for locator on the first action of a test | timing, viewport or environment difference |
| Same, but mid-test after an action | re-render race or navigation not awaited |
StaleElementReferenceException | element re-rendered between find and act |
ElementNotInteractableException / element is not visible | hidden duplicate matched first, or animation |
ElementClickInterceptedException | overlay, toast or cookie banner covering the target |
Playwright strict mode violation | duplicate that only exists in CI’s data or viewport |
| Test passes on retry | timing or data, not the locator |
Cause 1: viewport differences
Locally you run a 1920-pixel window; CI runs headless at 800x600. Responsive layouts render a different DOM: a hamburger menu instead of a nav bar, a card list instead of a table, a hidden desktop nav plus a visible mobile nav.
Tell-tale: the locator matches two elements in CI (strict mode violation) or matches a hidden one (not interactable).
Fix: set the viewport explicitly in CI to match local, and anchor locators to the variant you mean:
//nav[@aria-label='Main']//a[normalize-space()='Pricing']
//header//a[normalize-space()='Sign in'][not(ancestor::*[@data-testid='mobile-menu'])]Cause 2: timing and re-renders
Frameworks render a skeleton, then data, then re-render on state change. A find that runs between two renders gets a node that is about to be replaced.
Tell-tale: StaleElementReference, or an action that “did nothing” because it hit the old node.
Fix: find immediately before acting; in Selenium re-find inside a retry, in Playwright use locators (which re-resolve) and avoid holding ElementHandles. Also anchor to something that renders once. Rows re-render; the table does not.
Cause 3: data differences
CI seeds a database with different data than your local environment: more rows, different names, another user with the same first name.
Tell-tale: duplicate matches, or text-based locators that fail because the text is different.
Fix: make the test own its data (create the entity it will click), and match on the created entity’s unique attributes rather than shared text:
//tr[@data-testid='user-row'][.//td[normalize-space()='qa-user-8f2a@example.test']]//button[@aria-label='Edit']Cause 4: overlays and animations
Cookie banners, onboarding tours, toasts and loading spinners appear in fresh CI browsers and not in your logged-in local session.
Tell-tale: ElementClickIntercepted, or a click that lands on the overlay.
Fix: dismiss known overlays at the start, wait for spinners to disappear, and in Playwright let auto-wait handle animations. XPath for the wait:
//*[@role='progressbar'] wait until count is 0
//*[@data-testid='cookie-banner']//button[normalize-space()='Accept']Cause 5: locale and copy
CI runs with a different Accept-Language, or the app’s copy changed and your local build is stale.
Tell-tale: text locators fail; attribute locators pass.
Fix: pin the locale in the browser context, prefer attribute and role locators for controls, and keep text for assertions rather than navigation where it varies.
Cause 6: the locator was always fragile
An index or absolute path that happened to work locally. CI renders one extra banner, and //div[3]/button now points at something else.
Tell-tale: consistent failure after a deploy, not intermittent.
Fix: rewrite with an anchor. The robustness improver and the rules on the cheat sheet cover this.
A CI-specific debugging routine
- Capture a screenshot and the DOM (
page.content()/driver.getPageSource()) on failure. Attach both to the report. - Load the saved DOM into the browser locally and test the locator with Ctrl+F in DevTools. If it matches there, the cause is timing or interaction, not the expression.
- If it does not match, diff the saved DOM against a local one around the target. The difference is the cause.
- Run the test alone, then in the suite. Suite-only failures point to shared state or leftover overlays from a previous test.
- Fix the root cause, not the retry count.
What retries hide
Retrying a test that fails one time in ten makes the suite green and leaves the flake in place. Use retries to keep pipelines moving, but track which tests needed them and treat a retry as a bug report. A flaky locator is usually a fragile locator that has not failed consistently yet.