Shadow DOM and iframes

Two structures hide content from an XPath evaluated on the main document:

1. Shadow DOM: a separate tree attached to a host element, used by web components (Lit, Stencil, Salesforce Lightning, Shoelace, many design systems, and browser-native controls).
2. iframes: a separate document embedded in the page (payment widgets, chat, ads, embedded editors, legacy apps).

document.evaluate walks one document tree. It stops at both boundaries. No XPath syntax changes that, so the fix is always in your test framework, not in the expression.

Recognising a shadow root

In DevTools Elements, look for #shadow-root (open) under an element. That element is the host; everything under the shadow root is invisible to XPath from outside.

<my-dropdown>
  #shadow-root (open)
    <button class="trigger">Choose</button>
    <ul>...</ul>
</my-dropdown>

//button[@class='trigger'] returns nothing. //my-dropdown returns the host.

Closed shadow roots (#shadow-root (closed)) are not reachable by any locator from any framework. If you meet one, ask the developers for a test hook.

Getting inside: Selenium 4

Selenium 4 exposes the shadow root as a search context. XPath does not work inside it (Chrome’s shadow root context supports CSS only), so switch to CSS for the final step:

WebElement host = driver.findElement(By.xpath("//my-dropdown[@data-testid='country']"));
SearchContext shadow = host.getShadowRoot();
WebElement trigger = shadow.findElement(By.cssSelector("button.trigger"));

host = driver.find_element(By.XPATH, "//my-dropdown[@data-testid='country']")
shadow = host.shadow_root
trigger = shadow.find_element(By.CSS_SELECTOR, "button.trigger")

Use XPath to reach the host (text matching, axes, all the strengths) and CSS for the inside.

Getting inside: Playwright

Playwright’s CSS engine and text engine pierce open shadow roots automatically. Its XPath engine does not. So:

// works: css pierces shadow DOM
await page.locator('my-dropdown button.trigger').click();

// works: role locators pierce too
await page.getByRole('button', { name: 'Choose' }).click();

// does NOT find the button: xpath stops at the host
await page.locator('xpath=//my-dropdown//button').click();

// chain: xpath to the host, css inside
await page.locator('xpath=//my-dropdown[@data-testid="country"]').locator('button.trigger').click();

Getting inside: Cypress and WebdriverIO

Cypress: cy.get('my-dropdown').shadow().find('button.trigger'), or set includeShadowDom: true in config so cy.get searches inside shadow roots.

WebdriverIO: $('my-dropdown').shadow$('button.trigger'). WebdriverIO also offers deep selectors (>>>) that pierce shadow roots.

iframes

An iframe is a whole separate document. //iframe selects the frame element in the parent; nothing inside it is visible from there.

Selenium switches context:

driver.switchTo().frame(driver.findElement(By.xpath("//iframe[@title='Secure card frame']")));
driver.findElement(By.xpath("//input[@name='cardnumber']")).sendKeys("4242...");
driver.switchTo().defaultContent();

Playwright uses frame locators, no switching:

const frame = page.frameLocator('iframe[title="Secure card frame"]');
await frame.locator('xpath=//input[@name="cardnumber"]').fill('4242...');

Cypress needs a plugin such as cypress-iframe, or you reach in through its('0.contentDocument.body').

Inside the frame, XPath works exactly as usual. Nested iframes need one switch (or one frameLocator) per level.

Locating the frame itself

Frames rarely have good attributes. Useful hooks:

//iframe[@title='Secure card frame']
//iframe[contains(@src, 'stripe.com')]
//iframe[@name='editor']
//div[@data-testid='payment']//iframe
(//iframe)[1]                                only if there is exactly one

title is the accessibility name and the most stable option.

Decision guide

Symptom	Likely cause	What to do
Locator works in DevTools search but not in test	Test is in the wrong frame	Switch to the iframe
Locator returns nothing, element shows under #shadow-root in DevTools	Shadow DOM	Reach the host, then CSS inside
$x() in the console finds nothing but the element is visible	Either of the above	Check for #shadow-root and <iframe> in the ancestor chain
Works in Playwright CSS but not XPath	Shadow DOM	Use CSS or role locator for the inner part

Try It Yourself

The playground renders samples in a plain document, so it cannot demonstrate the boundary. Instead open any site that uses web components, run $x("//button") in the console, and compare it with document.querySelectorAll("button") after enabling “Show user agent shadow DOM” in DevTools settings.

Next Steps

1. Testing XPath in DevTools - $x() and the Elements search box
2. XPath in Playwright - Mixing engines in one locator chain
3. XPath in Selenium - Frames, shadow roots and search contexts