Your scraper reads a product card and stores ₹1,249₹1,39910% off. Three values went in and one string came out, and no code can reliably split it back.
innerText returns an element’s text as the page renders it, and elements that sit side by side in a line (inline or inline-block) get no separator when the markup has no whitespace between them, however far apart CSS draws them. So a price, a list price and a discount (₹1,249, ₹1,399, 10% off) come out as ₹1,249₹1,39910% off, and an average score with its count (4.5, 720 Ratings) comes out as 4.5720 Ratings. Keep the original string if your code already reads it, and collect the card’s text node by node with a TreeWalker beside it, plus the card’s images. Each text node is one value, so nothing has to be split.Table of Contents
The symptom: numbers that run into each other
A crawler drives a browser through a marketplace’s search results and keeps each product card’s text with Playwright’s inner_text(), which reads the element’s innerText. Titles look fine. The price line does not:
₹1,249₹1,39910% off
4.5720 RatingsCode language: CSS (css)
Try to read it back. ₹1,39910% off could be ₹1,399 and 10% off, or ₹13,991 and 0% off. 4.5720 could be an average of 4.5 over 720 ratings, or 4.57 over 20. A regular expression can guess from typical ranges, and every guess is a value you did not observe. In the affected crawl, price, list price and discount ran together on about 7,850 of 7,932 cards, and rating and rating count on 325. Only the first price could be read back with confidence.
A glued string has already thrown away the boundary; no parser downstream can put back what the browser removed.
Step 1: Reproduce it on a blank page
Before changing the crawler, pin down which layouts glue. The test below builds the same card in six ways and reads it three ways in headless Chromium:
from playwright.sync_api import sync_playwright
CARD = ('<div class="price"><X>₹1,249</X><X>₹1,399</X><X>10% off</X></div>'
'<div class="rate"><X>4.5</X><X>720 Ratings</X></div>')
WALK = """(root) => {
const out = []; const w = document.createTreeWalker(root, NodeFilter.SHOW_TEXT); let n;
while ((n = w.nextNode())) { const t = n.nodeValue.replace(/\\s+/g, ' ').trim(); if (t) out.push(t); }
return out; }"""
with sync_playwright() as p:
page = p.chromium.launch().new_page()
html = CARD.replace("<X>", "<span>").replace("</X>", "</span>")
page.set_content(f"<div id=card>{html}</div>")
card = page.locator("#card")
print(repr(card.inner_text()), repr(card.text_content()), card.evaluate(WALK))
Code language: Python (python)
Swapping the <span> for other elements and layouts gave this (Chromium 145):
| How the values are laid out | innerText | textContent | Text nodes |
|---|---|---|---|
<span>s, markup with no spaces | ₹1,249₹1,39910% off | glued | separate |
<div>s set to inline-block | ₹1,249₹1,39910% off | glued | separate |
<span>s with a CSS margin between them | ₹1,249₹1,39910% off | glued | separate |
List price in an <s> strike-through | ₹1,249₹1,39910% off | glued | separate |
<span>s with a newline between them in the markup | ₹1,249 ₹1,399 10% off | keeps the markup’s newline | separate |
<div>s as items of a flex or grid row | one value per line | glued | separate |
Plain block <div>s | one value per line | glued | separate |
Only the text-node list kept every value apart in every layout.
Why the page looks spaced but the text is not
innerText follows the rendering rules in the HTML standard: a block-level box adds a line break before and after itself, a table cell adds a tab, and a <p> adds a blank line. Inline content is joined the way it would be read along a line of text, using only the whitespace that is actually in the markup. In these tests, flex and grid items counted as blocks and got their own lines.
A gap you see between two prices is usually a CSS margin or padding. That is layout, not a character, so innerText has nothing to emit. Sites that ship minified markup, with no whitespace between tags, therefore glue every inline neighbor. Copy the line from the page with a mouse and you get the same run-together string.
textContent is no escape. It ignores CSS entirely and returns the text nodes joined with nothing between them, so it glued even the block layouts. MDN’s page on innerText notes one more case: an element that is not rendered (for example display: none) returns the same as textContent. In the test, two block <div>s inside a hidden parent came back as Price 1249MRP 1399, with no line break at all.
Margins draw a gap on screen, but they put no character between the values for innerText to keep.
Step 2: Read the card text node by node
The DOM already holds the values apart: each price is its own text node. A TreeWalker visits text nodes in document order, so collect them as a list instead of asking for one rendered string. Collect the card’s images while you are there; a listing card’s first image is usually the product photo and the rest are badges.
(el) => {
const root = el.closest('[data-card]') || el; // the whole card, not just the link
const nodes = [];
const walker = document.createTreeWalker(root, NodeFilter.SHOW_TEXT);
let n;
while ((n = walker.nextNode())) {
if (n.parentElement && n.parentElement.closest('script,style,noscript')) continue;
const t = (n.nodeValue || '').replace(/\s+/g, ' ').trim();
if (t) nodes.push(t);
}
const imgs = [];
for (const img of root.querySelectorAll('img')) {
const src = img.getAttribute('src') || '';
if (src && !src.startsWith('data:') && !imgs.includes(src)) imgs.push(src); // skip inline placeholders
}
return { nodes, imgs };
}
Code language: JavaScript (javascript)
Pass it to locator.evaluate() in Playwright, or run it in any page context. The price line now arrives as ['₹1,249', '₹1,399', '10% off'] and the rating as ['4.5', '720 Ratings'].
A value the site splits across elements, such as <span>₹</span><span>1,249</span>, comes back as two nodes. That is fine: the list is lossless and in order, so joining a currency sign to the number after it is a small, safe step downstream. Joining is always possible later; splitting a glued string never is.
Step 3: Keep the original string beside the new list
If extraction code already reads the innerText string (pulling a title out of its first line, say), do not replace it. Store text_nodes and card_images as new fields next to it. Adding a field changes nothing for existing readers; changing one they parse is a migration. Move each reader to the node list when you next touch it, and drop the old string only when nothing reads it.
The same trap in static HTML parsers
Parsing saved HTML without a browser has the same failure under other names. In BeautifulSoup, get_text() with no argument returned ₹1,249₹1,39910% off on the same markup, while get_text(" | ") returned ₹1,249 | ₹1,399 | 10% off and stripped_strings gave the three values as a list. lxml’s text_content() behaves like textContent. Always pass a separator or iterate the strings.
Check the fix on saved pages
A browser crawl usually saves the raw page next to what it extracted. Run the new script offline against a saved page in a local Chromium and compare: on the test crawl’s saved pages it returned '₹1,249', '₹1,399', '10% off' as separate values. Across 208 saved pages and 7,932 cards, price, list price, discount, rating and product image were readable on every card. Records stored before the fix can be filled the same way, from their saved pages, without fetching anything again. To find what else a card shows that the record never keeps, audit what the scraper parser drops against those same saved pages.
The short version
innerText gives you text as rendered, and inline neighbors with no whitespace between them in the markup render as one run, whatever spacing CSS draws. Prices, list prices, discounts and rating counts are exactly that shape on product cards. Read the card’s text nodes with a TreeWalker and keep its images, store them beside any string your code already reads, and in static parsers always pass a separator.
Frequently Asked Questions
Can I use Playwright’s all_inner_texts() instead?
It returns one innerText per element a locator matches, so it separates values when you can target each value’s own element. On sites with generated class names those selectors break often. Walking a card’s text nodes needs only a selector for the card itself.
Why not split the glued string with a regular expression?
Because the boundary is ambiguous. ₹1,39910% off reads correctly as ₹1,399 and 10%, but nothing in the string rules out ₹13,991 and 0%. Any split rule is a guess about the data, and the wrong guesses look exactly like right ones.
Does innerText work on hidden cards?
Not as you would expect. An element that is not rendered, such as a collapsed panel with display: none, returns its textContent, so even block-level values run together. Text nodes are read the same way whether the card is visible or not.