Journal · Ranking
The click gap between rank 1 and rank 3
Every catalogue app we have measured shows a steep drop in clicks after the first result. Product teams treat that drop as proof that ranking is “working”: the best item is on top. Sometimes that is true. Often the first card is simply larger, carries a badge, sits under the thumb, and is the only result fully visible before the fold on a mid-range Android in Bangkok traffic.
In-app search analytics that ignore presentation bias will congratulate a layout change as a relevance win. The Query Intelligence Lab spends a week on this on purpose, after instrumentation is trustworthy. If you cannot tell an impression from a click, stop here and fix events first.
What the gap usually contains
Three ingredients, mixed differently per app. Position bias: people click the first thing that looks plausible. Layout bias: image size, price prominence, “best match” chips. Reach bias on mobile: the lower third of the screen is easier to tap while standing. A carousel of sponsored tiles above organic rank 1 confounds everything; log those impressions separately or admit you cannot evaluate organic ranking this month.
A banking mini-app we taught swapped a compact list for large cards. Rank-1 CTR rose. Rank-3 CTR collapsed further. Downstream bill-pay did not move. They had not improved retrieval. They had made the first card impossible to ignore and the third card easy to skip. The honest report said exactly that. It was not popular. It prevented a bad feature weight from shipping.
A modest way to report ranking
We ask alumni to bring three numbers to a search review, not fifteen. Impression-weighted click rate by position, with a note on card template. A pairwise or team-draft relevance sample on a frozen query set — even fifty queries judged by two people beats a dashboard nobody trusts. Time-to-first-useful-action, where “useful” is defined in the event contract, not in a slide.
If your ranking win disappears when you freeze the card design, it was never a ranking win.
Interleaving without the theatre
Full interleaving infrastructure is more than most mid-size Thai product teams will build this year. A poor substitute that still helps: freeze layout, swap only the order of organic results for a slice of traffic, and read downstream actions — not just clicks. If you cannot freeze layout, do not claim a relevance experiment. Claim a UI experiment and sleep better.
Where click models fit
We teach a simplified examination of examination+click ideas so that “CTR by position” stops being treated as a relevance oracle. We do not pretend you will estimate a production DBN in week four. Literacy is the goal: know what your metric cannot see. If you want deeper instrumentation on small screens, the three-day Mobile Ranking intensive is the narrower tool; the Lab is the wider education. Start at programs or the field guide.