Recommendation systems examples: what the cases show

Every case here is invented, and every one of them will be familiar. They are the situations people bring to anyone who claims to understand recommendation, and each has a structural explanation that does not require knowing what any platform does. The method is the same each time: what you saw, what the system could have seen, what it probably inferred, and what evidence would tell the explanations apart.

What to take away

• A recommendation that looks wrong is usually an accurate prediction of the wrong thing. Ask what was being predicted before asking what went wrong.
• Most puzzling cases are cold start, thresholds, or aggregation, and each leaves a different trace in your numbers.
• The distinguishing evidence is almost always something you can check yourself in an hour. The explanations are not the point; the check is.

Case one: the old item that suddenly moved

What you saw. Something published months ago, which did nothing at the time, is suddenly being viewed by strangers every day.

What the system could have seen. The item's attributes have not changed. What changed is the evidence around it: a new item of yours may have brought a cluster of readers whose behavior resembles the old item's few early viewers; or the old item's attributes now match something that has become popular; or a threshold was crossed by a handful of new interactions.

What it inferred. That people in some cluster respond to this item. Late, because the evidence arrived late.

The check. Look at where the new viewers came from and what else they viewed. If they arrived through a recent item, it is the cluster explanation. If they arrived through search or a topic surface, it is the attribute match.

Case two: the follower who never sees you

What you saw. Someone who chose to follow you says they have not seen anything of yours in weeks.

What the system could have seen. A follow is one signal, recorded once. Since then, that person's behavior toward your items has been a run of non-interactions, and non-interaction is evidence too. On a surface with more chosen sources than slots, a stale follow loses to an active one.

What it inferred. That the relationship is weak, correctly by its measure and wrongly by the person's.

The check. Ask them what they did with the last item of yours they saw. If they saw it and scrolled past, the system's inference is a fair reading of the evidence. The two families of evidence a recommender weighs, and why a single follow is a weak one, are in what a recommender can and cannot infer.

Case three: the account with one hit and nothing since

What you saw. One item found a large audience. Everything since has done what everything before did.

What the system could have seen. One item produced strong early response and was pushed. Its audience assembled around that item, not around the account. Later items do not resemble it, or do not produce the same early response from that audience, so the evidence that promoted the hit does not transfer.

What it inferred. Nothing about the account at all. It inferred something about one item, and the inference held.

The check. Compare the later items' early response from the hit's audience against their early response from your previous audience. If the hit's audience is the cold one, the audience was the item's, and the arithmetic of why a spike does not repeat applies.

Case four: the item that lands in the wrong place

What you saw. Something careful and specific is being shown next to material it has nothing in common with, to people who react badly.

What the system could have seen. Whatever attributes are cheap to read: words in the title, a category chosen from a list, a format, a thumbnail. If the cheap attributes resemble a popular category, the item is classified into it before any behavior evidence exists. This is the cold start problem from the item's side: with no history, attributes are all there is.

What it inferred. That this belongs with the things it superficially resembles.

The check. Change nothing but the cheap attributes on the next comparable item and watch where it lands. If the placement follows the attributes, the classification is the cause, and it is one you control.

Case five: the two audiences that cancel out

What you saw. Your overall numbers are flat, yet two very different groups both say they see more of you than before.

What the system could have seen. Two clusters, each growing, whose behavior toward you is opposite: one completes and saves, the other opens and leaves. Aggregated, the signals average to nothing, and the system's overall estimate of the item does not move.

What it inferred. That the item is middling, when it is strong for one group and wrong for the other.

The check. Split every metric you have by any segment you have: source, region, new against returning. A flat aggregate over strongly opposed segments is a known statistical trap, Simpson's paradox, and it hides inside averages until someone splits them.

Case six: the plateau in one country

What you saw. Growth everywhere except one region, where nothing of yours gains an audience however it performs elsewhere.

What the system could have seen. Behavior evidence from that region is either absent or gated. Absence: your items are never shown there, so no evidence forms, so they are never shown. Gating: a constraint outside scoring, such as language, age classification, or a regional rule, removes the item before ranking.

What it inferred. In the first case, nothing, which is the problem. In the second, nothing again, because the constraint acted before inference.

The check. Constraints produce a step; absence produces a flat line from the start. Look at the shape. Constraints are gates rather than scores, which is the distinction drawn in what a ranking input can and cannot be, and they are checked by reading the rules, not by adjusting the item.

Common questions

Why does every case come back to evidence rather than to what the platform wants?

Because the platform's wants are encoded as a countable objective, and the system only ever sees evidence about that objective. Explanations in terms of intent are stories; explanations in terms of evidence are checkable.

Is there a case where the item really is the problem?

Cases four and, sometimes, three. In both, the fix is on the item and the evidence tells you which part.

Can I ask a platform for an explanation of any of these?

Some surfaces show a reason for a recommendation. The reason shown is a simplification chosen for display, and it is worth reading as a hint about which family of evidence was heavy, nothing more.

What if none of the six fits?

Then run the general reach checklist in order, starting with measurement, as set out in the constraints that force a feed to exist. Most cases that fit none of the six turn out to be measurement.