digital marketing, seo, google, emarketing, sem, serp, business, informatics, computers, web pages, diagrams, internet, analyst, search engine optimization, modern, tablet, phone, smartphone, advertising, corporation, competition, analysis, adwords, stylus, google adwords, it, samsung, digital marketing, digital marketing, digital marketing, seo, seo, seo, seo, seo, google, business, business, business, business, advertising, analysis. A ranking signal is not the same thing as a ranking factor
Photo by AS_Photography on Pixabay

Reviews

A ranking signal is not the same thing as a ranking factor

Ranking signals sorted by the five producers of evidence, why a signal is not a factor, four tests for any claimed input, and examples read the right way.

A signal is anything a ranking system can observe about an item, a person, or the moment they meet. That definition is deliberately wide, and the width is the point.

Most arguments about "the signals" are arguments about a short list someone has published, and a short list is what the definition tells you not to trust. The observable inputs to any modern system number in the thousands. What matters is which kind each one belongs to, because the kind decides whether you can do anything about it.

What to take away

  • Sort every claimed input by who produces it. There are five producers, and you control only one of them outright.
  • A signal is not a factor. Observing something and acting on it are separate claims, and public lists routinely assert the second while showing the first.
  • The useful examples are not "do X to rank"; they are patterns that let you tell which kind of input moved when your numbers did.

Five producers of evidence

What it can tell a system

The item itself
Format, length, language, topic, labels, technical quality, when it was made
The publisher's history
Cadence, past response, prior policy record, how long the account has existed
The audience's response
Opens, completions, saves, skips, reports, how fast they arrive
The context of the request
Time, device, location, session length, what was just seen
The relationship
Whether this viewer has interacted with this publisher, and how recently

What it cannot tell it

The item itself
Whether anyone will want it
The publisher's history
Whether this item is like the previous ones
The audience's response
Why anyone did any of it
The context of the request
Anything about the item
The relationship
Whether they still care

Your control

The item itself
Full
The publisher's history
Partial, and slow
The audience's response
None directly
The context of the request
None
The relationship
Indirect

Read down the last column. The only row you fully own is the first. The second you own slowly, by what you did last quarter. The third is the row everyone wants to influence, and it is the one you can only reach through the first two. The fourth and fifth are not yours at all.

Five producers of evidence

Producer

The item itself
Format, topic, quality
Publisher's history
Cadence, prior policy record
Audience's response
Opens, saves, skips, reports
Context of request
Time, device, location, session
The relationship
Whether viewer interacted before

Can tell

The item itself
Whether anyone will want it
Publisher's history
Whether this item is like previous
Audience's response
Why anyone did any of it
Context of request
Anything about the item
The relationship
Whether they still care

Cannot tell

The item itself
Full
Publisher's history
Partial, slow
Audience's response
None directly
Context of request
None
The relationship
Indirect

Your control

The item itself
Publisher's history
Audience's response
Context of request
The relationship

This is why most tactical advice disappoints. It is advice about the third row, dressed up as though it were about the first. You cannot post a save. You can only post something that people save.

A signal is not a factor

Two separate claims hide inside every "ranking factor":

Two claims inside a factor

  1. Claim 1: The system can observe this
  2. Claim 2: The system changes output because of it
  3. Claim 1 is often true and easy to check
  4. Claim 2 is a behavior claim, hard to establish
  5. Gap between them is where superstition lives

The first is often true and easy to check: if the attribute exists in the data, it is observable. The second is a claim about behavior, and it is close to impossible to establish from outside. A published list usually shows the first and asserts the second. The gap between them is where superstition lives.

The two-claim test misses another gap: even when the second claim holds, direction and size are not fixed. An input that helped one surface can be neutral on another, negative on a third.

The word "factor" means an observation, a hypothesis, and a measured effect, so any honest statement about an input says which of the three it is.

Why can nobody outside just ask? Partly secrecy, but mostly because the answer does not exist in a form that can be handed over.

A learned model carries no weight per input a person can read off, and what it does with one input depends on every other input present. This is the ordinary problem of explaining a learned model, and it applies to the people who built the model as much as to you.

Four tests for a claimed input

Before treating any claim as worth acting on, run it through these. A claim that fails two is folklore.

  • Is it observable at all?A system cannot rank on your intent, your effort, or your expertise. It can rank on things that leave traces. If the claim names something that leaves no trace, it is describing a wish.
  • Is it cheap to compute at the scale involved?Anything applied to every candidate has to be cheap. Expensive inputs can only apply to a shortlist, which changes what they can do. See how a feed is forced into stages by the constraints every feed shares.
  • Does it plausibly predict the objective?The system is optimizing a countable proxy. An input earns its place by predicting that proxy. If you cannot say what proxy an input would predict, you cannot say why the system would use it.
  • Would it survive being known?An input that is trivial to fake gets faked, and then either stops predicting anything or gets discounted. Inputs that are expensive to fake are the ones that persist.

There is one more habit worth naming. People notice that the items which ranked all had some attribute and conclude that the attribute caused the ranking. That is the base rate fallacy in a new coat: if nearly everything published has the attribute, then nearly everything that ranked will have it too, and the observation carries no information.

Practical examples, read the right way

These are not recipes. Each is a pattern you can recognize in your own numbers and use to narrow which producer of evidence moved.

Every item you publish falls by a similar fraction on the same day. The item row did not change, because the items are different. The audience row rarely changes evenly. That leaves the publisher row or the context row: something about the account, or something about who is being asked. This is the pattern that most often gets blamed on "the algorithm" and most often turns out to be a measurement or eligibility change.

One item does far better than its neighbors weeks after publishing. Nothing about the item changed. Something in the relationship or context row did: a new audience found it, or it began being shown next to something that made it relevant. This is recommendation working as designed, late, not a mistake being corrected.

A format change improves early response but not completion. You moved the item row and the audience row answered on one measure and not another. Whether that helps depends on which proxy the surface is optimizing for, and you can find that out only by watching which of the two the distribution follows.

Reach is fine in one country and gone in another. Items and publishers do not vary by country. The context row does, and so do the hard constraints that sit outside scoring altogether. Look at eligibility before you look at anything else.

What this leaves you with

Work on the first row, because it is the only one you hold. Watch the third row, because it is the only feedback you get. Accept that the fourth and fifth are weather. And treat any list of "signals" as a list of things that might be observed, not a list of levers.

Common questions

Is there a master list I should keep?

Keep a list of inputs you have seen move your own numbers, with the date and the pattern. A list copied from someone else is a list of their hypotheses about a different account.

Why do the same attributes keep appearing in every published list?

Because they are the cheaply observable ones from the first row, and because lists copy lists. Appearing everywhere is evidence of visibility, not of weight.

Do negative inputs exist?

Yes, and they are usually constraints rather than scores: reports, policy flags, duplicates, expired links. They tend to act as a gate, which is why their effect looks like a cliff rather than a slope.

Can I test whether an input matters for me?

Only roughly, by changing one thing on the item row while holding the rest steady and comparing against items you did not change. Even then you are testing your account on this surface this month. That is worth knowing, and it is the most anyone can honestly claim.

In this guide

  1. Why there is no honest list of ranking signals, and the four stages insteadWhy nobody can give you a list of ranking signals, plus the four jobs every ranking system has and how to read a symptom back to the stage it belongs to.
  2. How a claim about ranking signals turns into folkloreRanking signals timeline of a claim, from one account's odd week to folklore: seven stages, why aggregation is the dangerous one, and how to date a claim.
  3. Ranking signals comparison: evidence typesRanking signals comparison for social feeds: what correlation studies, before-and-after tests, and account-level controls can show.
  4. How to run ranking signals research on your social postsRanking signals research on your own social account: choosing a signal to test, building a split that cannot flatter you, and reading the result honestly.
  5. Ranking signals updates under review: the good, the weak and the verdictRanking signals updates under review, ranked: the ones with published documentation, the ones resting on inference, and the verdict on each.
Filed underranking signals

More in Reviews

Latest from Field Desk