Industry
Part of Feature updates: methods, tools and useful context
Feature updates comparison: facts, examples and context
Feature updates comparison of six places to look for a change: what published guidance is and is not, and what to do when two sources disagree with each other.
When something changes, you have several places to look and they will not agree. The official help page, the release notes, the developer reference, the in-product text, a write-up by someone else, a forum thread full of people reporting the same thing, and your own logs. Each is a different kind of object with a different failure mode. Ranking them once, in advance, saves you from ranking them in the middle of a bad week when you are motivated to believe whichever one is most comforting.
What to take away
- Published guidance is deliberately general and always behind the code. It is still the best source you have, because every alternative is a guess about the same thing with less access.
- Community reports are a good tripwire and poor evidence. The people who post are the people who lost something.
- Your own data is the only source that is about you, and the only one that can be wrong in ways you can check.
Six places to look, and what each is for
| Source | Good for | Updates | Will never tell you |
|---|---|---|---|
| Help center and policy pages | What the publisher says it wants and forbids | Slowly, often after the change | Weights, thresholds, or the model |
| Release notes | Dated record of what shipped and when | With each release, sometimes late | Why, or what it was replacing |
| Developer reference | Precise behavior of interfaces you call | With the interface | Anything about surfaces you do not call |
| In-product help text | What the feature does for the account looking at it | Per rollout wave | Whether other accounts see the same thing |
| Third-party write-ups and forums | Early notice that something moved | Within hours | Whether the reports are representative |
| Your own logs and analytics | What happened on your property | Continuously | Why, or what happened to anyone else |
The middle column is the one people get wrong. A help center page and a forum thread do not sit on the same scale of reliability with the forum slightly lower. They answer different questions. A help page cannot tell you that something broke this morning. A forum cannot tell you what is intended. Asking either for the other's answer produces confident nonsense.
What published guidance is, and is not
It is a statement of intent, written to be defensible for a long time and across every account the publisher has. That is why it reads as vague. Vagueness here is a feature of the document's job, not evidence of bad faith.
It is not a description of the implementation. It is not a promise about outcomes. It is not complete, and it does not claim to be. It usually cannot say when a behavior started, because behaviors arrive through staged switches rather than releases, and a page written for users has no way to express that some readers are on and some are off. If you want the mechanism behind that particular gap, it is the ordinary use of feature flags: the code ships to everyone, the behavior is turned on gradually, and no single document is true for the whole population during the window.
Given all of that, it remains the best available source, for one reason that survives every complaint about it. It is the only source with the authority to say what the publisher is trying to do, and the publisher is the only party that can act on that intent. Everyone else in the list, including the careful analysts, is inferring the same thing from the outside with less information. Preferring an outsider's confident account over a publisher's cautious one is choosing certainty over accuracy.
Where a changelog earns its place
Between the vague help page and the noisy forum sits the dated record. A proper changelog gives you something the other sources cannot: a timestamp you can line up against your own numbers. That is the single most useful property for diagnosis, because most of the work in explaining a drop is establishing what else happened in the same window.
Two cautions. Release notes describe what shipped, which is not the same as what turned on, so the date is an upper bound on when behavior could have changed and not the date it did. And changelogs are usually written for the people who consume an interface, so anything that changes only the user-facing product may not appear at all. Keeping your own dated log of what you observed, alongside theirs, is covered in the guide to tracking feature changes.
When two sources disagree
Prefer the source closer to the mechanism, and note the disagreement instead of resolving it. A developer reference beats a blog post about the same interface. A blog post beats a summary of the blog post. Your own logs beat all of them for the question of what happened on your site, and lose to all of them for the question of why.
Disagreement that persists past a rollout window is worth escalating into a question rather than an answer. The most common resolution is unglamorous: two people were looking at different populations, different surfaces, or different date ranges, and both readings were locally correct. Checking the population before checking the claim resolves more of these than any amount of argument. For the specific case where the disagreement is about availability rather than behavior, the status tracking material covers what the official channels can and cannot confirm, and the wider background on what these announcements are made of sits in the overview of feature updates.
Common questions
Is an unofficial write-up ever better than the official page?
For timeliness, often. Somebody usually notices a change before it is documented. For intent, never. Use the write-up to know that something moved and the official page to know what it is for.
How much weight should a forum thread carry?
Enough to make you check your own data, and no more. Reports are self-selected: people post when something goes wrong and stay quiet when it does not, so a thread of fifty complaints tells you nothing about the other fifty thousand accounts.
Should I contact support to confirm a change?
It is worth doing and worth discounting. Front-line support sees the same public documentation you do, plus a script. A clear answer is useful; a vague one is not evidence that nothing changed.
What if there is no documentation at all?
Then you are reasoning from behavior, and you should say so out loud whenever you write about it. That is also the moment to read an announcement carefully if one exists, which the notes on reading an announcement cover, and to remember that the general structure of ranking signals constrains which explanations are even available to you.
