Reviews
How to read an annual trend report in about twenty minutes
Annual trend reports as a practical reference: how one gets made, reading it in twenty minutes, description against prediction, and taking a single decision.
The trend report is a genre, and like every genre it has conventions that tell you more than its contents. Learn how one is produced, and you can read any in twenty minutes, take one decision from it, and score it a year later.
This page is the reference for that routine. It names no report and quotes no figure, because the routine is the same whichever one is on your desk.
What to take away
- A report is produced backwardsthe publisher's reason for publishing comes first, the narrative second, and the data is gathered to fill the narrative. Read it in the order it was made, not the order it is printed.
- Separate description from prediction on every page. The descriptions are usually fine. The predictions are the marketing.
- Keep last year's report and score it. Nobody else will, and a report's track record is the only evidence of its worth.
How a report gets made
Every annual report publisher has a reason other than "the public needs to know". A tool vendor wants to be cited. A platform wants a year that flatters its surfaces.
An agency wants to look like it can see ahead, and a survey house wants press coverage. None of these is disreputable, but all of them shape the document.
From the reason comes a narrative: a few themes the publisher wants associated with itself, then the data.
The data comes from what the publisher reaches cheaply: its own product usage, a survey of its own customers or a panel it can rent, and public sources. The survey methodology section, when there is one, is where the reachable population is quietly described. A report about "marketers" is about the marketers who answer that publisher's surveys.
Finally the charts are made, and the ones that do not support the narrative do not appear. This is not fraud. It is editing, and the result is a document that is systematically more confident and more coherent than the year it describes.
The general version of this filter is publication bias: interesting results get published, dull ones do not. A trend report is a small private instance of it.
Which layer of a platform the report describes matters as much as who wrote it. The slow product layer separates from the fast content layer in what beginners should know about platform change.
Reading one in twenty minutes
Read it in the order it was produced.
Twenty-minute reading order
- Publisher and sponsortwo minutes
- Methodologyfive minutes, none means downgrade
- Definitionsthree minutes, check changes
- Charts without captionsfive minutes
- Predictions markedfive minutes
- The publisher and the sponsor.Two minutes. Who pays, what they sell, and which of the report's themes would help them sell it.
- The methodology.Five minutes, and if there is none, stop and downgrade everything that follows. Who was asked, how many, how they were reached, when, and what the response rate was. Note any question wording quoted.
- The definitions.Three minutes. What counts as an "active" anything, what the time period is, what region. A trend is a change in a defined quantity, and a report that changed the definition since last year is describing two different quantities.
- The charts, without the captions.Five minutes. Read the axes first. Percentages without bases, changes without the level they changed from, and growth rates on tiny starting numbers are the usual three.
- The predictions, marked.Five minutes. Go through with a pen and mark every sentence that is about the coming year rather than the past one. These are the sentences to score in twelve months. They will be far fewer than the report's tone suggests, and the ones about content usually expire before the scoring date, for the reasons set out in how a content trend rises and dies.
Description or prediction
A description says what a measured quantity did over a stated period in a stated population. It can be wrong, but it can be checked, and it is usually roughly right because the publisher would be embarrassed otherwise.
Description versus prediction
Description
- What it says
- Measured quantity did
- Can be checked
- Yes, now
- Cost if wrong
- Embarrassment
- Where narrative lives
- Rarely
- Reader action
- Report to team
Prediction
- What it says
- What will happen
- Can be checked
- Only later
- Cost if wrong
- Nothing now
- Where narrative lives
- Usually
- Reader action
- Treat as wishes
A prediction says what will happen. It cannot be checked until later, so it costs the publisher nothing now, and it is where the narrative lives. Predictions in trend reports cluster around whatever the publisher benefits from your believing, and they are phrased to be unfalsifiable: a format "will continue to matter", a behavior "is set to grow".
The reading discipline is to strip the verbs. For each sentence, ask whether it reports a number or forecasts one. Report the numbers to your team. Treat the forecasts as the publisher's wishes, which they largely are.
Descriptions have a limit too. A description of last year, however accurate, is information about a system that has since changed. The argument that ranking systems change faster than any description of them is made in what a ranking input can and cannot be, and it applies with equal force to any report about what worked.
Take one decision
A report earns its reading time if it changes one thing you do. Choose the one thing before you close the document, and write it down with the page that prompted it.
Choose one decision
Did the report change one thing you do?
write it down with the page that prompted it
close the document without a new priority
Good candidates are decisions you were already weighing. The report supplied a description that tipped them: a surface to try, which your change register will already have logged; a format to stop; a segment to ask about.
Poor candidates are decisions the report invented: a new priority that did not exist before you opened it. If every report you read generates a new priority, the reports are running your plan.
Most of the decisions worth taking from a report are about the product layer of platforms, which changes slowly enough that a yearly document can describe it usefully. Content-layer trends in an annual report are already over, which is why a yearly document cannot catch one.
Score last year's
This is the step that nobody does and everyone should.
Score last year's predictions
- Keep every report with predictions marked
- Twelve months on, grade eachhappened, did not, cannot tell
- Tally by publisher for a track record
- Drop sources that score badly
- Read high scorers first next year
- Close each with a dated single sentence
Scoring teaches you which predictions you believed, and why. Usually they matched what you already thought. The record shows those were no more accurate than the rest.
Close each one like a row in a change register: one sentence, written on a date you fixed in advance.
Common questions
Are reports from platforms themselves more reliable than third-party ones?
More reliable on the platform's own usage numbers, because they see them directly. Less reliable on what they mean, because the platform is the interested party. Read the methodology and the sponsor line with the same care either way.
How many reports should I read a year?
Fewer than you receive. Three or four, chosen by track record, read properly, beat twenty skimmed.
A report contradicts my own numbers. Which should I believe?
Yours describe your audience; the report describes its sample. Both can be right. The interesting question is what is different about your audience, and that is a question worth an afternoon.
What if there is no methodology section at all?
Then you have a document of opinions with charts. Read it as opinion, take nothing from it as a number, and note the absence in the track record.
In this guide
- Annual trend reports examples and how to read themAnnual trend reports examples worth citing, what each one measures, and the method checks that keep a trend score from misleading you during planning.
- Trend reports: six places where measurement tools divergeTrend reports: the six design choices that make measurement tools disagree, why the sample is the deep problem, and what to check before quoting a figure.
- Annual trend reports risks: reading sponsored researchAnnual trend reports risks often lie in framing, not just numbers. Read sponsored social media trend research without treating disclosure as proof.
- Annual trend reports timeline: find the field dateThe annual trend reports timeline runs from design to publication. Why the cover date is not the real date, and the three places to find the field date.
- Before you act on annual trend reports comparisonAnnual trend reports comparison: the six method changes that quietly break a year-over-year series, and what to publish when the series is broken.
- Annual trend reports research: what outlasts the next updateAnnual trend reports research up close: how two annual surveys are fielded, why tactics built on a report age fast, and which work outlasts the next update.
