Getting annual trend reports updates right the first time

Most of what a trend report says about the future is not a measurement, and the document rarely marks that boundary. A projection uses the same typeface and confident tone as a finding and is quoted with the same authority.

This page covers telling them apart, how forward-looking parts are produced, and deciding what a projection is worth to you.

What to take away

Four ways a forward-looking claim gets made

Extrapolation is the most common and least defended. Continuing a line is a real technique, and linear trend estimation does exactly that with stated error bounds. It works at short horizons on a stable series.

Almost every failure of an extrapolated projection is a turning point. Turning points are exactly what the method cannot see.

Expert elicitation done well is a structured process, not a survey of opinions. The Delphi method is the standard formalization: a panel answers, sees the anonymized distribution, then answers again.

It reduces the effect of the loudest voice. When a report says it asked experts, check whether it did anything of the kind.

Forecast and scenario are different products

A forecast says what is expected to happen and can be scored against the outcome. A set of scenarios says what could happen and cannot be scored, because it never claimed one of them would.

Scenario planning prepares an organization for a range of futures rather than picking one. Used properly, a scenario tests whether your plan survives a world different from the one you expect.

Used improperly, the most flattering scenario gets extracted and quoted as a forecast. The whole point of having several is lost.

The tell is whether the document assigns probabilities or commits to one path. If it does neither and still uses the future tense, you are reading a scenario sold as a forecast. Read it for its reasoning, not its conclusion.

A claim that no outcome could contradict has the same shape as any assertion no observation could count against.

Reading a projection

Four questions, and if the document cannot answer them the projection is decoration.

What is the horizon, and is it stated? A projection without a time frame cannot be wrong, which is a reason to discount it rather than to trust it.

What are the assumptions, and are they listed? A projection whose assumptions are hidden is an opinion in the shape of an analysis.

What would falsify it? If nothing would, it is not a claim about the world at all.

Was the last one right? This is the question nobody asks, and it is the most informative. A publisher who reviews their previous projections is telling you something about how they work. Almost none do, for the reason described in why a projection that failed leaves no trace.

Readiness, and the gap it hides

A specific failure worth naming is confusing what exists in a demonstration with what exists in ordinary use. The two are separated by years and by a large amount of unglamorous work, and a projection that treats them as the same thing will always be too fast.

A technology readiness level is a useful check, even in rough form. Ask whether a thing has been shown once, deployed at small scale, or is running in production for people who did not choose it.

Most projections about a fast subject are wrong because they collapsed those three into one.

Where a thing sits on that path also tells you what evidence about it exists, which is the same reading described in placing something on the adoption curve.

What to do with the forward-looking half of a report

Use it as a list of what the authors think matters, which is genuinely informative about the field and costs you nothing to accept. Do not use it as an input to a decision that would be wrong if the projection failed.

If a decision does turn on the future, the useful form is not a better projection. It is asking what would have to be true for each option to be right, and then watching for those specific things. That converts a prediction problem into an observation problem, which is the only version of it anyone can actually solve.

Common questions

Are any trend projections reliable?

Short-horizon extrapolations on stable series often are, and the rest is better handled by a standing watch than by a sharper forecast. Those extrapolations are also the least interesting, because everyone can make them. The projections people want are the ones about turning points, and nothing available produces those reliably.

Should I ignore the forward-looking sections entirely?

Read them for the reasoning and the assumption list. That content is real and useful. Skip the numbers, which carry a precision the method cannot support.

Why do reports include projections at all?

Because readers ask for them and because a document that only describes the past is harder to sell. That is not cynical: a report is a product with an audience, and the interests behind it shape this section more than any other.

How should I write about the future myself?

Say what you expect, say what would change your mind, and put a date on when you will check. That check is the comparison against last year's edition, turned on your own writing. Three sentences, and it is more honest than most of what you will read.