AI can make a weak comparison look convincing

A useful report starts with a clear brief and ends with a decision the evidence can support. AI speeds up production and makes those judgments more important.

Long before AI, a new leader could ask a team to show improvement and receive a chart that looked more encouraging than the result deserved. An analyst making one of their first charts might be pleased to report a four percent revenue increase. The missing question is what that increase is worth to the business.

This is partly a problem of direction. The person leading the work needs to know which decision the analysis should inform and ask for the evidence needed to make it. A request to “show growth” gives a new team less to work with than a request to explain what changed, what the business gained and whether the growth can continue.

AI can carry that familiar problem into a finished report much faster. Anshu Chimala's September 1 article in Lenny's Newsletter offers a useful review method. His subject is AI design: how to get beyond familiar layouts and improve the work through criticism. The part worth bringing into research is his use of a separate agent that sees a screenshot without the code or the conversation that produced it. The reviewer has to judge what is on the page.1

For McQueen Analytics, the review has to check the research behind the page. The finished report is where the client encounters the finding and decides what to do with it.

A yellow pencil rests on a printed blue line graph

the numbers can be right while the chart exaggerates the change

Consider a simple example using invented figures. One survey records 52 percent; a later survey records 56 percent. Both numbers appear correctly on a column chart. If the columns begin at 50 instead of zero, the second column is three times the height of the first.

The measured difference is four percentage points, about a 7.7 percent increase relative to the earlier result. The shapes suggest a much larger change because most of each column has been removed. A reader can miss that even when the axis labels are technically present.

The Office for National Statistics advises starting bar and column chart axes at zero because their lengths represent the values. The visible proportions should match the comparison being made.2

Two charts of invented 52 and 56 percent values. The first begins at 50 and exaggerates the visible difference; the second begins at zero.
Both charts use the same invented values. Removing the first 50 percentage points changes the apparent size of the increase.

Restoring the baseline repairs one problem. It does not tell us whether the rise reflects a real change in the population. The surveys might have different samples, different question wording or enough uncertainty that the apparent movement would not support the proposed decision. None of those questions can be answered by making the columns more attractive.

a better chart cannot repair a changed question

Suppose the earlier survey asked whether respondents were “very confident” and the later one combined “very confident” with “somewhat confident.” The percentages describe different answers. Drawing a line between them would suggest continuity that the questionnaire does not provide.

Before the designer begins, the research record has to preserve the exact wording, the response options and the people represented by each result. An agent that receives only a tidy table has fewer ways to recognize that the comparison has changed.

McQueen Analytics has more than twenty years of studies and a question database shaped by that work. The company has said that agents can compare questions across waves and keep calculations tied to their sources.3 Bringing those records into report production would let the reviewer check what a number means before approving the story around it.

Hypothetical 52 percent very confident compared with 56 percent very or somewhat confident. The response groups differ.
The baseline is only one check. Results also need to represent comparable answers from comparable people.

The same discipline applies when the wording is consistent but the estimate is uncertain. ONS guidance recommends showing uncertainty when it changes how the reader should interpret the result. It also recognizes that adding every uncertainty range can clutter a chart without helping the reader.4 The editor has to decide which qualification will help the reader use the finding responsibly.

a large percentage can describe a small business gain

Consider a hypothetical consultant's traffic test. A website goes from 30 visits to 90 visits over equal periods. Advertising a 200 percent increase would be arithmetically correct. It would also leave out the starting point and the absolute gain of 60 visits.

Sixty additional visits may be a useful result for the business. A competitor with 3,000 visits would need evidence from its own audience, traffic sources and demand before expecting the same experiment to produce 9,000. Evidence from a small test cannot establish the same return at a much larger scale.

A hypothetical increase from 30 to 90 visits equals 200 percent and 60 additional visits. It does not establish a forecast from 3,000 to 9,000 visits for another business.
The relative increase, the absolute gain and the scale of the business answer different questions. All figures are hypothetical.

The practical value needs its own calculation. A four percent revenue increase means $100 million on a $2.5 billion base and $4 on a $100 base. Even then, revenue alone does not show whether the additional business was profitable. For a traffic experiment, the next questions concern customers, their value and the cost of reaching them.

Small starting numbers leave room for spectacular growth rates. They do not remove the limits of the market. A forecast needs an estimate of how many customers could buy, how many the business can reach and what happens to the cost of winning the next customer. The early growth rate cannot answer those questions by itself.

For McQueen, the research brief should ask for the starting value, the absolute change and the decision the result might alter. A team can then examine whether the improvement warrants action before anyone chooses the most impressive number for the headline. The same direction is needed whether the work is assigned to a person or an agent.

a fresh reviewer can show what the page says

Chimala gives the critic a fresh view of the result, separate from the builder's earlier reasoning and the effort already invested in a layout. His examples concern interface design. Applying that separation to research would require a reviewer who can also check the study.1

An analyst who has spent days with a study knows what every abbreviated label means. The client may open the report between meetings or receive a single page forwarded by a colleague. A qualification that was clear during the presentation can disappear when that page travels on its own.

A separate reviewer can read the finished page and explain what it appears to say. That reading can then be compared with the analyst's conclusion. If the reviewer thinks the study proves that confidence increased, while the analyst can defend only the later estimate, the difference identifies a concrete editorial repair.

That repair may be in the headline. A chart showing 56 percent can sit under “Confidence is rising” or “56 percent report confidence in the latest survey.” The first title makes a claim about movement; the second describes a result. The supported title depends on what the study can defend. The review has to check movement before it approves the headline.

the recommendation page needs two reviews

At McQueen Analytics, a practical next step is to put one completed report through both reads. The visual reviewer receives the pages, the intended reader and the decision the report should inform. The research reviewer receives the underlying evidence and the interpretation taken from those pages. A model can help flag a disagreement; the analyst remains responsible for resolving it.

The recommendation page is a useful starting point because the cost of a misleading emphasis is clearest there. A heading, a number and a short paragraph often carry most of the decision. They need to make the same claim, with the necessary limits visible at the point of use.

The heading also needs to work with the prose around it. Read the sentence before the heading, the heading and the sentence after it. If the heading turns a cautious paragraph into a confident promise, changing the paragraph alone will leave the strongest claim on the page untouched.

Two report reviews: read the page for its claim and intended action, then check the research for comparability and support.
A proposed review for McQueen reports: compare the interpretation of the finished page with the evidence behind it.

A correct analysis can lose its meaning in a cropped axis, a missing qualifier or a headline that promises more than the study found.

The person leading the work has to set the question clearly enough for the team to investigate it. The report is ready when the client can see what changed, what it means for the business and which decision it informs. If those points depend on a spoken correction from the analyst, the page needs another edit.

Source notes

  1. Anshu Chimala, “How to turn your AI into a world-class designer,” Lenny's Newsletter, September 1, 2026. His practitioner examples support the description of screenshot-based independent design criticism. They are not empirical evidence of research accuracy or client comprehension.
  2. Office for National Statistics, “Axes and gridlines.” The zero-baseline guidance applies to bars and filled areas whose lengths or areas encode values. The 52 percent and 56 percent figures here are invented to demonstrate the effect; they are not McQueen findings.
  3. “We are using AI to do better research,” RadiationBox, September 1, 2026. Source for McQueen Analytics' research history, question database and stated capabilities for agents. The report-review trial proposed here is not presented as an adopted or measured company process.
  4. Office for National Statistics, “Showing uncertainty in charts.” Guidance on showing uncertainty when it changes interpretation, while avoiding unnecessary visual complexity. The question-wording, website-traffic and revenue examples in this article are hypothetical. They are not company results or forecasts.

Earlier read: we are using AI to do better research