Learn to decipher data on product claims: expert

Terry Buss struggles to contain his contempt when he sees the phrase “positive yield response.”

Buss, a Manitoba Agriculture soybean specialist in Beausejour, Man., said “positive yield response” regularly appears in literature promoting foliar sprays and soil fertility products. He said those claims are often meaningless because they’re rarely supported by valid data.

“Positive yield response … is that greater than zero?” said Buss, who spoke about the validity of agricultural research during a presentation at Manitoba Ag Days in Brandon Jan. 22.

“I don’t like it.… I’m seeing it more and more. The reason I don’t like it is because… I don’t know what it means.”

Buss said he regularly receives calls from growers who have questions about the latest or greatest agricultural product and associated yield boost claims of five to 10 percent.

“A lot of soil fertility products, a lot of foliar applied things … more and more fungicide type products.”

Buss said the marketing campaigns for such products typically feature data, graphs and tables to support the claims. The information overload can be overwhelming, he added, but it is possible for growers to separate the bona fide from the b.s.

“It’s not that complicated. You don’t need to get a master’s degree or a PhD to evaluate research.”

He said products often cost less than $10 per acre and typically promise yield increases of 10 percent or less.

Growers are more likely to bite when the financial risk is low.

“I’ve had people say that to me. The product is $5 per acre…. If it works, great. If it doesn’t, I’ve only lost five bucks.”

Still, $5 adds up quickly if the product is applied to 1,000 acres.

“Do you have $5,000 to throw away? I don’t,” he said.

“I’m saying, don’t spin the wheel until they’ve shown you something that you consider valid.”

Rigas Karamanos, senior agronomist with Koch Agronomic Services, said several years ago that there is a basic rule for claims surrounding fertilizer supplements: distinguish between Type A and Type B claims.

A Type A claim is when a fertilizer supplement vendor says its product will boost yield by five to 10 percent.

The Type B claim is when the vendor says its supplement will produce the same yield but with less nutrient inputs.

Karamanos said the simple rule for farmers is that Type B is possible and Type A is impossible because yield is determined by crop genetics, solar radiation, water and nutrients.

“When you get to the maximum yield with best management practices … you will find that it is very difficult to push the yield potential (higher).”

Buss said recognizing the quality of agricultural research is another way to decipher product claims. A graph with a low bar and a high bar to represent yield differences isn’t reliable if researchers don’t explain how confident they are in the data, he added.

“Two very common confidence levels that are used are 95 percent and 90 percent confident,” he said.

“I see more and more stuff where there is no discussion of that (statistical confidence) at all.”

Promotional results should also show the least significant difference, he said. Using soybean research on tillage systems as an example, Buss said the least significant difference in that experiment was 2.8 bushels. As a result, any yield difference less than 2.8 bu. is statistically insignificant and should be ignored.

“That’s the type of thing we should be able to get in all of the research we look at, that ability to determine real differences,” he said.

“Anything else won’t do.”

Experimental design is another consideration.

Buss said growers should look for information on site years of data and the number of replications.

Karamanos has touted the importance of replication for years.

“In God we trust. All others bring replicated data.”