4 Comments
May 2, 2023Liked by Maxwell Tabarrok

Isn't it likely that the the bottom % of papers have negative impact? (e.g., Wakefield's MMR autism fraud paper). I think I agree with your overall point but steelmanning the argument for peer review would probably mean drawing an s-curve with the negative spiking left tail causing significant harm.

Expand full comment
Apr 15, 2023Liked by Maxwell Tabarrok

"Although the average input quality increases by the same amount as in the weak link model, the average final impact barely changes."

This isn't true - peer review which filters out the bottom half of research improves the average impact *more* in the increasing returns graph than in the linear graph. Try comparing the areas in the graphs.

Intuition: if every piece of research has nonnegative impact, then the best you can do by filtering out the bottom half of research is to double the average impact of research. That's what happens if the bottom half of research all has zero impact. And the closer the bottom half of research is to having zero impact, the closer you get to doubling. And increasing returns keeps the bottom half closer to zero, relative to the rest of the graph.

Some math which you can check by taking some integrals: If the lowest quality research has 0 value...

... and the graph is linear (y=kx), then removing the bottom half is a 1.5x multiplier on average value

... and the graph is quadratic (y=kx^2), then removing the bottom half is a 1.75x multiplier on the average value

... and the graphic is cubic (y=kx^3), then removing the bottom half is a 1.875 multiplier on the average value

Expand full comment