Great musings Jason. Part of the reason why innovation seems to slow down after 1950 could also be changing research incentives, not just the “low hanging fruit” effect. As I wrote:
"Since the 1970s, there has been a growing expectation that researchers not only publish frequently but also publish works that are frequently cited by others (have a large impact). Together, these metrics, known as the “h-score,” are a kind of “batting average” for researchers. The more a researcher publishes and the more his work is cited, the more likely he is to obtain grants and further his career."
In other words, research that stays within well-established and understood areas is rewarded with grants while emerging areas tend not to. The former is likely to be cited more often, while the latter is not.
Two big reasons for a slowdown of effective scientific and technological progress in spite of an increase of measurable inputs:
1. Low hanging fruit effects. The new discoveries have a certain tendency to be harder than the old discoveries.
2. Creativity getting swamped by imitation. If you try to scale up creative endeavors, you often get mimetic modified reruns and lose the essential insight generating capacity. Whole ages of intellectual production are sometimes, in retrospect, worthless because of this problem.
In almost any field, it's easy to contrast the greatness of the founders with the mediocrity of their successors. It might be a kind of optical illusion, sometimes, but it underscores how easy it is to slip into stagnation, decadence, and decline.
Re (1), I don't think “low hanging fruit” is sufficient to explain it. The low-hanging fruit may be gone, but there are also more scientists now, more money to fund them, better instruments for them to use, and better communications technologies for them to share ideas with faster. To explain the rate of progress in science you have to explain the balance of those forces; why don't all those advantages allow us to go after more high-hanging fruit?
I read the link. Good points about evolvable institutions there and in this post. I agree about the virtues of VC, but always bear in mind the limits of the profit motive. Lots of great ideas create social value but don't monetize. Maybe the social value creation spills over into externalities. Maybe imitators come in behind and capture it all. "You can always tell the innovators; they're the ones with the arrows in their backs." Somebody, government or universities or private individuals like the Wright brothers just serving mankind in their own dime from altruism and a sense of adventure, needs to work for the public interest.
It sounds like you agree that the low hanging fruit is real, but you don't think it's enough to explain what's arguably a slowdown in scientific and technological progress since the mid 20th century, given that the resources going into science are so much greater. That's reasonable.
But I still think low hanging fruit effects are a huge factor in the history of technology that must always be borne in mind. If you hold everything constant in resources and institutions, you shouldn't expect the rare of technological progress to stay the same. It might slow down because of low hanging fruit effects. Or it might speed up because of "standing on the shoulders of giants" effects. The landscape of technological possibilities is complex and variable, and we can see ahead only through a kind of fog: undiscovered technologies are hard to put into a dataset.
But it's very important and you're trying, which is awesome! Keep up the good work!
Yes, the low-hanging fruit effects are real and important, I agree. Chad Jones has established this pretty well.
I'm just pointing out that they aren't the *only* factor or even the *main* factor. If they were, then progress would have been fastest in the Stone Age! There was *so much* low-hanging fruit in the Stone Age. But of course, that's when progress was *slowest*, with tens or hundreds of thousands of years elapsing between even significant changes to stone tool sets. Last ~50 years aside, progress has been *accelerating* through most of human history, which is evidence that the factors I mentioned outweigh the low-hanging fruit factors.
Great musings Jason. Part of the reason why innovation seems to slow down after 1950 could also be changing research incentives, not just the “low hanging fruit” effect. As I wrote:
"Since the 1970s, there has been a growing expectation that researchers not only publish frequently but also publish works that are frequently cited by others (have a large impact). Together, these metrics, known as the “h-score,” are a kind of “batting average” for researchers. The more a researcher publishes and the more his work is cited, the more likely he is to obtain grants and further his career."
In other words, research that stays within well-established and understood areas is rewarded with grants while emerging areas tend not to. The former is likely to be cited more often, while the latter is not.
Great piece on creating evolvable scientific institutions and the currated summary of links after! Thanks.
Two big reasons for a slowdown of effective scientific and technological progress in spite of an increase of measurable inputs:
1. Low hanging fruit effects. The new discoveries have a certain tendency to be harder than the old discoveries.
2. Creativity getting swamped by imitation. If you try to scale up creative endeavors, you often get mimetic modified reruns and lose the essential insight generating capacity. Whole ages of intellectual production are sometimes, in retrospect, worthless because of this problem.
In almost any field, it's easy to contrast the greatness of the founders with the mediocrity of their successors. It might be a kind of optical illusion, sometimes, but it underscores how easy it is to slip into stagnation, decadence, and decline.
Re (1), I don't think “low hanging fruit” is sufficient to explain it. The low-hanging fruit may be gone, but there are also more scientists now, more money to fund them, better instruments for them to use, and better communications technologies for them to share ideas with faster. To explain the rate of progress in science you have to explain the balance of those forces; why don't all those advantages allow us to go after more high-hanging fruit?
Now, (2) I agree is an issue, see this: https://rootsofprogress.org/stagnation-scientific-incentives-and-funding-models
I read the link. Good points about evolvable institutions there and in this post. I agree about the virtues of VC, but always bear in mind the limits of the profit motive. Lots of great ideas create social value but don't monetize. Maybe the social value creation spills over into externalities. Maybe imitators come in behind and capture it all. "You can always tell the innovators; they're the ones with the arrows in their backs." Somebody, government or universities or private individuals like the Wright brothers just serving mankind in their own dime from altruism and a sense of adventure, needs to work for the public interest.
https://lancelotfinn.substack.com/p/golden-rule-capitalism
Well, I'm not claiming that VC or any for-profit model could fund all of basic research
It sounds like you agree that the low hanging fruit is real, but you don't think it's enough to explain what's arguably a slowdown in scientific and technological progress since the mid 20th century, given that the resources going into science are so much greater. That's reasonable.
But I still think low hanging fruit effects are a huge factor in the history of technology that must always be borne in mind. If you hold everything constant in resources and institutions, you shouldn't expect the rare of technological progress to stay the same. It might slow down because of low hanging fruit effects. Or it might speed up because of "standing on the shoulders of giants" effects. The landscape of technological possibilities is complex and variable, and we can see ahead only through a kind of fog: undiscovered technologies are hard to put into a dataset.
But it's very important and you're trying, which is awesome! Keep up the good work!
Yes, the low-hanging fruit effects are real and important, I agree. Chad Jones has established this pretty well.
I'm just pointing out that they aren't the *only* factor or even the *main* factor. If they were, then progress would have been fastest in the Stone Age! There was *so much* low-hanging fruit in the Stone Age. But of course, that's when progress was *slowest*, with tens or hundreds of thousands of years elapsing between even significant changes to stone tool sets. Last ~50 years aside, progress has been *accelerating* through most of human history, which is evidence that the factors I mentioned outweigh the low-hanging fruit factors.