One small insight I've gained from studying the humanities—I'm currently taking a philosophy degree on the side—is that conceptual wins or paradigm shifts have required laborious work and the passage of time.

By "conceptual wins" I mean things like:

- Explaining natural phenomena not in terms of greek or Roman anthropomorphic gods, but with naturalistic, physical explanations
- Understanding acceleration as distinct from motion
- Science as an experimental methodology
- The is/ought distinction
- Bayesian reasoning
- ceasing to accept the divine right of kings as a justification for monarchical governance
- randomized trials as a more robust way of generating generalizable knowledge
- the genealogical argument: understanding that systems (such as the details of the current prison system, our monetary system, or the lack of color in men's clothes) is the result of random historical ocurrences which could have gone differently, even if it has been rationalized afterwards.

One of the classical reasons to be afraid of artificial intelligence might be because, in the limit, it might be able to completely outmaneuvre us because of its superior epistemology. It might do so in the same way in which a current chemist can completely outmaneuvre previous alchemists, that is, by using their superior understanding of natural laws to producing better explosions or more subtle poisons.

You might arrive at this fear by thinking about how systems behave as they become arbitrarily, in the limit. And this might be a useful frame to generate potential problems. But it's not clear whether it applies to current AI systems, as they currently exist. 

It is thus of interest to me the extent to which *current* AI systems are able to come up with better understandings of the world, and thus acquire an advantage over humans. I say "the extent" because I think they might be able to already do this kind of thing. For instance, Robin Hanson has a way of looking at a common human behavior, at its purported justification and at its actual results. Current AI systems such as GPT-3 can probably generate insightful Robin Hanson-like criticisms. But I'm curious about how far this goes.