AGI

This whole debate summarized:

"Maybe, but maybe it looks like this"

I think I agree that the last 20% is going to be insanely hard for AI to learn. Probably harder than all the rest per Pareto. What is this 20%? Almost certainly subjective, non-verifiable tasks.

The only thing I could see changing my mind on this is if the recursive self-improvement loop really started taking off.