Perhaps the hardware landscape we're in is a shallow minimum that we're just stuck with. Perhaps other computer architectures like Lisp would allow much more efficient hardware in general?
Perhaps OS's and high performance libraries are designed badly leading to inefficiencies all round?