Why was assembly language used in early 80s instead of cross compilers?

1. Compilers weren’t as good at optimising back then.

2. If I’d saved up to buy a C64… exactly how was I supposed to get access to a more powerful machine, and why would I use it as a build farm instead of as my main home computer?

3. Graphics and sound were very not portable between different makes of home computer, so you’d have to rewrite your game if you wanted it to run on other makes anyway?

Some people used minicomputers to develop microcomputer software, for instance Infocom developed games like Zork on a DECSYSTEM-20

https://www.filfre.net/2013/03/the-top-of-its-game/

When I was in high school I developed a program for a teacher who had a CP/M machine based on a Z-80 on a much more powerful 80286-based PC using an emulator.

I think you are conflating assembly language vs. C and cross vs. native development.

All four combinations were possible and all were used in the early 1980s too. Cross-assemblers, native assembler, cross compiler, native compiler.

Not everything was a C64 either, but in that example and other early home computers, game, graphics, music tasks required hand written assembly to squeeze out every last ounce of performance and code size. Whether that was cross or self-hosted probably varies.

The choice of cross or self-hosted development probably depends much more on the availability (or non-availability) of a self-hosted toolchain (assembler or compiler), and the availability or not of a host machine than the performance.