For other old systems, such as the PDPs, Vaxen and others, there's simh (http://simh.trailing-edge.com/ and https://github.com/simh/simh).
[1] http://www.ic0nstrux.com/products/gaming-systems/xgamestatio...
[2] https://www.amazon.com/Black-Video-Game-Console-Design/dp/06...
You will want to have a general idea of how a computer is internally structured but at a deeper level than is usually presented in popular material. I would suggest at least skimming through Tannenbaum's "Structured Computer Organization" (far more amenable to skimming than Patterson & Hennessy) and gleaning the high level concepts. Some may suggest Petzold's "Code" but I feel it's more aimed at a general audience than a technical audience. Others may suggest Nisan & Schocken's "The Elements of Computing Systems" used in their Nand2Tetris course but that's probably too low-level for what you want.
To get started with physical hardware, Ben Eater has videos for building a breadboarded computer based on the 6502 processor, which was used in the Apple II and Commodore 64: https://www.youtube.com/playlist?list=PLowKtXNTBypFbtuVMUVXN... that you can use to build your first computer. He also has parts kits available to make things easier: https://eater.net/6502. Some may suggest the videos he has for building an 8-bit computer from scratch (https://www.youtube.com/playlist?list=PLowKtXNTBypGqImE405J2...) but that also might be too low level for what you want.
From there, you'll probably understand enough to build breadboarded machines using other processors. Switching to an Intel 8088 which was used in the IBM PC will give you a 16-bit machine and the Motorola MC68000/MC68008 which was used in the early Macs, both of which are available in breadboardable DIP packages on eBay and elsewhere, will even give you a 32-bit machine to play around with.
I can totally recommend going for a 6502 because the community is welcoming and there's still a lot of people in the demoscene hacking together projects with it. Lots of emulators too, so you don't necessarily need to debug with an oscilloscope :D
There's also the C64 mini that recently came out but I didn't get my hands on one yet so I dunno whether it's 1:1 compatible with the old instruction sets.
If you want to work with real things in your hands, the cheap and easy way these days is to get an Arduino[2] (5 Volt I/O, and a bit slower) or Raspberry Pi Pico (3.3 Volt I/O, and quicker) and build something.
The Raspberry Pi Pico is a machine that has 2 cores, a lot of IO and a hugely powerful I/O engine called the PIO, which is fast enough to output video (VGA or DVI/HDMI) or almost anything else you need done.
[1] - https://www.nand2tetris.org/
[2] - https://www.arduino.cc/
[3] - https://www.raspberrypi.com/products/raspberry-pi-pico/
Outside of buying an off the shelf machine and playing around with it, you could just build your own machine, using a popular CPU from that era, like a z80 or a 6502. I'm more of a z80 guy in that regard, so I can speak more to that world. There, I can say that there is a LOT of information, manuals, plans, books, videos, etc "out there" to help you with the process[1]. The same is probably true for the 6502 but I have less experience with that world, so can't say so from first hand experience.
Another interesting thing to do would be to follow Ben Eater's Youtube series[2] where he walks through building an 8-bit computer basically from scratch. He really digs down to the very lowest level of how things work as he builds up his machine.
Finally, I might recommend the books Code by Charles Petzold and The Elements of Computing Systems: Building a Modern Computer from First Principles by Noam Nisan and Shimon Schocken, and the associated website for the latter[3].
Neither is "about" retro-computing per-se, but both books work on explaining computing fundamentals at a very low level, in a way that's generally easy to follow, and which maps pretty cleanly to understanding something like an 80's era 8-bit CPU.
And, of course, once you have a handle on simple 8-bit machines, and have the basics mastered, you could start looking to branching out into building and/or buying more sophisticated machines based on 16 or 32 bit architectures.
People have done some wild stuff in the name of retro-computing. You'll hear stories of people who put IBM mini-computers like an AS/400, or even a full-fledged IBM mainframe[4] in their home. Be careful, this can be addictive... :-)
[1]: http://www.z80.info/
[2]: https://www.youtube.com/playlist?list=PLowKtXNTBypGqImE405J2...
[3]: https://www.nand2tetris.org/
[4]: https://www.channele2e.com/technology/datacenters/college-st...