So far the only book I’ve picked up on the subject is Game Engine Architecture by Jason Gregory. I don’t exactly expect a consensus or for there to be a holy grail of game programming books (like an “SICP for games”) but I’d love to see what you all think.
[1] https://news.ycombinator.com/item?id=34312248
Foundations of Game Engine Development (4 volumes), by Eric Lengyel.