What are those books for CS?
For instance, there is a whole subset devoted to turing machines, the halting problem, finite state machines, provability, incompleteness, the set of all sets, diagonalization proofs etc
There is a whole subset devoted to programming languages: computational complexity, reducing the time spent in loops, and so on.
Then there is a whole bunch spent on best practices: git and other version control, review, commenting style.
Then there is this actual thing called programming: learning different languages, arguing when functional is better than object oriented, please build a web server in prolog as an exercise, and so on.
Then there is a bunch about operating systems, and where the hardware meets the software, and garbage collection, and parsing and all that stuff.
Then databases are their own thing: boyce-codd normal form, ACID properties, is the web a database? how do you prove the ACID properties? Distributed databases, and B-trees time for storage and retrieval.
These are just various sort of "verticals".
I myself like to think of it in terms of specializations: data science/machine learning is one, web apps are another, mobile apps are another and so on. Each specialization depends on what useful thing you are trying to achieve.
So the closest parallels are original research papers like Turing's and Shannon's; but you're probably better off reading more recent textbooks.
That said, to balance out the heavy emphasis on programming and engineering books here, some recommendations:
- Sipser. Introduction to the Theory of Computation.
- Russell, Norvig. AI: A Modern Approach.
- For algorithms I don't have a preference. There is Dasgupta-Papadimitriou-Vazirani; Cormen-Leisersen-Rivest-Stein; Kleinberg-Tardos; and more.
A bit farther from core CS, I also like:
- Cover, Thomas. Elements of Information Theory.
- Li, Vitanyi. An Introduction to Kolmogorov Complexity and Its Applications
Computer Systems: A Programmer's Perspective has a lot of good explanations of hardware/os subsystems that can be hard to find detailed explanations of elsewhere. I especially like the sections on virtual memory.
This one's for you if you want to learn or just recap principles of computers (and read on evolution as well). I just started reading it and found it suprisingly easy to follow. It's perfect if you like things explained step-by-step and in a simple way.
It's an extensive and well-rounded look at various fundamental system administration topics with examples included from multiple different *nix distributions (topics like work management, logging, networking, security, the kernel, systemd, permissions, etc.). If you're at all interested in Linux, this is a great book to have on your shelf.
I dispute this idea. For some disciplines, this may be so. For other, such as physics, biology, chemistry, mathematics, civil engineering, poetry...actually, for most disciplines they aren't based on fundamental books. They're based on a living community of practice.
In these disciplines a book is important only insofar as it can be used to increase your abilities. No one goes and studies the textual details of Landau & Lifshitz as a guide to anything, any more than they do so with TAOCP.
Rather than worry about fundamental books, worry about opening up black boxes at a steady rate.
Then I studied a matrix of math books, along with other CS books regarding compilers, operating systems etc. My aim was to get good enough understanding to comfortably digest the famous research papers.
I am yet to find definitive books that teach a way of thinking on their own merit as a single book. Cs like others is a matrix of knowledge that clicks as a whole 'chunk' of understanding, changing the way you have thought about things like programming in general. I'm having a hard time explaining this.
If you're not a CS graduate, I highly suggest simply going through academics of places teaching theoretical CS.
If I had to say one text that is the fundamental one, I would go with a paper: “On Computable Numbers, with an Application to the Entscheidungsproblem” [1], by Alan Turing.
---
But if you just started your course in Computer Science, then I will give you some “bookshelf advice" based on my experience. I had the pleasure to study all the books I am going to recommend you, and most of them I was able to read cover-to-cover during the university years. And they are also in my bookshelf for reference.
The book order does not represent rank of importance.
* “Introduction to the Theory of Computation”, by Michael Sipser.
* I also recommend the “Elements of the Theory of Computation”, by Christos Papadimitriou.
* Another good complement is “Introduction to Automata Theory, Languages and Computation”, by Hopcroft and Ullman. (Thanks, @reacweb, for the reminder.)
* If you feel you want to go deeper, and like a book which you cannot skip a single word, I strongly recommend “Modern Graph Theory”, by Béla Bollobás: it is one of my favorite textbooks ever!
* While a lot of people seems to praise either the Cormen et al. or the Sedgewick books, I have the feeling that “Algorithms” by Dasgupta, Papadimitriou and Vazirani is my choice for “fundamental” book. But I decided to
* “Modern Operating Systems”, by Andrew Tanenbaum.
* “Artificial Intelligence: a Modern Approach”, by Russel and Norvig.
* “Modern Compiler Implementation (in ML)”, by Andrew Appel.
I would like to also recommend “Concrete Mathematics”, by Graham, Knuth and Patashnik, but I remember to not feel it the most pedagogical book on the subject.
Good luck!
---
[1]: https://www.cs.virginia.edu/~robins/Turing_Paper_1936.pdf
Structure and Interpretation of Computer Programs
Purely Functional Data Structures
Programming in the 1990s
A Logical Approach to Discrete Math
[0]https://www.nand2tetris.org/book [1]https://www.nand2tetris.org/
It's a book that explores hardware from the ground up. It's one of the most insightful books I've found for getting a real intuition for hardware.
- Theoretical CS: - Compilers: Principles, Techniques, and Tools (The Dragon Book) - Introduction to the Theory of Computation, Sipser
- Programming: - Java Concurrency In Practice, Brian Goetz - Generics in the Java Programming Language, Gilad Bracha - Professor Frisby's Mostly Adequate Guide to Functional Programming - Unix for Poets - Modern C++
- Discrete Mathematics And Its Applications, Kenneth H. Rosen (this is more foundational, but it's definitely targeted at a CS audience and touches on things like automata theory)
- Types and Programming Languages, Benjamin C. Pierce
- Semantics of Programming Languages, Carl Gunter
Data Structures and their relationship with each other to be a great technical programmer. These books (just highlight not everything) I would think -> Algorithms Sedgewick, Lisp Programmers Manual, Designing Distributed Systems. .To be a great collaborative programmer (in a work enviornment) -> Pragmattic Programmer, Code Complete, Mythical Man Month, A Philsophy of Software Design by Ousterhout. For philsophy of programming itself -> The Soul of a Machine
Edit: Programming is very broad and operates in many context from the technical, social, to the ideas behind it. I would say it is almost to early to have those definitive fundamental books
2. "Programming Pearls" by Jon L. Bentley IMO this is the most concise yet accessible gateway to the inner Mysteries of Computer Programming. Read between the lines, the prime thesis is implied not explicit.
3. "Thinking Forth" by Leo Brodie (Available as a PDF here: http://thinking-forth.sourceforge.net/ ) Most software is grossly too large. This book shows how to grok and extract the essence of your problem into minimal code. (Chuck Moore had a 3D wireframe CAD program he used to carry around as a deck of punch cards in his shirt pocket.)
The actual science in computer science? Weyuker, Sigal & Davis - Complexity, Computability and Languages; CLRS - Introduction to Algorithms. Harel - Algorithmics, the spirit of computing; And nand to tetris, in whatever format you find it.
Feynman Lectures on Computation: https://www.amazon.com/Feynman-Lectures-Computation-Frontier...
In pure Feynman style he builds up a the theory of computation from first principles. YMMV but it really helped me put many of the ideas and concepts from my CS MS into context and find new ways in which they relate with each other.
If you want to learn how a computer does what it does at the absolute lowest levels, it's a great starting point. It's all written in laymans terms so it's good as a conceptual overview or for people without a strong math background. If you're interested in diving super deep into that kind of stuff I imagine it would be a great high level overview to anchor yourself.
(1) "Understanding Computation" by Tom Stuart. Not "fundamental" as a deep textbook, but very approachable for programmers intro into a big chunk of CS, explaining deep ideas about languages using rigorous working clean code (in Ruby, no prior knowledge needed). I especially loved the first few chapters about what it means to define a programming languange and various kinds of formal semantics.
(2) Designing Data-Intensive Applications, Martin Kleppmann. This gives you a phenomenally good survey of concepts and practice of distributed systems. This is more software engineering than pure CS, but in my view you can't approach the field of distributed systems without blending both anyway.
(3) POODR — Practical Object-Oriented Design, in Ruby, by Sandi Metz. This is 100% software engineering, where there is no single definition of "foundational", but many people who read this swear by it. It's remarkably thin but lucid distillation of ideas that were "in the air" but Sandi nailed them down. An important thesis is that good code is not an aesthetic judgement of how it _now_ looks, but objective question how easy it will be to _change in the future_. Not Ruby-specific at all, but it teaches the original Smalltalk "message-passing" view of OOP, that for people that only learnt statically-typed Java, C++ etc view of OOP is a fundamental idea they're missing on.
Finally, not a book, but "the morning paper" https://blog.acolyer.org/ is excellent "return on your time" if you want to sample academic papers, both classic foundational ones, as well as cutting edge.
Chapters: Introduction to Computers, Computer Organization, The Theory of Computation, Coding and Information Theory, Reversible Computation and the Thermodynamics of Computing, Quantum Mechanical Computers, and Physical Aspects of Computation.
First comment is I did get the Knuth books but I never got the college courses and problem solving work to use them. As Professor Marc Nicolet said: You have got to do the problems to master the subject (referring to his Caltech area of physics).
The second comment is computing is all binary. Go beyond that and learn about fractals, chaos, topology and knot theory. For instance, in Kaufman's Knot Theory, there are obviously true but rigorously unproven theorems in the first chapters of the book.
The third comment is after Turing's paper and Godel's uncertainty arguments, all real world computer programs, no matter how carefully they employ something from a fundamental book, must be tested.
> Alan Kay has famously described Lisp as the “Maxwell’s equations of software”.
Maxwell's equations could be considered fundamental axioms in electromagnetism and physics. Lisp, likewise, expresses the heart of computer science at its core. I'm still working on grokking it myself, but believe that studying Lisp could be precisely what you're looking for (even if you also mean writing software in addition to strict computer science).
Paul Graham's essays could prove interesting as well.
The last time I did any kind of math that was more complex then simple operations was a long time ago and math was never my strong subject.
I have pretty much forgotten any kind of algebra, calculus etc I learnt. So assuming I know very little math, what is the best way/book/method to learn algebra/calculus etc to tackle SICP ? I would like to do that before I begin my SICP journey.
Any suggestions appreciated!!
I realize that for practically any topic covered by this book that there are more heavily optimized solutions available in the best numerical libraries, but this book nicely rounds out my reference books with clear and easy to understand coverage of subjects I don't know. (e.g. Savitzky-Golay Smoothing Filters).
I enjoy reading this book.
It's detailed enough to use when trying to understand how a shell works or how to write a signal handler, but the writing couldn't be clearer.
I don't have a concrete recommendation off-hand - but certainly something by Codd on databases/normalization should be on such a list?
A lack of understanding data normalization can hurt even "simple" code that uses structs and procedures.
Software should be written mostly with humans (code readers) in mind, and not machines. The soft sciences are much harder to study; it's expensive to use trial and error on real projects in order to compare outcomes. Equations on a blackboard or chips in a lab are probably not where most real IT revelations will come from.
There are pet theories that focus on overly-narrow factors. For example, those who focus on linguistic parsonomy (less code) may conclude that functional programming is "better" because it usually can be made the most compact. However, a good many coders have a hard time working well with functional programming. Some take a Darwinian view and say "fire them all and hire elite functional coders". But so far that has rarely paid off in the market-place, barring a few grow-fast-at-all-costs startups. Short code is not always legible code to the majority of coders.
Code Complete by McConnell
[0] https://news.ycombinator.com/item?id=20729252 (What book to read to get a footing in CS theory?)
Patterns of Enterprise Application Architecture by Martin Fowler.
Domain Driven Design by Eric Evans.
Growing Object-Oriented Software, guided by tests by Steve Freeman and Nat Pryce.
I find it funny 'Structure and Interpretation of Computer Programs' is considered fundamental when it's basically an MIT textbook. Presumably it's a very good textbook, but it's not fundamental because the rest of the world doesn't really study it. The world does study Aristotle.
CS is also a wide field (and I presume you said this on purpose to separate the software engineering books?). Something by Chomsky for formal grammars comes to mind.
don knuth's art of computer programming [if you like some rigor][it is a bit time consuming to read the whole thing , but be sure to atleast try reading the first volume]
Software Engineering? no clue
And I am surprised no one mentioned Operating System Concepts (the dinosaur book).
It was the most useful CS text I read in college. I see a lot of SICP love; I never had it for a college course, but it looks good as well.
It's not like classical art or architecture where the principles closer to all commanding or 'figured out'. Your everyday work is like swimming through code solving problems as you go. Pushing blocks into permanent place never works, they find a way of getting crufty, old and replaced to fit new requirements. I imagine any field that deals with flowing quantities of objects, or fluid-like systems would have structural overlaps with programming. Fields like plumbing, finance, the energy sector, ect.
I would describe programming to be at it's very base to be ascribing meaning to quantities of meaningless numbers and building a structure to pipe those numbers around to a qualitative result.
Measuring principles in depth here is not that useful, the river of valuable information runs left to right in code. You'll likely restrict how deep and how abstract you go and instead focus on churning out a high quantity of solved problems. It's just not that useful to be thinking in machine code when you're programming in C++, but knowing all the functions of C++ and where it should be ending up is very important.