Why certain things can be proved and others not? And in between all these how did Computer science or computation emerge as a branch of mathematics that had the confidence of a mountain that it can solve all issues of humanity even with the warning that all problems can't be proven.
I am myself confused what I am asking about. But, I want answers to all these questions in a comprehensive way.
I will love book suggestions that have answers to all my questions and maybe more.
Edit: I have done my bachelors and masters in electrical and communication engineering. I know calculus, matrix algebra and prob stats on a level of engineering. Not in much depth.
IMO what you need is an overview of the philosophy of mathematics/logic (as someone pointed out)
Being a philosophy, you should not expect definite answers. Mathematics itself is a historical process, so there are always trends and fashions to be aware of.
To get a quick comprehensive map, I personally always start with online articles (before books) on encyclopaedias like Plato [0] or even Wikipedia [1].
>Why certain things can be proved and others not?
One of the most captivating events of the 20th century is how Kurt Gödel's incompleteness theorems "destroyed" Hilbert’s dream of creating pure formalized foundations of all mathematics (without paradoxes and inconsistencies). I don’t think we ever fully comprehended the implications of this.
> how did Computer science or computation emerge as a branch of mathematics
Besides Gödel’s incompleteness, there is Turing's completeness. Turing Machine is the associated model of computation. These models are mathematical abstraction of computers (see "Theory of computation")
[0] https://plato.stanford.edu/entries/philosophy-mathematics/
[1] https://en.wikipedia.org/wiki/Category:Philosophy_of_mathema...
Now, we could invent a conceptual tool to have arbitrary meaning, but without rigid definitions, symbols have no meaning, and therefore they cease to be useful for solving/comprehending anything.
I mean, say the word "XUNS" could mean anything, be it "cat", "sovereign", "hallelujah" or whatever. And I just say "XUNS XUNS XUNS XUNS XUNS". Would you know exactly what I said? No it is nonsense, because, without out giving a strict definition it could mean anything.
So by ascribing rigid definitions/meaning to things, we can now reason about them in sensible ways. Which is what mathematics does, and why it is useful tool for proving/comprehending something.
As for the axiomatic method... it produces truths, mainly because it's built beforehand upon pre-defined truths. By this I mean, if I define an inch to be so long, and measure an object to be 3 inches, then the statement that "the object is 3 inches in length" is true, because we defined it so. If instead, you changed the definition of an inch to be any arbitrary length, then it ceases to be useful, and we can no longer prove exactly how many inches the object is in length.
A few words on your questions:
Mathematics is no different from any other field except that its objects of interest are all abstractions. As an engineer you use it, so it appears to be about number and symbol pushing, but the theory of how and where to push the numbers and symbols and why it works was discovered by a mathematician at some point. The proofs are not ultimate but hopefully convincing given certain usually acceptable assumptions or axioms. We like the axiomatic method because axioms are usually not controversial and everything proceeds from them using strict logical reasoning.
I think, as a physicist and a scientist my view of what math is, is very data driven and concept oriented. Others probably have different ideas. I'm not even sure I like the description I've offered here haha. But I tried to keep it as simple as possible.
I don't know of a single book that captures all that (there probably is one though).