To be very blunt, in 2020 most ML is still glorified statistics, except you lose the insights and explanations. The only tangible improvements can be random forests - some times. 99% of the stuff you can do with basic statistics. 99% of the coders I know just don't know statistics besides the mean (and even with that, they do senseless things like doing means of means)
So learn statistics - basic statistics, like in the "for dummies" book series.
If you want to be a little more practical, stats "for dummies" is often found in disciplines that depends on stats, but are not very good in math - biology, psychology, and economics are great candidates.
So just download biology basis stats (to know how to compare means - this gives you the A/B test superpower), then psychology factor analysis (to know PCA - this gives you the dimension reduction superpower) then econometrics basic regression (to know linear regression)
With these 3 superpowers, you will be able to do more than most of the "machine learning" people. When you have mastered that, try stuff like random forest, and see if you still think it's as cool as it's hyped to be.
Personally, I enjoyed both Andrew Ng's and Geoffrey Hinton's respective courses on ML and Neural Networks on Coursera. You may also want to check out Michael Neilsen's online essay on deep learning (http://neuralnetworksanddeeplearning.com). Ultimately I would also encourage you to supplement your understanding by applying this work to your own applications. The universe is often the best teacher.
https://fast.ai - good intro on practical neural networks.
I wrote a guide to ML based NLP. We identify if a sentence is a question, statement or command using neural networks:
https://github.com/lettergram/sentence-classification
The truth is you don’t need to understand all the math right away with neural networks. Mostly it’s getting an understanding of why you use a given layer, bias, etc and when. Once you get some intuition then I’d learn the math.
That’s at least how I instruct others. In any case, there are lots of guides for any flavor. I’d start with deep learning and focus on the “practical” then move to the “theoretical”.
* https://www.youtube.com/watch?v=UzxYlbK2c7E: Andrew Ng's machine Learning course, the recommended entry point by most people
* https://mlcourse.ai/ : More kaggle focused, but also more modern and has interesting projects
Do both courses simultaneously, take good notes, write useful flashcards, and above all do all the exercises and projects
Deep Learning
* https://www.fast.ai/ - Very hands-on, begin with " Practical Deep Learning for Coders" and then "Advanced Deep Learning for coders"
* https://www.coursera.org/specializations/deep-learning : More bottom-up approach, helps to understand the theory better
Do those two courses in parallel (you can try 2 weeks of coursera followed by one of fastai in the beginning, and then just alternate between them), take notes, write good flashcards and above all do the exercises and projects.
After that you will be done with the beginning, your next step will depend on what area interested you the most, and getting way too many resources right now can be extremely confusing, so I would recommend doing a follow-up post after you worked through the above resources. Also as non-ML stuff I recommend Scott Young's Ultralearning and Azeria's self improvement posts (https://azeria-labs.com/the-importance-of-deep-work-the-30-h...)
- MIT: Big Picture of Calculus
- Harvard: Stats 110
- MIT: Matrix Methods in Data Analysis, Signal Processing, and Machine Learning
If any of these seem too difficult - Khan Academy Precalculus (they also have Linear Algebra and Calculus material).
This gives you a math foundation. Some books more specific to ML:
- Foundations of Data Science - Blum et al.
- Elements of Statistical Learning - Hastie et al. The simpler version of this book - Introduction to Statistical Learning - also has a free companion course on Stanford's website.
- Machine Learning: A Probabilistic Perspective - Murphy
That's a lot of material to cover. And at some point you should start experimenting and building things yourself of course. If you'are already familiar with Python, the Data Science Handbook (Jake Vanderplas) is a good guide through the ecosystem of libraries that you would commonly use.
Things I don't recommend - Fast.ai, Goodfellow's Deep Learning Book, Bishop's Pattern Recognition and ML book, Andrew Ng's ML course, Coursera, Udacity, Udemy, Kaggle.
You can find it from O'Reilly here (http://shop.oreilly.com/product/0636920215912.do) or on Amazon here (https://www.amazon.com/Building-Machine-Learning-Powered-App...).
If you want to really understand the fundamentals of machine learning (deep learning is just one subset of ML!), there is no substitute for picking up one of the classic texts like: Elements of Statistical Learning (https://web.stanford.edu/~hastie/ElemStatLearn/), Machine Learning: A Probabalistic Approach (https://www.cs.ubc.ca/~murphyk/MLbook/) and going through it slowly.
I'd recommend a two pronged approach: dig into fast.ai while reading a chapter a week (or at w/e pace matches your schedule) of w/e ML textbook you end up choosing. Despite all of the hype of deep learning, you really can do some pretty sweet things (ex: classify images/text) with neural nets within a day or two of getting started. Machine learning is a broad field, and you'll find that you will never know as much as you think you should, and that's okay. The most important thing is to stick to a schedule and be consistent with your learning. Good luck on this journey :)
1. Michael Nielson's book: http://neuralnetworksanddeeplearning.com/
2. Stanford CS231n course: http://cs231n.stanford.edu/
3. DRL hands on book: https://www.amazon.com/Deep-Reinforcement-Learning-Hands-Q-n...
After this churn through research papers or medium articles on conv net architecture surveys, batchnorm, LSTM, RNN, transformers, bert. Write lots of code, try things out.
I'd compare ML with weather models: we understand physics driving individual particles, we understand the high level diff equations, but as complexity builds up, we have to resort to intuition to develop at least somewhat working weather models.
I hear that C++ is a nightmare to work with and was wondering if Rust,Julia, or even Swift would be worth learning instead.
I know Python but deep learning frameworks seem to be written in C++, so to come up with new layers I need to understand C++, which I was told has lot of peculiarities that takes time to pick up. Compiler isn’t also very user friendly (what I’ve read)
I’ve seen some really bad machine learning work in my short career. Don’t listen to the people saying “ignore the theory,” because the worst machine learning people say that and they know enough deep learning to build a model but can’t get good results. I’m also unimpressed with Fast AI for the reasons some other people mentioned, they just wrapped PyTorch. But also don’t read a theory book cover-to-cover before you write some code, that won’t help either. You won’t remember the bias-variance trade-off or Gini impurity or batch-norm or skip connections by the time you go to use them. Learn the software and the theory in tandem. I like to read about a new technique, get as much understanding as I think I can from reading, then try it out.
If I would do it all-over again I would:
1. Get a solid foundation in linear algebra. A lot of machine learning can be formulated in terms of a series of matrix operations, and sometimes it makes more sense to. I thought Coding the Matrix was pretty good, especially the first few chapters.
2. Read up on some basic optimization. Most of the time it makes the most sense to formulate the algorithm in terms of optimization. Usually, you want to minimize some loss function and thats simple, but regularization terms make things tricky. It’s also helpful to learn why you would regularize.
3. Learn a little bit of probability. The further you go the more helpful it will be when you want to run simulations or something like that. Jaynes has a good book but I wouldn’t say it’s elementary.
4. Learn statistical distributions: Gaussian, Poisson, Exponential, and beta are the big ones that I see a lot. You don’t have to memorize the formulas (I also look them up) but know when to use them.
While you’re learning this, play with linear regression and it’s variants: polynomial, lasso, logistic, etc. For tabular data, I always reach for the appropriate regression before I do anything more complicated. It’s straightforward, fast, you get to see what’s happening with the data (like what transformations you should perform or where you’re missing data), and it’s interpretable. It’s nice having some preliminary results to show and discuss while everyone else is struggling to get not-awful results from their neural networks.
Then you can really get into the meat with machine learning. I’d start with tree-based models first. They’re more straightforward and forgiving than neural networks. You can explore how the complexity of your models effects the predictions and start to get a feel for hyper-parameter optimization. Start with basic trees and then get into random forests in scikit-learn. Then explore gradient boosted trees with XGBoost. And you can get some really good results with trees. In my group, we rarely see neural networks outperform models built in XGBoost on tabular data.
Most blog posts suck. Most papers are useless. I recommend Geron’s Hands-On Machine Learning.
Then I’d explore the wide world of neural networks. Start with Keras, which really emphasizes the model building in a friendly way, and then get going with PyTorch as you get comfortable debugging Keras. Attack some object classification problems with-and-without pretrained backends, then get into detection and NLP. Play with weight regularization, batch norm and group norm, different learning rates, etc. If you really want to get deep into things, learn some CUDA programming too.
I really like Chollet’s Deep Learning with Python.
After that, do what you want to do. Time series, graphical models, reinforcement learning— the field’s exploded beyond simple image classification. Good luck!
I've started/stopped a few courses with Georgia Tech's OMSCS program as well which might have been useful, but I still feel like I'm missing some of the mathematical foundation to allow me to make more sense of those courses so Fast.ai's approach seems like it could be a better fit for someone like myself that's more interested in the practical aspects of using it (I just haven't made the effort to go through their content myself).
If you're learning it for career purposes, keep in mind that many corporate ML use-cases are problematic at best. At worst, you will produce something that kills someone inadvertently, possibly more than one person.
Learn about the many pitfalls and limitations of ML. Learn about inadvertent bias in datasets. Learn about the issues with inputs not represented (or not adequately represented) in your training dataset.
Most importantly, understand that ML is not magic and without significant guardrails in place, there's a good chance something will fuck up.
One thing I would add is replicate a couple of ML papers. It can help develop a lot of intuition about the specific area.
He published a lot of free ML blog posts, in easy-to-understand writing with nice examples, so it never made anything seem out-of-reach. I found that a lot of other material was a little too abstract, so his stuff was great.
The blog posts are here: https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec...
And I also bought his paid course with code samples -- it's affordable and good value.
Any tool needs an applied field but any applied field does not need all the tools. You have an applied field already (pharma), so start looking for one or two state-of-the-art ML papers for that? Happy 2020 and good luck, it’s going to be fun!
Has anyone done non-DL courses on their website? For e.g., any thoughts on Rachel's Computational Linear Algebra?
Also bear in mind that courses like fast.ai (as you see plastered on here), aggresively market themselves by answering questions all over the internet. Its a form of SEO.
EDIT (Adding this here to explain my point better):
My opinion is that the theory starts to make sense after you know how to use the models and have seen different models produce different results.
Very few people can read about bias variance trade off and in the course of using a model, understand how to take that concept and directly apply it to the problem they are solving. In retrospect, they can look back and understand the outcomes. Also, most theory is useless in the application of ML, and only useful in the active research of new machine learning methods and paradigms. Courses make the mistake of mixing in that useless information.
The same thing is true of the million different optimizers for neural networks. Why different ones work better in different cases is something you would learn when trying to squeeze out performance on a neural network. Who here is intelligent enough to read a bunch about SGD and optimization theory (Adam etc), understand the implications, and then use different optimizers in different situations? No one.
I'm much better off having a mediocre NN, googling, "How to improve my VGG image model accuracy", and then finding out that I should tweak learning rates. Then I google learning rate, read a bit, try it on my model. Rinse and repeat.
Also, I will throw in my consiracy theory that most ML researchers and such push the theory/deep stats requirement as a form of gatekeeping. Modern deep learning results are extremely thin when it comes to theoretical backing.