HACKER Q&A
📣 cromka

As someone with obscure CUDA knowledge, how to become AI developer?


I did quite a lot of CUDA, algorithms, and parallel computing in general during my university time. I wrote my Master’s thesis on GPGPU. This was before it was applied to crypto mining or AI. I was supposed to stay at the Uni and do PhD in GPGPU, but eventually accepted a much better paid entry-level job at a company and, subsequently, I haven’t touched low-level programming since graduating and moving on to coding in Java/.Net/SQL, etc.

Over a decade has passed and you can imagine the level of my 20/20 FOMO frustration for not having carried on with the more advanced engineering stuff.

So my questions is: what resources and what direction would you recommend for someone like myself to narrow that gap of 10 years, such that I could hopefully apply for an interesting job in the field sometime 12 months from now? I am absolutely aware it would take a lot of studying, especially on AI/ML maths and algorithms.

Perhaps someone has already been in a similar situation and can share their experience?

I will much appreciate any input here!

  👤 nikonyrh Accepted Answer ✓
Since with CUDA you are programming so close to hardware, and hardware (and CUDA itself) has advanced so much, I recommend you to go trough very carefully all the major CUDA versions and see how it has evolved. Well, strictly speaking I'm talking about different versions of "compute capabilities". Of course Wikipedia has a good summary: https://en.wikipedia.org/wiki/CUDA#Version_features_and_spec...

An other point is that you don't need to write any CUDA code to be able to utilize GPU computing. If you need ML models, you have frameworks like PyTorch and Tensorflow. You just need to express your mathematical problem, and the framework will take care of the rest.

Even if you needed to write custom GPU code, you don't need to do it in C anymore! For example you can JIT Python, using Numba or Triton.

Usually writing custom code is only required when:

- You are doing something novel, like PhD level stuff

- You must optimize the ML project for performance and trough-put at interference time

- You need to brute-force solutions (be it crypto-hashes, passwords, NP-complete problems, ...)

My last point to you is that do you want to learn to use these pre-existing frameworks and libraries, or learn to develop them or maybe even create new ones? What ever your answer is, I'd say that the first option is a great stepping-stone to advance to the second one.

👤 ttyprintk
I want to say that HackerNews is much more receptive of CUDA at the low-end (locally hosting LLM, for example) than the research end (CUDA-Q, the quantum features, for example).
👤 GianFabien
From my very limited understanding GPUs have become even more general purpose. If your past expertise was programming / architecture areas, then you might need to brush up on your linear algebra. However, it appears that the majority of AI/ML work is in data cleansing and using the various toolkits and frameworks.
👤 brudgers
Start developing AI. It is the simplest thing that might work. You will probably start off doing it poorly, but you don't need permission, a professor or an employer. Good luck.
👤 cromka
Somewhat ironically, I asked ChatGPT and got a pretty comprehensive response:

Transitioning back into a technical field like AI development after a decade of working in other areas can be challenging but definitely achievable. Here’s a structured plan to help you leverage your existing CUDA and parallel computing knowledge while updating your skills to meet the demands of AI development:

1. Refresh and Update Your Knowledge Since your background is in CUDA and parallel computing, start by refreshing these skills:

CUDA Programming: NVIDIA’s official CUDA documentation and tutorials are great starting points. Utilize resources like CUDA Programming Guide and CUDA Toolkit Documentation.

Parallel Computing: Revisit concepts and explore modern parallel computing frameworks like OpenCL and Thrust.

2. Learn AI and Machine Learning Fundamentals Online Courses: Platforms like Coursera, edX, and Udacity offer excellent AI and ML courses. Key courses include: Machine Learning by Andrew Ng on Coursera. Deep Learning Specialization by Andrew Ng on Coursera. CS231n: Convolutional Neural Networks for Visual Recognition by Stanford (available online for free).

Books: Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Pattern Recognition and Machine Learning by Christopher Bishop. Mathematics for AI: Brush up on linear algebra, calculus, probability, and statistics. Khan Academy and MIT OpenCourseWare can be good resources for this.

3. Gain Practical Experience with AI Frameworks TensorFlow and PyTorch: These are the two most popular frameworks for AI development. Follow official tutorials and documentation: TensorFlow, PyTorch. Engage in projects and Kaggle competitions to apply your knowledge.

4. Dive into GPU-Accelerated Machine Learning NVIDIA’s Deep Learning SDK: Explore NVIDIA's deep learning libraries such as cuDNN, TensorRT, and RAPIDS. Implementations: Try converting existing CPU-based ML models to run on GPUs using your CUDA skills.

5. Build Projects and a Portfolio Personal Projects: Start small by replicating classic ML projects and gradually move to more complex ones. Document your projects on GitHub. Kaggle Competitions: Participate in Kaggle competitions to solve real-world problems, improve your skills, and build a portfolio.

6. Stay Updated and Network Follow AI Research: Regularly read research papers from conferences like NeurIPS, CVPR, and ICML. Websites like arXiv and Papers with Code are useful. Join Communities: Engage with AI communities on platforms like Reddit, Stack Overflow, and specialized forums. Attend local meetups, webinars, and conferences.

7. Consider Advanced Education Professional Certificates: Consider earning AI/ML certificates from universities or companies (e.g., Google, Microsoft). Graduate Programs: If time and resources permit, a postgraduate program in AI/ML could provide structured learning and credibility.

8. Apply for Internships or Junior Roles Internships: Many companies offer internships or junior roles specifically tailored for career switchers. Apply for these to gain hands-on experience. Open Source Contributions: Contributing to open-source AI projects can also provide practical experience and demonstrate your skills to potential employers.

9. Prepare for Job Applications Resume and LinkedIn: Update your resume and LinkedIn profile to reflect your new skills and projects. Mock Interviews: Practice AI/ML interview questions, focusing on coding problems, ML theory, and system design. Summary

Re-entering the AI development field requires a strategic approach: refreshing your existing knowledge, learning new AI/ML concepts, gaining practical experience, and continuously networking. With dedication and structured learning, you can narrow the gap and position yourself for exciting opportunities in AI development within a year.