HACKER Q&A
📣 evnix

Can build CRUD apps, what now?


I have been building good amount of CRUD apps for the past 10 years or so, but I feel stuck, like a massive wall in front of me.

I started coding in 2010 when I was in High school using JScript and Microsoft PageBuilder with the Help Manual(chm file) that it came with (I had no internet connection). Later I picked up PHP, JS, Java and a whole lot of other technologies which allowed me to be an amazing CRUD app developer, My resume is filled with the latest trendy words like Docker, Scala, Kafka, Spark, Hive, react, K8s or any new framework that seems to make a buzz. All this was amazing but I recently realized that I am not so smart and I can't seem to build anything other than CRUD apps and it is quite depressing.

There is a world out there about compilers, run-times, drivers, emulators, VMs and OSes that I can't seem to grasp, it is just too complicated for me. My Computer Science degree and the newer degrees do not focus on these, they have become 4 year long coding boot camps that focus on getting people job ready with AWS, React, Ruby or whatever trendy. This is true for the majority of the not so prestigious universities out there.

I see a pattern and I feel the older generation is way more capable and knowledgeable when it comes to Computer Science in general. I am pretty sure a lot of people are or were in the same boat as me. I don't really have the time(Have wife & kids) to take a compiler course or set out to build my own OS, what would be the easiest way to learn these lower level things?

Things I have tried so far,

1. Reading the Dragon book(I find this would be useful only once you have built a bad compiler) 2. Paying people older than 40-45 to teach me some C/C++ and some tricks of the trade (expensive but I gained a huge amount of knowledge in a short time)

any other tricks, or comments about your own journey would be helpful.


  👤 zikzak Accepted Answer ✓
I don't want to be dismissive but I think we all go through this. My dream of building robots was kind of crushed when I realized as a mature student coming into a cs program after eight years of working blue collar jobs, that I might not have the chops to get into the electrical engineering program. A few years later I'm graduating in the middle of the dotcom crash. Hey, building contact us forms and customizing or building CMS software landed me a job so let's do that.

Ten years after that I was thinking "great, I've spent 10 years validating email addresses on contact us forms" (the reality was quite different, of course, but I was feeling like I'd lost the thread). But then I started doing a little more with ecommerce and got good at mentoring people and so on.

Now, it's ten years after that. I am managing a bunch of programmers and still write a little code. And I'm wondering how I will stay relevant for the next twenty or so years but so far, so good.

And I keep learning more and more. Not about compilers and Docker (this does come up, I guess) but about projects and people, business, domain knowledge about the area we build software for, and so on.

On paper, yes, I'm nothing. Just some average dude. But I really like my job, my team, and my skills.

I'm not saying this will happen to you but what I am saying is you shouldn't feel inadequate because you can't build a C compiler in assembly or something. That's the kind of thing only someone truly, truly interested would do. It's not something you do because you should or because it will make you a "real programmer".

Find an industry or business that fascinates you and figure out what they need to see to hire you and go for that.


👤 sweetsocks21
I took plenty of OS and compiler classes in college and I was still struggling to grasp some concepts. I think the issue was I was able to understand each individual class in isolation, but when I tried linking everything together, something just didn't click.

I came upon this book/course: https://mitpress.mit.edu/books/elements-computing-systems-se...

From a Hacker News post IIRC, and it helped more than I can express. At least for me it filled in all the little gaps in the computing stack and really made me "get it". I could relate to the old timers complaining about how everything is so bloated now.

The course is very hands on and you build a "full stack" from logic gates to an operating system that can run programs. It's truly amazing, and I think I learned more from this book than most of my college courses.

I believe it's also offered for free on coursera and has a website: https://www.nand2tetris.org/

But if you really want to know how a full computer works from the ground up, I think this book would be well worth the time.


👤 mathgladiator
There are three questions: how do you want to make money? how do you want to be entertained? how do you want to sell yourself?

It's very easy to get stuck in the world of CRUD because it pays the bills, but as you noted there are a lot of technologies.

Shifting out of that world requires risk tolerance because there will be failure after failure. That wall you speak of is the nature of hard things and the time you can dedicate bashing your head against that wall.

My advice for climbing/breaking the wall:

Pick a small project that you find interesting: build an OS, game, programming language, VM, etc... and just do it. The key is to cross the hump of understanding and get something simple working, and the good news is that conceptual understanding isn't that far away. The tool to help you get unblock is writing. Write the comments of what you are trying to do, and break it down until it becomes code-like. Read books on the subject and bounce back to writing what you read.

However, going from conceptual understanding to professional environment is different because the concepts start to inter-relate and the complexity manifests.

Prior writings on this:

http://jeffrey.io/writings/thoughts-for-young-hackers-and-so...

http://jeffrey.io/writings/build-first-buy-second.html


👤 yeswecatan
Do you want to do something non-CRUD because you do not enjoy building CRUD apps, or do you feel like you _have_ to do something else because CRUD is too "easy?"

Most of the world, however, does not work this way. As an example, take Restbucks (https://www.infoq.com/articles/webber-rest-workflow/). You can see how there's a web component to this, though it's much more complicated than your normal CRUD view. I've discovered new challenges in dealing with these problems ranging from building a rich domain to handling side effects.


👤 throwawayboise
> My resume is filled with the latest trendy words like Docker, Scala, Kafka, Spark, Hive, react, K8s

Guess what? I'm in the older generation, I have a CS degree from the late 1980s. I'm comfortable with hardware, compilers, run-times, drivers, operating systems, etc. but all that stuff you listed is like a mystery. I don't know what it is, why or when I would use it, etc other than at a very high level.

I mostly write CRUD apps.


👤 sdevonoes
> There is a world out there about compilers, run-times, drivers, emulators, VMs and OSes that I can't seem to grasp, it is just too complicated for me

Regarding VMs and OSes:

Since it seems that you have been working as a web developer, I think a soft transition for you would be to get into modern devops practices:

- start with vagrant to spin up a linux virtual machine and start playing with it. E.g., how to setup ssh keys, how to install packages in an automated an idempotent way (e.g., using Ansible), etc.

- while you are dealing with virtual machines, you'll learn also a bit about networking (e.g., what's a private network? what's NAT?)

- when you know enough you could start using Terraform to spin up virtual machines in the cloud. There's some magic the moment you run one command and you have a linux machine up and running in some cloud provider! Even better: you run another command and the machine is totally destroyed :)

In such a journey you'll be exposed to different and potential new concepts for you: vpcs, virtualization, a bit of security, firewall rules, ssh, tcp/ip, etc.


👤 yawn
This might help: https://www.nand2tetris.org. If you go through this, you will build a computer from the ground up starting with boolean logic. It touches on hardware design/computer architecture, building a compiler for an oo language, and building a basic operating system.

👤 Zvez
Not sure if useful for anybody, but here is my story. I have no real cs-background, I just liked computers in high school and participated in some local programming contests. I started working on my 4th year of uni. It was java enterprise. So yeah, a lot of crud. But even then we managed to build abstract stuff to make crud building easier and fun.

After around 6 years of crud I decided to try something different, and went to yandex (it is like google, but in Russia). First time it was just the same crud (lot of rest api), but after some time I managed to land on some infra projects - kafka, streams, microservices. And it became much much more fun. You really build stuff from the scratch, you design it, test it under high load. Anyway my tips:

1. Don't go for buzz word, find job where you can actually choose technologies. Soon you will understand how to see through the hype. And cost of choosing wrong tech.

2. Stop trying to learn what 'everyone think is cool'. Learning compilers won't make you better developer, esp if you are not into it. And Dragon book is not the best intro to compilers anyway. If you want to learn something fun, find what it is fun for you in the first place.

3. If you want to become better developer, try to improve things you do at your day job. Not something like 'we should use k8s or we die'. More like 'we will have to make a lot of these crud in near future, lets spend some time to build framework that makes building and support them 2x faster and cheaper'. Look at small and achievable improvements first.

4. Invest time to understand how things work. And build your own opinion about technological decisions.


👤 king_magic
I went to a pretty unremarkable state school and took compiler construction classes, OS classes, AI classes, deep networking classes, etc. Pretty standard course load for many CS BS programs - then (16 years ago) and now.

I really don’t think most modern CS programs are glorified bootcamps. Not sure where you’re getting that from.


👤 tarkin2
* NAND2Tetris - https://www.nand2tetris.org/

Teaches you how computers, languages and operating systems work from basic principles

* Little Schemer, Seasoned Schemer

Teaches you about the power of functions, recursion, various CS topics and lisp's syntax

* K&R C Programming Language

Teaches you about lower-level programming


👤 ivanech
I strongly recommend classes at Bradfield School of Computer Science which teach about as fast as possible (https://bradfieldcs.com/) and their self-study guide (https://teachyourselfcs.com/).

I also got a lot out of Bob Nystrom's "Crafting Interpreters" book


👤 apapli
I taught myself to code, started in 2010 after 10 years in IT infra/networking. I’m not a professional developer and it’s a hobby for me - I did a humanities degree with no comp/sci background.

Initially I built a simple crud app to track water changes for my aquarium. For a while there I had it online as a free SaaS app too. It was using Ruby on Rails, and I had no idea how to use JavaScript to make the UX better.

Fast forward a few years and I was getting fed up with voicemail, so I read up on APIs and figured out how to use GCPs transcription service, pointing to audio files in twilio (I diverted my mobile to twilio to act as my voicemail). Actually pretty simple even though it might sound complicated.

Fast forward to last year and I built a free directory to raise the profile of primary producers who were missing out on foot traffic as a result of COVID, closely followed by another project I’m close to launching. These last two projects were in nodejs as I figured I had better finally figure out JavaScript.

When I was completely stuck on something I did a lot of googleing and watching YouTube - that happened a lot, particularly when trying to get my head around promises.

Moral of the story, don’t be put off with CRUD apps, they are very rewarding. Just build stuff you enjoy when you have spare time (for me evenings after child in bed).


👤 fendy3002
Web dev nowadays don't need to learn how to build compilers, vms, etc. For me they are separate branch / subject, same with gaming which is also a separated one.

Depending on your field / industry, having good knowledge of accounting is magnitude times better than compiler knowledge.

Learning about optimization (both db and app), write good / clean code, debugging are useful skills wherever you go.

Unless you want to change course, it's perfectly fine.


👤 rgoulter
One model of motivation is that you get motivated with autonomy, mastery, and purpose/belonging.

I understand the 'mastery' bit is less about whether you can do it (clearly you rate yourself well at that) so much as the feeling that you're improving at it.

"just CRUD apps" suggests that you're not feeling you're mastering it, and that you don't feel like it's the best fit for what you're interested in doing.

I think any professional task you can competently accomplish without having to learn anything is going to be boring. I think e.g. Scala, K8s are complicated enough that there's room for learning more about it. Not necessarily "do hard stuff for the sake of being interesting", so much as "being able to evaluate whether this is the best tool for the job".


👤 SturgeonsLaw
> I see a pattern and I feel the older generation is way more capable and knowledgeable when it comes to Computer Science in general.

One thing to consider is that the older generation has simply been at it for longer than you have, and this is an industry where the vast majority of experience is gained on the job.

Another consideration is that due to a number of timing factors, a certain demographic (the elder Millenial/younger Gen X cohort) came of age at a time when computers were just recently cheap enough to become a mass-market good and viable for home use, but before they were commoditised and standardised, and before there was an internet full of helpful information, so if you wanted to learn how they work under the hood, you needed to get your hands dirty.


👤 etaioinshrdlu
Well, one idea is you could help us get a compiler up and running for our home-brew CPU project: https://github.com/hsc-latte/hs32core-rtl

We are at the stage of lacking a lot of software help. We have an assembler, but that's it.

We're expecting some initial chips back in a few months from using the Google/Skywater/Efabless shuttle program: https://efabless.com/open_shuttle_program

Currently we are simulating the CPU in verilog simulators, as well as fpga's. There's a lot of fun very low-level work to be done.


👤 TheDudeMan
What are you trying to do? Just stimulate and challenge yourself? Or are you trying to get a better job? It does not sound like you have a passion for compilers and OSes. Neither do I, so I don't try to deeply understand them. There are plenty of other areas that do interest me. Don't force yourself into something that is not particularly interesting to you.

👤 bogwog
Game development has really intense math and engineering stuff, and it's extremely fun and addicting for some people. If you can get yourself into that world, it'll set you on a track to learning what you're asking for here.

Of course, it's not for everyone, but I'd say it's worth giving it a shot. Definitely start with C/C++ if you haven't already, and try not to stay with existing game engines like Unreal or Unity for too long (unless you're specifically trying to start a career in game development; a bad idea)


👤 nednar
You don't have to be smart, you just have to surround yourself with people smarter than yourself. The rest will happen automatically.

Have you tried joining one of these projects you mentioned as open source contributor? I know at least some of them are always happy to get more people to join and help. And not just with code contributions. They also have (CRUD) websites that need maintenance, docs to be written, events to be organized, and training/marketing material to be produced. You might be surprised, how these seemingly smarter people end up being super happy about your contribution, as usually what is easy for you will be hard for them, just as you might see it the other way around.

And because you said you have wife+kids, your time might be limited: there are also ways to get paid to participate, at least around the bigger projects.


👤 max0563
I often feel the exact same way. I have had to really be honest with myself to understand these feelings. What I have realized is that this feeling can mostly be reduced to fear. Fear of becoming obsolete, fear of being not so smart, fear of not being a "real programmer". It also doesn't help that a lot of people in other fields really do look down on web developers in this way. At least for me, this induces a fear response and I get anxious and it is almost entirely out of insecurity. I start to "stress learn" things that I feel would make me a "real programmer" even if I have no real use for or interest in them. It's pointless, of course, but I have found it to be a viscous cycle that's hard to fight sometimes.

A few things have helped me. The first is that you need to embrace what you do and love what you do. It doesn't sound like you hate being a web developer, more that you feel you should be doing "harder" things. Web development is far more than CRUD apps, so it might be worth looking for another job that will challenge you more as a web developer.

The second thing is that you need to understand what you do _deeply_ instead of _broadly_. I have for a long time considered myself a generalist, but this has its drawbacks. When you really understand a tool set deeply and become an expert a lot of the lower level internals really reveal themselves. You end up learning things that you've always thought you wanted to learn in isolation, but instead there is now a useful context in which to learn them. Not to mention that this helps with the feeling of being "not so smart" because you will form informed opinions as a result of your deep level of understanding.

The last thing is that you do need to feed your curiosity and the only way to do that is through projects. You want to build a compiler, build an emulator, a game engine, a database, or an OS? You really just need to do it. I have dabbled in each one of these things. Some of which I got very deep into, others not so deep. Each time I learned a little bit more. The hardest part is picking a project and running with it, especially when there are so many different options, but it is the only way.

Sometimes it's important to take a step back and appreciate what you do. It's very easy to beat yourself up and compare yourself to others. If you really are unhappy with what you do, then you need to start deeply diving into something that you do find interesting. Nothing in this field is easy and it all takes time to gain any level of proficiency at.


👤 Pandabob
I know it's not for everyone, but being able to build CRUD apps is pretty useful if one wants be an indie hacker. So if you think you'd enjoy running an (indie) online business, I'd encourage you to take a look at Indie Hackers [0], Microconf [1] or Y Combinator's Startup School [2].

[0]: https://www.indiehackers.com/

[1]: https://microconf.com/

[2]: https://www.startupschool.org/


👤 linguae
I wouldn't be so hard on yourself. It's unfortunate that there are a lot of CS degree programs that are basically vocational prep teaching today's tech du jour at the expense of teaching the fundamentals, but this is not a reflection on you, and it's never too late to learn and master the fundamentals. You might not have the time for college classes, but the nice thing about pursuing CS as a hobby is that you set the timeline.

(Note: There is nothing wrong with vocational schools for those who need to learn specific technologies. In fact, many people with CS degrees could benefit from courses in specific technologies. However, the purpose of a bachelor's degree in computer science is to teach students the fundamentals of computer science, and I believe undergraduates students are done a disservice when they are not taught the fundamentals.)

I highly recommend going through the book Structure and Interpretation of Computer Programs (https://mitpress.mit.edu/sites/default/files/sicp/full-text/...). This may be your first exposure to functional programming in Scheme. Even though it was originally designed as the textbook to MIT's introductory CS course (before switching to Python in the late 2000's), this book is useful for anyone interested in computer science, and it's easier to read for those who already know how to program. By the time you get through the end of this book, you will have written a Scheme interpreter, a logic programming interpreter that can handle unification (don't worry about what that means right now), and a Scheme compiler.

Once you finish that book, then it should be easier for your to delve deeper into the topics of your choosing. Want to learn more about computer architecture? I highly recommend getting the latest edition of Patterson and Hennessy's Computer Organization and Design (https://www.elsevier.com/books/computer-organization-and-des...). Want to learn about operating systems? I highly recommend Remzi and Andrea Arpaci-Dusseau's free Three Easy Pieces (https://pages.cs.wisc.edu/~remzi/OSTEP/). For compilers, you already have the dragon book, which should be easier to read once completing SICP.


👤 temporama1
- Take CS50

- Take Nand2Tetris

- Take Casey Muratori's upcoming course StarCode Galaxy.

- Get some inspiration from Jonathan Blow and Casey Muratori on YouTube, and start the rabbit hole there.

- Be prepared to question a lot of the dogma that you have already absorbed.


👤 godot
Firstly, this is the first I've heard about newer CS degrees not having classes on compilers, etc. and are now just 4 year boot camps that focus on AWS, React, etc. If true, thanks for this insight! Helpful to know about the current landscape of CS education. (at least for the not so prestigious universities as you mentioned)

Secondly, I'm not sure if you've spent time in different industries? While you may still be building what amounts to CRUD apps in different industries, your work can be very different and you can have very different learnings. For example, I worked in social games (i.e. FB) and mobile games a decade+ ago, for several years. Engineering was all about high traffic, high performance backends; squeezing every bit of speed possible, handling large amount of concurrents and DAU while keeping systems up. Over the next couple of jobs, eventually I landed in real estate transaction tech, which is almost a polar opposite -- very small amount of traffic relatively, but needing to be very robust and well-designed at every level from API to data store to schemas to ETL. Design patterns and concerns etc. are completely different; to me it's a whole different world of software engineering. I imagine if you move the opposite direction, you would see just as much drastic differences.


👤 Const-me
> any other tricks

I would look for jobs in software companies who make stuff you want to learn to make, but for some parts of their products they also need these Docker/Scala/Kafka/etc. from your resume. If you are good at these things, they gonna pay you well i.e. enough to support a family. I think over time you’ll inevitably learn their lower-level stuff you have to integrate with. You can then change position within the company should you want to.


👤 jujodi
This will probably be an unpopular opinion but most software is just connecting a couple layers of abstraction to form a new, novel layer of abstraction that you get to design. You do this all the time with CRUD, what makes you think writing drivers in C is all that much different? If you know to any extent how Node's single threaded event loop works to process HTTP requests, then you know how drivers work too, there are just different best practices, APIs, etc. These things you mentioned are mostly practical and can be learned by doing.

That's different though than "Computer Science in general," which is far more theoretical, mathematical, and logical. These things, while they can be learned by doing, are much easier to learn by being taught. Paying someone to teach you C/C++ isn't going to make you better at Computer Science in General, it will just teach you a new programming language, which is practical. Computer science in general are things like distributed systems, discrete math, algorithms, etc.


👤 m3nu
I do think it was easier to develop a full understanding of how IT systems work a few decades ago. We played around with Windows 3.11, XP and early Linux distros, setting up networking, etc in the old days™.

Nowadays the level of abstraction is higher (smartphones, FaaS, VMs, UI frameworks).

My tip to OP would be to set up a home lab on hardware from eBay. Run some services on it and manage it from the metal up.


👤 Matthias247
What is what bothers you most about the current situation? Is is that you feel you haven't learned enough for yourself and want to getter a better understanding how other domains work? Or is there are fear that you are missing out on some interesting career opportunities?

If it's the latter than I can tell you with high confidence that you are not missing much there. Most companies need CRUD apps, simple Frontend apps, etc., and there is a high demand for engineers with the relevant skills and therefore also salaries are fine. Building OSes, compilers, embedded devices, networking stacks is super interesting, but there isn't that high of a demand since it's rarely what drives business. Therefore even if the topics are sometimes more complicated, they aren't payed that different.


👤 softwhale
> There is a world out there about compilers, run-times, drivers, emulators, VMs and OSes that I can't seem to grasp, it is just too complicated for me.

Do you want to become a specialist or a generalist? You can dive into one or two of these topics and specialize the rest of your life into it. By the way: a specialist in any of these topics would probably not know any of the fancy buzzwords you've listed in your resume. As a matter of fact, I think you'd have more career opportunities instead.

Keep in mind that the older generation had a "few layers" to explore and could easily understand each slice until the bare metal. These days, it is much more broad and in-depth.

So you want to specialize in something, or become a generalist?


👤 Zelphyr
Regardless of where you focus your attention on next, you might want to consider filling in your understanding of really low-level foundations with books like “Code” be Charles Petzold and “The Elements of Computing Systems” by Noam Nisan and Shimon Schocken.

👤 zem
if you want to break out of the crud trenches, one nice way to do it might be to learn some functional programming. to that end, give "realm of racket"[1] a try - it takes a pedagogical approach, and you might find the earlier chapters a bit trivial, but also you should never feel out of your depth, and you should be able to follow along and have fun while learning some new ideas. after that you should be in better shape to tackle some more serious areas of programming and computer science, but i feel the easy intro to racket will stand you in good stead.

[1] https://nostarch.com/realmofracket.htm


👤 austincheney
Stop thinking in terms of technology, language, and tools. Learn to think in terms of product. Imagine the end state of something original and awesome, then build it. It’s not about what you know or don’t know, it’s about action and goals.

👤 halfmatthalfcat
Sounds like you have breath but not depth. Go deep into the technologies you've mentioned.

To be honest, it sounds like youre disillusioned more with application development and not the particularities of the technologies you work with.


👤 joshka
>I see a pattern and I feel the older generation is way more capable and knowledgeable when it comes to Computer Science in general. I am pretty sure a lot of people are or were in the same boat as me.

The older generation are in this place because back then none of the technologies you listed existed and so they had to know the more low level things.

A book that might help some of your knowledge gaps (at least directionally) is The Imposter's Handbook[1] by Rob Conery.

If you're specifically aiming to get away from CRUD take a look at the various Domain Driven Design books by Eric Evans[2] and others. This is the opposite direction to your ask though (more high level than low level).

If you want to go deeper in Java, read Effective Java[3] and Java Concurrency in Practice (JCIP)[4]

Lastly, if you want to experience the reasons why some of that early comp-sci stuff happened the way it did, playing with expensive hardware is the least effective way to find understanding of constraints. Instead consider grabbing devices with significant limitations like an Arduino, ESP32, or similar (I'm partial to the Wio Terminal[5] and M5Stack[6] as interesting easy to get started devices, but others would suggest various Arduinos or Raspberry Pi devices). Realistically you're going to learn much more by coming up with a self-challenging project and completing it than just by learning it for the sake of learning.

[1]: https://bigmachine.io/products/the-imposters-handbook/

[2]: https://www.amazon.com/Domain-Driven-Design-Tackling-Complex...

[3]: https://www.amazon.com/Effective-Java-Joshua-Bloch/dp/013468...

[4]: https://www.amazon.com/Java-Concurrency-Practice-CONCURRENCY...

[5]: https://www.seeedstudio.com/Wio-Terminal-p-4509.html

[6]: https://m5stack.com/


👤 matt_s
In Rails-land there is a term DHH used (not sure if he coined it) called "conceptual compression". All of these OS, compiler, run-times, emulators, etc. have been conceptually compressed down so far that a typical application writer doesn't have to dive deep into those to solve problems. In fact, in Rails you might not get past the ORM much into actual SQL. You can spend an entire career solving good business problems writing CRUD and related apps.

I would consider myself in older generation, having graduated around when Google was operating in a garage. I took assembly programming and played with circuit boards and logic gates in college. I don't have specialized knowledge of OS writing or compilers but I have enough from a CompSci degree (or maybe it was building gaming rigs in the days of dealing with IRQs) to know if I see issues in those areas. A lot of the concepts in computing haven't changed in ages, yet the buzzwords have.

I would only suggest diving into those areas deeply if you are looking for a job/career change. I say career because working on writing HW drivers or other things is almost a different career than writing business apps since the job opportunities will be different. I've run into folks that got "stuck" into deeply specific areas of computing and have no buzzwords on their resume. They got laid off and are having a hard time finding work.


👤 User23
For someone such as yourself who clearly has some practical ability to write and understand code, Michael Sipser's Introduction to the Theory of Computation is an absolutely marvelous text to fill in gaps in theoretical Computing Science that may be missing from your education.

👤 baryphonic
> There is a world out there about compilers, run-times, drivers, emulators, VMs and OSes that I can't seem to grasp, it is just too complicated for me.

If you've already determined that you can't do something because it's just too complicated, there's a strong chance you'll be proven right. In my experience, all of the things we software people deal with down to networking, hardware drivers, task scheduling, you name it, were created and explained by humans. Nature constrains us with time and space performance and quantum mechanics. If another human was smart enough to create this stuff ex nihilo, certainly with some work you can be smart enough to understand it.

> My Computer Science degree and the newer degrees do not focus on these, they have become 4 year long coding boot camps that focus on getting people job ready with AWS, React, Ruby or whatever trendy. This is true for the majority of the not so prestigious universities out there.

No one is going to give you a checklist for how to be creative or pursue your own goals. You have to do it. If you're capable of asking this on HN, I think you're capable of setting a few goals. Just write them down and put the deadline a long ways away.

> I see a pattern and I feel the older generation is way more capable and knowledgeable when it comes to Computer Science in general. I am pretty sure a lot of people are or were in the same boat as me. I don't really have the time(Have wife & kids) to take a compiler course or set out to build my own OS, what would be the easiest way to learn these lower level things?

I'm in my thirties. When I was in school, there were a lot of kids who were just punching the requirements and doing what they were told to get by. Any time they got stuck, they'd have to ask the professor or TA in office hours. Others of us learned that we could solve a lot of problems ourselves if we cut them down to their essences and wrote small programs to see what was actually going on. That heuristic was very helpful.

Once I really did get stuck after doing this, and went to office hours. My professor (an old school Unix graybeard) was so thankful that I had put in the leg work of examining the problem with "toy programs" as he called them that he gave me an exceptionally detailed explanation for what was going on, and how I could show this to myself. He made the comment that a lack of curiosity was a problem among my peers, but that it can be grown and cultivated like other skills.

One other thing the older generations did better than we do is look at the history of our field. If you don't know who Alan Kay, Doug Engelbart, Nikolaus Wirth, Alan Perlis, Ivan Sutherland, Leslie Lamport, Margaret Hamilton or Ted Nelson are, go find some information about them (most have talks or demos on YouTube). Then find out the rest of the missing pantheon of computing. Our industry goes through fads in sort of a fashion culture, and in my experience it leads people to feel burned out our stuck exactly as you described. There are some enduring truths which our field does a poor job of passing onto posterity. Turns out more people had interesting ideas beyond Dijkstra and Turing.

> 1. Reading the Dragon book(I find this would be useful only once you have built a bad compiler) 2. Paying people older than 40-45 to teach me some C/C++ and some tricks of the trade (expensive but I gained a huge amount of knowledge in a short time)

My advice: pick a small, interesting problem and tackle it for some set amount of time each day, maybe after the kids are in bed or your wife is enjoying a bubble bath or whatever. It doesn't have to be long - maybe a half hour to an hour. Just do it consistently.

I've found that GitHub, YouTube and HN itself have loads of interesting project ideas and tutorials. I find those much better than just reading a book. In those cases, the book becomes supplementary material, and is usually most useful at exploring theoretical issues, edge cases, history and other more detailed topics.

If you are going to read, always do the exercises. Also, keep in mind that a lot of the books recommended like the Dragon Book or SICP or whatever else are typically a very thorough treatment, but may not be great for a great read. (There are exceptions to this: I find Tanenbaum's OS books to be highly engaging, interesting and helpful. He is both a practitioner and a theoretician, so I find his perspective valuable.)

Not only is my understanding better with experience (rather than just reading), but I find that I feel accomplished, which is a virtuous cycle. And I also find that I learn enough to be able to discover other interesting things for myself.


👤 cultofmetatron
at the crossroads of everything (machine learning, graphics, constraint optimization, cryptography, simulations) I've wanted to do with coding beyond basic crudapps is linear algebra. I'd recommend learning it well enough to implement the operations in code.

👤 villgax
Try recreating popular applications & see what it takes to make them live!

👤 throw1234651234
As an expert in CRUD - is security a financially viable field for someone who knows nothing about it past SQL and JavaScript injections?

👤 jfdi
In general - something you find interesting. Ideally on a frontier. E.g., Smart contracts, Web3, dapps.

👤 pabs3
I learned a lot by running, tinkering with and contributing to open source projects.

👤 xixixao
Learn Rust. Pick a project, something native, and build it in Rust.

👤 devoutsalsa
I wrote a shitty database. That was a great way to learn!

👤 f00zz
Write a Tetris clone.

👤 webmaven
> My resume is filled with the latest trendy words like Docker, Scala, Kafka, Spark, Hive, react, K8s or any new framework that seems to make a buzz.

This grab-bag of technologies is kind of all over the place. I see two possible paths forward (these aren't exclusive):

First path: everything you've listed is open source. Pick something, and start digging into the code to figure out what makes it tick. Try to fix a bug from the issue tracker, or review a PR. At some point you'll dig down far enough to encounter a tool or technology (also open source) you're unfamiliar with, and you can learn that and keep digging.

Lather, rinse, repeat.

I've found that most recommendations are to study things bottom up (eg. NAND to Tetris), but for many people it is very difficult to see the point of the lower levels, they are just too many layers of abstraction removed from anything they might find interesting. Top down learning tends to be self-directed, less formal, and ad-hoc, but the motivation for learning is much clearer (ie. you're digging down from the interesting thing made possible by the layer you're digging into), which tends to help a lot. Sure, you're always finding out that the abstractions you know are arbitrary, illusory, or even false in important ways, and in any case always hiding nasty messy truths underneath, but TBH that's how most real learning in our field happens anyway.

Second path, those technologies all have use-cases where they are (supposedly) useful. Start with a CRUD app, and try to figure out when one of those technologies makes sense to incorporate, and explore the edges of those use-cases, their failure modes, when it stops making sense to use them. Most CRUD apps in the wild are, well, pretty cruddy, which means that there is a ready market for skills to get them working again after they've broken down for some reason: too many concurrent users, too high volume of transactions, too large volume of data, too many monitoring alerts, backups take too long, transactions fail, specific queries always time out, inconsistent state, search autocomplete has started freezing the page, newly entered data takes 10m to become visible to users or to another app, hosting costs have started increasing non-linearly, a password reminder takes an hour to arrive in the user's inbox, and on and on.

When it comes right down to it, Twitter is basically a CRUD app. So are ebay, Wikipedia, Amazon, AirBnB, Yelp, Facebook, Reddit, LinkedIn, and so on. And yet not, as a Rails clone of these sites can quickly reveal once you start getting a few thousand users (real, or simulated). Sometimes solving a bottleneck can be done by swapping out a component or scaling horizontally or vertically, other scenarios require more extensive changes. Learning how to recognize and mitigate these inevitable growing pains can lead to some interesting places, and more interesting solutions and work.

Sure, CRUD apps as initially deployed, are boring. But they don't have to (and arguably can't) stay that way, especially as they evolve to encompass more users and data, not to mention functionality (eg. geodata, full-text search, stream processing, notifications, workflow, auditing, PII, security, etc.).

I hope this helps. Good luck on your journey!


👤 hexmiles
I admit I was never in this exact situation, so I'm not sure if what i have to offer can help, but I can relate to the feeling of "something being outside of my capability".

What I don't understand from your post is what your end goal is? Why do you want to learn low-level stuff like building compilers or operating systems? Is it for work? Desire to improve yourself? What can help you better depends on the answer to this question.

I can offer a couple of pointers I hope they can help:

1. I was frustrated by my lack of understating of low-level concepts and how it prevented me from understanding the more complex pattern. The thing that clicked from me was building a small CPU and related hardware (timer, VGA frame-buffer, etc...), I don't remember which site/guide I follow, but today ben eater videos or nand2tetris seems good choices.

I also read a lot about game console programming (GameBoy color in particular), a lot of that low-level concepts were really a lot simpler to grasp with the visual demo to demonstrate. I'm also constantly surprised how much concepts like interrupt, how a mmu works, etc... are useful even when writing high-level code like I'm doing now.

2. Stop considering Databases, compilers, and operating systems as a single unit to learn. They are made of a lot of smaller and simpler components that together form a system. And while when are together seem very complex, often when divided into smaller part became a lot easier to learn even if the total "complexity" remains the same. Take compiler, for example, there is code generation, optimization, and syntax parsing. Instead of trying to build or understand a compiler start by learning the basics of parsing, for example by building a simple chatbot (one of the first things I wrote was an IRC bot), evolve it to understand more complex queries (for example SQL).

Related to this: One thing I could not understand was how big relational databases, like MySQL, worked, a lot of the literature on the topic seem to have a perspective that didn't help me (like the math behind it). The ah-ha moment to me was with sqlite and seeing that at the base was a simple key-value store, and everything else was built on top of this. One takeaway, for me, was that when you feel that you don't understand something, maybe you are asking the wrong question! in my case I keep searching "how relational databases work" instead of what I was really interested in "how relational databases are implemented".

3. I would not start with a thing like "the dragon book", from what I see is an extremely complete book about "everything you need to know about compiler" (hasn't read it, so correct me if I'm wrong). Book like that can seem intimidating and you risk of reading a lot of material without really learning and I feel more like an "information dump". Search for smaller book or course that are an "introduction" to compiler instead of the "everything you need to know", they may not teach you everything, but you will have a better understating of "core" concepts that allow you to tackle more complex topics a lot easier.

Side note: in the specific case of compiler search for a book/course that focuses on building an actual small language! The book my professor gave to me only had a disjointed example for each topic. Something like "crafting interpreter" with a precise goal is MUCH easier to understand while teaching the same concept.

Hope this can help, or at least a starting point for a discussion.

edit: formatting


👤 cnotv
You can build a:

- library

- SW combined with the Pi

- find something to mine

- ML prototypes


👤 doelie_
Implement Scheme.

👤 throwaway8581
One nugget of wisdom is that if you are decently intelligent, most of your failure to understand “complex” topics is really due to poor quality explanations from other people. Few engineers are clear writers. Buy books, sort though university course syllabi, watch lectures. That is, consume materials meant for teaching beginners.

For consuming non-beginner material, it’s okay to read a blog post and not fully understand everything. Things will click together over time.