Why is web development such a daunting task?

web dev is so confusing to me (i'm an electrical engineer), but I've built plenty of (simple) websites. I try to avoid...everything. Right now I create websites with hugo which gets me up and running very fast. I hate dealing with backend stuff, so on my latest projects I have a cron job running a python script, that python script updates some markdown files, and then it does a push to redeploy my website on netlify every 1-2 days (the data doesnt need updated much faster). Maybe its insane, but its simple and it works.

Web development is like giving a box of 50 different lego kits jumbled together to your kids, while a lot of other development is like buying a new lego kit from the store and giving it to your kids.

For a lot of tech there are fairly settled abstractions with a low number of meaningful choices. For web development, there are a huge number of abstractions that are not so easily comparable because they run at different levels and have lots of choices that will have significant impact on how things work and how things scale.

You can render JS in a browser, in your server, in a server that your server talks to. You can serve relatively static files, or you can serve files that set up an environment on the browser. Even in your application itself, you can decide to make your view a layer or you can embed it within your business logic.

So you have 4 layers that determine what eventually show up in a browser -> CDN/cloud front end -> application front end -> backend, and the internet as a whole has not decided what behavior belongs in what location and where the hard lines for where those abstractions begin and end are. I mean, we even have webasm. You could serve a virtual OS to a browser and make it run a webserver that runs a browser that runs a web page.

That's before consideration of the infinitude of framework choices and lack of a centralized "authority" (authority from merit, not from force) telling people what the best choices are and why.

So web development is daunting because there aren't many well defined abstractions. The abstractions are weak enough that there aren't many barriers you run in to which limit what you can do. So it's like making a lego structure, but half of the pieces are actually play-dough.

The ecosystem sucks. Mainly because of Javascript and the frantic development of abstraction layer over abstraction layer. Everything mainstream in the web development side of things is being sold like pain-killers by frolic, anthropomorphized poodles... frenetically pushing the message that MORE IS MORE, AND YOU NEED MORE! They're literally flooding an entire practice with corn syrup for the benefit of no one. Compilers, transpilers, packers, package managers, frameworks on top of frameworks, javascript on the server, web applications as desktop applications, etc... It is all shit.

I don't blame you... The only way I would recommend anyone to get into webdev is if they actually have some sort of way of guaranteeing that they are going to be working with a good senior team that cares about building software that lasts... and refuse to hire ((new and shiny product)) devs.

My theory is just that Web Development is immensely popular, probably 90% of jobs in my country (as an example) are either frontend or API backend; 50% of those jobs might be frontend. It's much much more likely a company needs to ship a website than a device driver, a game or a desktop app. Nowadays even many of those are built using web technologies to have a bigger hiring pool and ship faster.

More companies doing web development => more developers doing and learning web development => more tools, libraries and frameworks for web development => more code being written for web development in different styles or with different architecture => bigger chances for spaghetti code and unnecessary complexity, also bigger chance for just bad code in general (because more code) => these ideas and tools just permeate back into the ecosystem.

Whereas in other areas I have a feeling that since there is not enough people working on a certain area, they might all use the same few tools and these would grow more mature.

I don’t know why but many people try really hard to make things far more complicated in JS than they need to be with getters/setters, class privacy, over use of this, and so on. You can really accomplish about 98% of everything you will ever need with just functions, events, and data structures. Not only does this eliminate a bunch of complexity but your application will almost certainly execute much faster as well.

Secondly, tree structures scare the shit out of most people. I am not sure why but most people will gladly perform 10x more work than navigate tree structures directly. Examples of tree structures are the file system, the DOM, process trees.

In many times I have seen developers brag about all their experience only to make a bunch of bad decisions out of emotional fear, a self-inflicted harm. That said I don’t find web tech challenging but for many people there is a great unnecessary self-inflicted challenge.

I think it helps to come to peace with the fact that the web is extremely diverse, then pick a tool and learn it. In reality, table stakes is HTML, CSS, and at least a general understanding of JavaScript (not a deep dive). Then I'd pick a framework that makes these things a little more intuitive. In my opinion, that's NextJS, React, and Tailwind -- almost solely because there is a pretty large mindshare around them so there are a lot of tools and libraries that support you. Once you get the hang of these things, frontend development becomes pretty easy.

As a non developer it's even more daunting because if I want to create something it's likely to be a web app and to learn about front end + back end + infra/hosting + security + database just to build a basic CRUD app it's a ginormous second step on the learning curve to push past the first step of taking a python bootcamp (with a session or two on Flask if lucky) for example.

I'm looking at the Odin project but more realistically I'll just keep slowly creeping upwards from no code (built a half dozen apps on Glide, Adalo) to Low-Code (Noodl, tadabase) to serverless (pythonanywhere).

Because you people won't just do http://motherfuckingwebsite.com/ and be happy. It might be a joke but I am serious.

Skip javascript entirely. Pynecone (https://pynecone.io/), htmx (https://htmx.org/), Vaadin (https://vaadin.com/), Buffalo (https://github.com/gobuffalo/buffalo) all exist and can help you avoid some of the mess that is web/JS development.

Web development is simple. But do it in the most productivity way is hard. Deployment is hard. Monitoring is hard. Correctness is hard. Scalability is hard. Testing is hard. Designing is hard,...

The only easy part of web development is code. That's the fact and the illusion for your question.

try to do something extremely basic with *only* HTML. Think "hello, world" in HTML. From there, add some tiny bits of complexity, like changing the font/colors, or adding a form. Learn how to submit the form without JS, then with JS. Now, add buttons with event handlers. By this point you've covered almost everything. I used a similar technique to teach my first employee how to be a web developer, and he started with almost no experience. Don’t worry about frameworks or libraries until you know how browsers and HTML work.

Simplify your idea for a first version. I'm an experienced web dev, but the last project I wrote was to republish a hard-to-read government feed into an easier format:

1) A very simple web scraper in python that scrapes one page every 2 minutes

2) A database insert if there's new data

3) A separate script launches to download and rename a jpg

For the web side of things:

1) PHP reads and displays the most recent N entries

2) With minimal HTML & CSS

The whole thing is a couple hundred lines, not counting the HTML/CSS. It provides a valuable service to 600+ people per day (including the intended user, myself), I built it in about 20 hours, and it's been humming along without change.

It doesn't send emails, notifications, doesn't bill anyone, doesn't have settings, doesn't even have user logins. An obvious improvement would be to cache the page, but it takes like 40ms to generate so who cares? Truth told, it doesn't even have javascript! Crazy in 2023, I know, but I couldn't find somewhere to use it.

Building something really small is a good way to get your fundamentals set. PHP is a language you don't have to already know to display a simple page, so just use it. It's a simple, familiar syntax and readily google-able. You can deploy by FTPing or SCPing a file up to your server. You already understand databases, so you can easily do this--just work within your current skillset, add in the HTML/CSS and get your functionality via minimal PHP.

I think it makes the most sense to work up from the most minimal version, and honestly the minimalism is a bit of a superpower in the web world right now IMO. Just serve HTML/CSS/JPG and you eliminate most of the complexity and can work on whatever the real problem is. Do that on the server in whatever language & paradigm you already know & love. PHP embeds into HTML without any server config, usually, so it Just Works and eliminates all that overwhelming stuff.

More importantly, your already-existing skillset means you can just 1) do the work however you want, get it to a database, 2) build the most minimal app around that data

Most of the magic in any web-app is usually happening server-side, anyway. You're like 95% of the way there, if you choose this well-trodden, boring path.

(For reference, I'm self-taught, started in the 90s on PHP, worked mostly in Rails, and I am just about old enough to start talking about the good old days. I have edited a .htaccess file this week.)

It is because there is just a lot to know:

- HTML

- CSS

- JS

- Backend language

- HTTP protocol

- DBMS

- Ops/Deployment/troubleshooting

... and that's just the high-level needs.

My advice:

- start with tech you know (I guess Go or Java)

- make a dead simple html-only site first. use forms.

- once you're comfortable, add css/JS as needed.

- increment, learning bit n bobs as you go.

I've come from the C++ realm back during the Bad Old Days before C++11, so I know exactly what you feel. My answer was to turn to Clojure and Clojurescript.

> I have considered Java and Go since I have experienced with both languages.

You might as well jump in with either of those. At the end of the day you will be fixing bugs and adding little new features. You can pick up most of the necessary things along the way, likely by fixing those pesky bugs! Probably the biggest takeaway I got from web development is security. Most non-web people never have to worry about that in a significant way. However most the web security problems are mature and well understood; everybody needs user auth, we're all sanitizing inputs, prevent SQL injection, etc.

In short, I think you will naturally come to grok the backend based on what you already understand.

> Write a web app? I've run away many times... it all feels so daunting.

I have rewritten this part three times, and all I can be is Sebastian from The Little Mermaid; The Frontend World is a mess. Life in the Backend is better than anything they got up there!

My advice is to mostly avoid a dynamic frontend if possible. Server-side rendered HTML still works and will work until the Internet falls into the sun. CSS can be a load of complexity, so I recommend you find a nice little CSS framework you like. Personally I reach for Bulma [0] as one of the less complex out there. If you need a little dynamism, note that every HTML element has a bunch of callbacks you can optionally attach JavaScript functions. The Mozilla Developer Network [1] is a godsend of documentation, and comparable to Javadocs in detail. It is huge but very exhaustive, so use search liberally.

When I can't avoid a dynamic frontend, I reach for Clojurescript and one of its React libraries. The simple reason is a backend developer like myself can make progress in something as messy as the Browser. Like actual JavaScript frameworks, Clojurescript structures how I approach and write frontend apps. Relatedly, Clojurescript makes it much easier to convince other backend Clojure developers to wade into the frontend waters and be productive. At the end of the day, its still a transpiler and so I work much like TypeScript and other dialects function.

Also as an anecdote, I bounced hard off frontend development several times before getting comfortable. This might be professional experience talking (see my second paragraph), but maybe it took getting real messy first before I became comfortable.

[0]: https://bulma.io/

[1]: https://developer.mozilla.org/en-US/

I think the worst part of webdev is that it is so fragmented. I mean all of it. The technologies, the tools, the software, etc. You can't do anything simple any longer... You basically have to be an expert in every aspect of the system in order to make sure that you aren't opening yourself up to attacks.

Example of the absurdity: Do you want a self-hosted website? Be prepared to know how to set up a firewall. And the webserver needs to be able to serve static files when appropriate, and PHP/Python/JS/Whatever when appropriate. Don't mess up your apache or nginx configurations!!! Most modern frameworks require you to know from the start all of you HTTP verbs. You can have HTTP/1.1/2/3 support... if you can figure out HTTPS and your DNS records (for HTTP3). Oh, and make sure that UDP is configured in your firewall for H3. You're writing something commercially? Better stay away from GPL-licensed libraries. How do you check on the libraries for your software and the dependency tree? Good luck. We haven't even decided on a backend framework yet, much less the front end! But when you get to the front end, it needs to be Accessible, lest you get sued. And it needs to support mobile. And you have to have GDPR cookie headers (why? again, so you don't get sued). SSO? If not SSO, how do you securely store passwords? Plain text? noobish and irresponsible. Hashed? also noob. Salted and Hashed? Out of date. Salted and Hashed with a variable cost cryptographic function? Better, for now. There's plenty of other things I can hit on.

LEGO is fun, but nobody can compromise your security simply because you put a block in incorrectly. In webdev, if you do something wrong, you can open yourself up to major security issues and potentially civil and criminal liability.

To put it another way, it's the Unix philosophy of "writing small, single-job tasks that can then be combined in amazing ways" versus monolith programs like Excel. Sure, you can do calculations in the command line using awk and sed, but you can also really mess it all up, even accidentally overwriting your data files if you write '>' instead of '<'. There's a reason that big programs exist: they are successful in bundling functionality and flexibility.

I started as self-taught full-stack developer using Drupal almost two decades ago. It was complicated then, and it's even more complicated now. That being said, I loved using Drupal because, as long as you stuck to their APIs, the security was in place. Drupal enforced patterns for extensions, and if you followed those patterns, you could do amazing things without worrying about the possibility of shooting yourself in the foot. Your forms were handled correctly. Your site could be multilingual. It had good SEO out of the box. Everything more or less just worked, and it was still flexible enough to build amazing things with it! (Yes, I know that there are security issues here and there, and you must update when a CVE is discovered, but that's true for any project.)

And yet, even Drupal has moved away from the simple (yes, I'll call it simple back in the 4.5, 4.6, 4.7, 5.x, 6.x, and 7.x days) to a more complicated ecosystem where I'm not entirely sure that I've set up everything correctly.

But what can we do? I don't want to be involved in the politics (project as well as societal) of many of the languages and projects today. In fact, I actively distance myself from those things now.

I miss the old days of writing software that I loved and that I could share with others. I just want to make cool stuff again!