These statements make sense to me, but even with more than 20 years of writing software under my belt, I haven't the first clue as to where the ideas come from.
I've read the PG essays. I'd be happy to work on an unsexy/grunt-work idea, but... what?
I've kept "scratch your own itch" in my head over the years, but my itches are either absurdly tiny (e.g. a Vim script) or require a massive company with the resources of FAANG to pull off.
I've looked at "sell shovels during a gold rush", but FOSS seems to have the AI gold rush covered.
I've asked friends in HR & recruiting what their paint-points are and they claim to not have any; at least not any that could be solved with tech.
I've become something of a SME in two problem domains outside of software and I haven't had any luck finding intersections with marketable solutions.
So, HN, I ask you: Where/how do you do your market research?
I have the drive, work ethic, and skill to pour myself into failure after failure, but my issue is that I have no idea what to write. It's like having writer's block before committing a single letter to page #1.
This is definitely putting the cart before the horse, as I'm sure many of you will point out, but I want to establish a source of income that is independent of my employability.
A couple of random thoughts:
1. You may just need to ask a slightly different question, or word your question(s) slightly differently. If you word the question in such a way as to come off as "what's a problem you are having that could be solved with tech" then you are not only soliciting the problem, you're asking them to at least partially imagine the solution. And your contact may not be equipped to do that, or may not want to invest the time/energy to do so. So really focus on decoupling the "tell me about what you do" part from "what would a solution be?"
2. If you have a contact who's in the right position to do this, and if you can spare the time, see if you can "shadow" somebody at work for a bit. Preferably multiple sessions spread out across multiple days. You may notice an obvious pain point that your friend wouldn't think to call out because they've already normalized the pain to the point that it doesn't register on their radar anymore.
3. Be wary of asking people who are actually personal friends (as opposed to mere acquaintances, or strangers). In some cases friends may hold back from saying certain things because they actually know you, respect your opinion, and fear embarrassment. In this regard a more distant contact might actually be better exactly because you're "just some stranger they'll probably never see again."
4. Related to all of the above, keep in mind the (probably apocryphal, but still useful) Henry Ford aphorism about "If I'd asked my customers what they wanted, they'd have asked for a faster horse." Again, the customer can't always imagine the solution. Sometimes the onus is on you to generate a candidate solution to some problem, mock it up sufficiently for the customer to understand / appreciate it, and present it to them from a position of "If this solution existed, would you want to use it?"
5. I suggest reading The Four Steps to the Epiphany by Steve Blank if you haven't already. The early chapters cover a lot of this ground in excruciating detail.