Why is it so hard to move from DA to software engineering?

Getting hired isn't about what you have done, it is about what you will do in the future. Sure, you need to have a base level of competence and your history need to show that. But once you have shown that, you are competing with everyone else out there that also has shown the same thing. Hiring managers are then trying to figure out who can deliver the most/best work for the least cost.

So rather than focusing on your newer engineering skills, you could focus on roles where your DA and business skills will enhance the engineering and make you more appealing than the others. Finding those roles is not going to be about the coding, but about your subject matter knowledge - look for roles where the coding supports DA or business processes. Or where the subject matter of the product matches past industry experience.

I looked at all three URLs you provided. They demonstrate AI expertise, but to what degree is impossible to know without your specifically stating your contribution. How does your AI expertise relate to Ruby/Rails SE work?

You claim DA skills -- in what industry?

You have an engineering degree? in what field??

If I were hiring for a ruby/rails SE, I don't see anything that anything that is relevant. I would be concerned that you are "too senior" for a junior SE role.

The only way to break the deadlock is to target companies where you can demonstrate domain expertise and have a number of specific, quantified examples of your contribution which would also benefit your target in comparable situations.

Junior positions are all about college or knowing someone. Business school doesn't count. No hiring manager is going to perk up when seeing it. Network more.

This is especially relevant if you're wanting Ruby jobs over a more mainstream language.

It's not about showing you're competent, it's not about being able to say you went to a bootcamp. It's not about listing the same AI projects that everyone else is doing on a piece of paper.

Seriously. Get to know people who are working in the ecosystems you want to target, and figure out a way to make them like you. That's all this economy is about. Raw skill is useless. Resumes are useless. Just get friends.

There's a bias towards hiring seniors and experienced developers so I presume when they see your CV with your experience, you get left out way earlier in the process.

I do't blame them either because the difference between experienced and non experienced developers is bit hard to explain.

If you must switch - better would be to find some junior positions and stick to that position for a few years (3 years or so) and then gradually move up.

Have you tried looking for positions that are adjacent to you professional experience, such as data engineering? Understanding the use cases of data analysts could put you in a great position to bridge.

don't use rails, use python