Has anyone gone back to academia once they were past 35 or so?

Yes, it's doable but what are your plans after? Do you intent on an academic career? I can tell you the market for permanent faculty positions in physics is extremely small and ultra competitive. I went the opposite way as you, never worked a real job and spent my whole life in academia until I received a PhD in particle physics at 27 and then went into industry.

The key qualities you need to succeed in your dissertation is a capacity to persist for a very long period of time on difficult problems - difficult not only technically but conceptually. One of the key skills is to learn how to formulate what is even the problem to solve. 90% of a PhD is just supreme stubbornness in the face of adversity.

If you genuinely enjoy it and are willing to forego a salary for 4-6 years (the average time it takes to finish a PhD in the US/Canada is 6 years), the average PhD stipend is about 20-25k - it's a very exclusive experience and people will always give you a default amount of deference on most matters if you don't care for modesty.

But if you just want a boost to your career or like Dr. in front of your name I advice against it. Generally speaking a PhD is always a net-negative for personal wealth creation.

A couple of women I know married men who were getting Ph.D.s in physics. Both husbands are now doing data science.