I don’t recall the names, but in doing some research on Postgres frontends recently I did find projects in github that integrated fastapi and Supabase auth (IIRC they support a bunch of auth backends, right?). Go forth and search lol
From a can-I-swap-out-my-database perspective, I believe the answer is Yes. The supabase open source tools (I am assuming you want to host it yourself, and just about everything they offer is open source) sit in front of the database instance(s) - the services for Functions, rest api wrapper, management tool, and realtime subscriptions are written in typescript and consume the database like your app does. You don’t need to run all of these services. They probably install their own management tables though.
AFAICT, it doesnt require any pg extensions of its own, though some of its functionalities do require extensions.
WRT overkill, if you don’t mind being the operator of some docker containers, and don’t use all the feature magic like Realtime, you’ll have some cheap database management. Overkill might be running 100 of them :) If you aren’t planning on hosting yourself, and don’t mind requiring an internet connection for your app, then the math is easy: is the money out worth the time you get back?