Open-source(?) tooling for unified API's and LLM integration

I’m curious if anyone here is working on or aware of open-source tools that unify APIs to simplify LLM integrations. Specifically, I’m looking for something that allows me to define a set of integrations, enable customers to configure their usage, and then convert those definitions into tool-use JSON for an LLM.

Currently, I’ve built a work around system like this in-house for my early-stage ai agent startup. While it works, the process is pretty manual and time-consuming. I’d love to find an open-source framework that could streamline or enhance this setup as we scale.

If you want a startup idea this is probably a pretty solid one and I would be your first customer.