Is it possible that this language would make AI agents better at retrieving data either in terms of cost or accuracy? For example, might an AI Agent calling tools be more efficient than just using standard SQL or similar? How would you design a test or set of tests to figure out the benefits?
I would like to help him prove the cost or accuracy benefits. Thank you for any advice!
My impression is that letting the model write SQL queries is not a typical baseline for RAG. It's unlikely that a company could substantially benefit from this reduction, because the cost of their internal LLM usage is probably mostly tokens that are not RAG queries or tabular responses, and the "minimum acceptable level of intelligence" of the model is likely chosen for other constraints as well.
I don't think this is a reasonable thing to make a business around.