Multiple LLM work together to give higher than gpt4o performance

Question

I am building a ChatGPT-like platform that, instead of asking one LLM a question, it will ask multiple LLMs all at the same time and create a final output by corroborate output from each mode. Model will work together to come up with the final answer and we see greatly improve performance and reduce hallucinations.Notably, on AlpacaEval 2.0, using solely open-source models, we achieved a margin of 7.6% absolute improvement from 57.5% (GPT-4 Omni) to 65.1% (Mix Model) and 65.7% using close source model.Is this something that people find to be useful? Not sure what are the use case, or is working with one LLM enough

p1esk · Accepted Answer

we achieved a margin of 7.6%Are you one of the original contributors of the work done at together.ai [1], or are you trying to take credit for it?[1] https://arxiv.org/abs/2406.04692

ricopags · Answer

You mean like this?https://www.together.ai/blog/together-moa

altdataseller · Answer

I would not find it useful, because that would entail 5X the costs, and the cost is not worth the extra benefit of less hallucinations

Multiple LLM work together to give higher than gpt4o performance

we achieved a margin of 7.6%
Are you one of the original contributors of the work done at together.ai [1], or are you trying to take credit for it?
[1] https://arxiv.org/abs/2406.04692

You mean like this?
https://www.together.ai/blog/together-moa

I would not find it useful, because that would entail 5X the costs, and the cost is not worth the extra benefit of less hallucinations