Show HN: AI Roundtable – Let 200 models debate your question
Hey HN! After the Car Wash Test post got quite a big discussion going (400+ comments, https://news.ycombinator.com/item?id=47128138 ), I spent the past few weeks building a tool so anyone can run these kinds of questions and get structured results. No signup and free to use. You type a question, define answer options, pick up to 50 models at a time from a pool of 200+, and they all answer independently under identical conditions. No system prompt, structured output, same setup for every model. You can also run a debate round where models see each other's reasoning and get a chance to change their minds. A reviewer model then summarizes the full transcript. All models are routed via my startup Opper. Any feedback is welcome! Hope you enjoy it, and would love to hear what you think!
Discussion Highlights (17 comments)
capitrane
https://opper.ai/ai-roundtable/questions/is-the-ai-roundtabl... seems like it is a good idea?
felix089
Whoever just asked this, very funny: https://opper.ai/ai-roundtable/questions/does-mr-krabs-evade...
totisjosema
Which AI lab has higher ethical standards: https://opper.ai/ai-roundtable/questions/8f5b4f55-617 Do you think its alright that AI labs scraped the internet without respect for copyright and now sell closed models? https://opper.ai/ai-roundtable/questions/86864de8-251 Very interesting to read the transcripts. And seeing how they manage to convince each other. Opus 4.6 seems to really get the others changing their minds
infosecphoenix
this is very interesting! I wonder if we need that many models to join the discussion. Have you tried fewer models?
Cider9986
What is the most important amendment in the constitution of the USA? https://opper.ai/ai-roundtable/questions/e4cb234e-be4
gsandahl
Oh lord, imagine asking ”serious” questions https://opper.ai/ai-roundtable/questions/you-are-standing-in...
cdnsteve
Cool project! This is also extremely useful to compare model bias across the board. There are some disturbing trends on certain topics.
Ancalagon
Love this. I asked about climate change cause that's been on my mind lately. Looks to be very split among the models.
chabes
Are there any dating apps that operate on incentives that favor the users? https://opper.ai/ai-roundtable/questions/e499206c-0c9
tonymet
great tool! I found it useful for challenging "lies my teacher told me". It would be nice to support collections of claims, with a table of summaries. I would love to list out a few dozen phony concepts from school, and have a sharable chart of the rejections, that expand. I really like the UI. It's nice to read the expanded results. But how do you afford the tokens?
whattheheckheck
Run it on the All Souls College Entry Exam
jacquesm
Great idea. I'd love for there to be an 'open ended answer' without giving multiple choice options. Like this they are not debating the question itself but the validity of the possible answers and the real answer to the question may not be contained within that set because the person asking is unaware of that option.
soared
Really cool! Surprising amount of value to seeing the models debate and disagree, I wish I had this at work to have models argue over whether the documentation they provided me are accurate. I would like to see a devils advocate - it seems some of the models kind of repeat the same ideas rather than considering incorrect ideas.
chabes
Been enjoying playing with this. It would be cool if the human user could be a participant in the debate, getting a vote and the chance to state their reasoning.
chabes
Oof, not good folks… What year is it? https://opper.ai/ai-roundtable/questions/7a0c31ce-aac
mizzao
It would be amazing to be able to ask open-ended questions without having to specify the answers in advance.
schrepa
reminds me of karpathy's LLM Council, I use variation of this in my workflow where I pass their opinions back and forth to various models until they achieve some sort of consensus