Artificial intelligence now beats some of the best human forecasters

xgulfie

Hasn't this been true for like 40 years

296012

That is too bad for The Economist. Exor N.V and Agnelli might replace some pundits at The Economist.

glimshe

Product idea: a LLM trained separately from mainline LLMs that anticipate market trends by analyzing how mainline LLMs will invest. As retail investors will probably use mainline AI for decisions going forward , one could get an edge. "The AI-driven Market Hypothesis" Please let me know where I should pick up my Nobel prize.

croes

Given the training data isn’t that more a win for the wisdom of the crowd?

bagels

Aren't forecasters already using 'artificial intelligence' for decades in the form of non-llm machine learning models?

qsbuilder

The test is when reflexivity kicks in and the prediction itself changes market behavior. LLMs usually melt there

tolugenius

Archive Link: http://archive.today/IVreS

autoexec

So I guess the AI companies can stop with their plans to infest AI with ads and they'll instead fully fund themselves by using their AI to gamble on stocks and the prediction market right? Surely the chatbots will just print money!

gyanchawdhary

At the risk of sounding extremely naieve i have a question for the Wall St / quant / HFT folks lurking here ... but how hard would it actually be to brute force the math/algos behind Medallion Fund (or something in that general class) or even some of the average quant funds I know it’s not just the math but execution, infrastructure, risk management, data, colocation (if ur an HFT) etc ... but LLMs seem like a pretty powerful apparatus for running experiments that .. a few years ago would have required fairly deep multidisplinary skills across coding .. stats .. and math .. So assuming you have decent intuition for ideas .. how difficult would it actually be to reverseengineer / rediscover some of the underlying stuff?

seanhunter

This has to be the least surprising development to date given ml is a universal function estimator

jesse_dot_id

It will be interesting to see if this changes because presumably AI is using very predictable historical models, but it seems like the climate is shifting into something unseen that we won't have models for?

mbil

See also The AI Superforecasters Are Here https://www.astralcodexten.com/p/the-ai-superforecasters-are... and discussion https://news.ycombinator.com/item?id=48806296

attels33

So my plan to go from a developer to an economist is scrapped. What now?

throwaway5752

The best human forecasters working with artificial intelligence are going to do even better than either alone, the dichotomy is artificial.

ratelimitsteve

If 10,000 people guess 10,000 fair coin flips each one of them will get more guesses right than any of the others, one of them will get fewer guesses right than any of the others, and the gulf between the two is likely to be over 4 standard deviations wide. I'm certain that I, being an untutored schmuck from Pittsburgh and having thought of this almost immediately after reading about this contest, cannot be the first person to realize this is a potential problem for a forecasting contest. But I can't find anything they've done to mitigate that problem. Can anyone clue me in?

johnecheck

The markets are a highly complex dynamic system. There are many instances of it exhibiting disastrous behavior, especially in response to changes and shocks. AI trading and investment advice meaningfully changes the system and its dynamics. It seems highly probable that this will result in it failing in new ways.

phyzix5761

Stock analysts have a success rate of 47% or lower for directional predictions. That's worse than a coin flip. All AI has to do is product fair 50/50 results and it can beat analysts. But you can do it too for the price of a quarter.

gertlabs

We measure skill differentiation between frontier / last-gen LLMs across our environments, and one of our curated coding environments is a closed-system market simulator, containing only other agents and some system participants (a market maker and a liquidity provider via issuance / buybacks) whose behavior is fully defined for all of the agents. This has the least measured skill differentiation of all of our environments, and not because forecasting/markets don't require skill or intelligence. Even the best models are so far from anticipating the behavior of the other agents and understanding the emergent effects that a 2025 model with a naive strategy can often outperform over the timeframes of the simulation simply because some other models in the simulation chose a similar self-reinforcing strategy. This likely happens to some degree in real markets. You can watch these simulations here https://gertlabs.com/spectate?game=market

baobabKoodaa

Anyone who believes this news story should create their LLM slop bot to trade on prediction markets like Polymarket and Kalshi. These acceletards provide a great influx of money to many human traders on these platforms.