More here https://archive.is/AaY5O (Original https://hunch.net/?p=13764006 )
smokel
This refers specifically to "AI Frontiers", a research group within Microsoft Research that focuses on fundamental advances in AI. There is no end in sight to real AI frontiers. Perhaps "The End of Microsoft Research's AI Frontiers Group" would be a less ambiguous title.
fraywing
To me, all signs point to US over investment in hardware capacity to support the "frontier" when in reality the next major breakthroughs are likely radical optimizations that end in increased consumer training and inference efficiency. I'm not sure how that, combined with open models inching up on frontier intelligence, is compatible with these massive AI IPO valuations? It seems like closed model AI demand has a lot of contingencies that are ultimately fragile and possibly short sighted.
outside1234
Microsoft is a really shitty place to work right now. There are a hundred stories like this all over the company like this.
AnotherGoodName
This is true in every large company right now. The transformer based approach is seen as a highway to race down in terms of training and building better models with minimal organizational distractions allowed while every company puts its foot down on this highway. Exploratory research into other AI methods is highly discouraged for now and organizations are encouraging transformer based scale out and iteration as a top priority. As an example Meta literally fired it's previously head of AI who held views that transformers won't scale. He wanted to veer off the highway and heavily fund exploratory new types of models. They got someone willing to scale the current method out. Essentially the messaging it "Foot down, there's a straight highway ahead don't divert". I have the view this makes sense too. Current top models are mere (mere in big datacenter terms) trillions of parameters right now. We definitely haven't hit the ceiling and everyone's noticing "Hey this Opus 5.5 with the increased parameters seems much better than the last one". Well that's a trillion parameter model. We'll soon start hitting 10's of trillions and 100's of trillions of parameter models. Unless there's a ceiling which nobody expects they should keep scaling just as they have been. I really do agree that it'd be insane to veer off this highway with exploratory research at this particular point in time. Foot down, get to 10 trillion parameters, see if the scaling holds, if so off to 100 trillion parameters. If the scaling continues to hold we'll be asking the AI to do the exploratory research for us soon enough.
alpha_trion
Lack of focus and long term planning at MS
ChrisArchitect
Some more discussion on source: https://news.ycombinator.com/item?id=50406589
skeledrew
New AI lab incoming?
shermantanktop
The headline makes good bait for AI skeptics. But this is a team of 50 people with a grandiose name. The more interesting idea is that MS has abandoned AI research entirely, presumably because their zillions invested in other companies are a better bet in that area.
antonvs
Real research at most big companies is dead. Unfortunately, what has replaced it is mostly venture-backed startups, where the timeline to a return is still expected to be quite short. As a result, you end up with companies doing basic research under the guise of developing a product, announcing timelines which are never going to be met. Fusion energy and quantum computing startups are some of the most egregious examples of this.
esafak
John was the creator of Vowpal Wabbit, which used to be a popular ML library: https://en.wikipedia.org/wiki/Vowpal_Wabbit It was my introduction to feature hashing.
AndrewKemendo
Well I never thought I’d see Digg used as a primary source This is certainly an interesting timeline in this section of the light cone
redmaple892
The Microsoft AI (MAI) part stood out to me. I bet the politics of being in Mustafa's group were not fun.
jphoward
Almost as fascinating is that the article references Digg as the source - genuinely hadn't heard of Digg for maybe a decade? I thought the DVD encryption fiasco + reddit had ended it?
ansk
If you value the freedom to pursue your own research direction and share your work freely, then choose academia. All the perks of working in an industry research role are provided in exchange for this freedom. If your research aligns with the incentives of industry, you can get the best of both worlds in industry. This was the case for machine learning researchers in the post-AlexNet, pre-ChatGPT era. The good times can't last forever, though.
kvark
His complaints sound childish to me. If the research products were truly innovative and useful, he wouldn't need to force Microsoft to adopt them.
zodwick
Not what i expected, but good read nonetheless
textwriter
Mr. S Altman just said almost all Startups will die within a couple years. This is sad.
brokensegue
That's a name I haven't heard in a while
user_7832
(In the context of Microsoft Research) Man, the history of how large corps fumble major wins should be studied. I remember during my thesis how Microsoft Academic (a Google scholar/Scopus alternative) was ridiculously good at its job. I'm talking, "input the key words and filter out the wrong fields and you've got 20 highly relevant papers that no other search term has found" level. ...Of course, they proceeded to close it down. I can't even express in words how mad and disappointed I was and still am at Microsoft for this alone. It felt the closest to a genie or an oracle in being amazing. (The old Swype vs modern Gboard is probably a close example for those who'd used both.) It's just... crazy, frustrating, and crazy frustrating.
Related Discussions
Found 5 related stories in 155.6ms across 12,324 title embeddings via pgvector HNSW
Discussion Highlights (20 comments)
srean
More here https://archive.is/AaY5O (Original https://hunch.net/?p=13764006 )
smokel
This refers specifically to "AI Frontiers", a research group within Microsoft Research that focuses on fundamental advances in AI. There is no end in sight to real AI frontiers. Perhaps "The End of Microsoft Research's AI Frontiers Group" would be a less ambiguous title.
fraywing
To me, all signs point to US over investment in hardware capacity to support the "frontier" when in reality the next major breakthroughs are likely radical optimizations that end in increased consumer training and inference efficiency. I'm not sure how that, combined with open models inching up on frontier intelligence, is compatible with these massive AI IPO valuations? It seems like closed model AI demand has a lot of contingencies that are ultimately fragile and possibly short sighted.
outside1234
Microsoft is a really shitty place to work right now. There are a hundred stories like this all over the company like this.
AnotherGoodName
This is true in every large company right now. The transformer based approach is seen as a highway to race down in terms of training and building better models with minimal organizational distractions allowed while every company puts its foot down on this highway. Exploratory research into other AI methods is highly discouraged for now and organizations are encouraging transformer based scale out and iteration as a top priority. As an example Meta literally fired it's previously head of AI who held views that transformers won't scale. He wanted to veer off the highway and heavily fund exploratory new types of models. They got someone willing to scale the current method out. Essentially the messaging it "Foot down, there's a straight highway ahead don't divert". I have the view this makes sense too. Current top models are mere (mere in big datacenter terms) trillions of parameters right now. We definitely haven't hit the ceiling and everyone's noticing "Hey this Opus 5.5 with the increased parameters seems much better than the last one". Well that's a trillion parameter model. We'll soon start hitting 10's of trillions and 100's of trillions of parameter models. Unless there's a ceiling which nobody expects they should keep scaling just as they have been. I really do agree that it'd be insane to veer off this highway with exploratory research at this particular point in time. Foot down, get to 10 trillion parameters, see if the scaling holds, if so off to 100 trillion parameters. If the scaling continues to hold we'll be asking the AI to do the exploratory research for us soon enough.
alpha_trion
Lack of focus and long term planning at MS
ChrisArchitect
Some more discussion on source: https://news.ycombinator.com/item?id=50406589
skeledrew
New AI lab incoming?
shermantanktop
The headline makes good bait for AI skeptics. But this is a team of 50 people with a grandiose name. The more interesting idea is that MS has abandoned AI research entirely, presumably because their zillions invested in other companies are a better bet in that area.
antonvs
Real research at most big companies is dead. Unfortunately, what has replaced it is mostly venture-backed startups, where the timeline to a return is still expected to be quite short. As a result, you end up with companies doing basic research under the guise of developing a product, announcing timelines which are never going to be met. Fusion energy and quantum computing startups are some of the most egregious examples of this.
esafak
John was the creator of Vowpal Wabbit, which used to be a popular ML library: https://en.wikipedia.org/wiki/Vowpal_Wabbit It was my introduction to feature hashing.
AndrewKemendo
Well I never thought I’d see Digg used as a primary source This is certainly an interesting timeline in this section of the light cone
redmaple892
The Microsoft AI (MAI) part stood out to me. I bet the politics of being in Mustafa's group were not fun.
jphoward
Almost as fascinating is that the article references Digg as the source - genuinely hadn't heard of Digg for maybe a decade? I thought the DVD encryption fiasco + reddit had ended it?
ansk
If you value the freedom to pursue your own research direction and share your work freely, then choose academia. All the perks of working in an industry research role are provided in exchange for this freedom. If your research aligns with the incentives of industry, you can get the best of both worlds in industry. This was the case for machine learning researchers in the post-AlexNet, pre-ChatGPT era. The good times can't last forever, though.
kvark
His complaints sound childish to me. If the research products were truly innovative and useful, he wouldn't need to force Microsoft to adopt them.
zodwick
Not what i expected, but good read nonetheless
textwriter
Mr. S Altman just said almost all Startups will die within a couple years. This is sad.
brokensegue
That's a name I haven't heard in a while
user_7832
(In the context of Microsoft Research) Man, the history of how large corps fumble major wins should be studied. I remember during my thesis how Microsoft Academic (a Google scholar/Scopus alternative) was ridiculously good at its job. I'm talking, "input the key words and filter out the wrong fields and you've got 20 highly relevant papers that no other search term has found" level. ...Of course, they proceeded to close it down. I can't even express in words how mad and disappointed I was and still am at Microsoft for this alone. It felt the closest to a genie or an oracle in being amazing. (The old Swype vs modern Gboard is probably a close example for those who'd used both.) It's just... crazy, frustrating, and crazy frustrating.