A Study of Microsoft's Early 2026 Rollout of Claude Code and GitHub Copilot CLI

gojkoa

The study unfortunately looks only at individual productivity, not any org gains, and the big claim in the PDF is that adopters "merge roughly 24% more pull requests" over a four month period. not exactly headline-making material. There's no data in the paper whether those 24% extra pull requests actually added anything more valuable or not.

anshumankmr

>281 billion tokens. Using the least expensive version of Claude Opus 4.6, which costs $5 for every million tokens, that one user alone could have cost Meta more than $1.4 million. And that is based on the pricing that they sell a dollar for less than a dollar so the costs may be much much more.

simonreiff

I really wish this study had chosen to focus on more useful endpoints like revenue, profitability, customer retention, conversion rate, etc....at a minimum, don't look at PRs merged without also looking at bugs in production code, number of incident reports, features shipped, debt log canceled, etc. Even if you only want to consider merged PRs, you should weigh the productivity of the reviewer and whether they are also 24% boosted or else it's just an accounting trick.