Meta's Muse is fantastic for web scraping

STRiDEX 25 points 21 comments October 02, 2026
sigh.dev · View on Hacker News

Discussion Highlights (3 comments)

dvt

I built an AI "web harness" running on a sandboxed Chromium (using a custom side-loaded plugin that talks over websockets to a "driver") to basically do anything a normal user could do in a browser. It totally bypasses any and all bot measures and only gets the ones you yourself would get as well (and passes those successfully, e.g. Cloudflare checkbox or those annoying OCR puzzles). Not sure if I should release it, but I'm sure more people are catching onto the power of agentic browsing.

arjunchint

their static ip's were initially good and didn't get flagged, but now most sites are recognizing their ip ranges and blocking. Muse's utility has significantly dropped with the blockages. To become truly useful again they will need to use residential proxies, but I can't see them use those due to the risks and reputational damage.

kevmo314

Muse ran into a captcha and asked me if I wanted it to solve it. So of course I clicked yes and it dutifully convinced the site that it was not a bot.

Semantic search powered by Rivestack pgvector
8,245 stories · 76,959 chunks indexed