Show HN: OculOS – Any desktop app as a JSON API via OS accessibility tree
Single Rust binary (~3 MB) that reads the OS accessibility tree and gives every UI element a REST endpoint. Click buttons, type text, toggle checkboxes — all via JSON. Works as an MCP server too, so Claude/Cursor/Windsurf can control any desktop app out of the box. Windows + Linux + macOS. MIT licensed.
Discussion Highlights (5 comments)
ktpsns
I wonder whether this requires particular GUI toolkits to be used, such as WFC. In any GUI there are enough "bad boy" toolkits which just "draw lines" and thus are not accessible at all.
Frannky
Cool idea! Does this work with electron apps? I tried automating some apps and the problem was that a lot of stuff was never reachable, only via screenshot + click
tadfisher
Maybe I need to familiarize myself with MCP, but wouldn't this make way more sense as a simple CLI tool instead of an HTTP service with a REST API?
lioeters
Good idea, I'm looking forward to seeing it grow - especially the Python and TypeScript bindings.
stif1337
Update: Since the initial post, I've shipped several features based on feedback: - Screenshot capture: GET /windows/{pid}/screenshot → returns PNG - Batch operations: POST /interact/batch → multiple actions per request - Wait/poll: GET /windows/{pid}/wait?q=Submit&timeout=5000 - Python & TypeScript SDKs (local install, PyPI/npm coming soon) - OpenAPI spec, Dockerfile, 7 example scripts - Demo GIF in README showing Calculator automation via Claude Code Thanks for the feedback everyone!