Benchmark: CadQuery vs. OpenSCAD for agentic CAD work

ariwilson

not interesting as the writeup as very AI sloppish :(. i'm sad about that as I would really like to have an intuitive sense for how good the current frontier models are at coming up with interesting 3d printable parts.

dofm

None of the "findings" here really needed an AI experiment to do it: - "CadQuery fails loudly and early... OpenSCAD fails silently and late" This should be obvious, from the documentation, from the way geometry construction works. OpenSCAD has no sense of failure where one shape means another won't work; you're just drawing the equivalent of 3D pixels in space. It will always potentially-meaninglessly succeed if the syntax is OK. - Renders caught nothing that mattered Objects with hollows are not going to show their major problems this way. - CadQuery can be interrogated, OpenSCAD cannot Isn't this explicit from the documentation of both? One works by iteratively building on top of a previous result that can be stored in a variable; the other doesn't. - OpenSCAD renders have no concept of a part edge Again — this should be explicit from the documentation, which describes no methods for operating on edges (and largely from the fact that it is declarative). - Speed favours OpenSCAD, and it barely matters Yes, being faster is no good if things are wrong. - "What decides it is verifiability rather than expressiveness, and CadQuery leads there by a wider margin than the syntax difference suggests." Yes, because the difference is semantic . Which you can get from the documentation. Building iteratively on the basis of existing geometry is inherently more verifiable, because stuff that can't work won't work. Honestly do people not try learning CAD before they try to build an AI tool to generate CAD models? This feels like yet another situation where people who have not done the foundational conceptual work seem to think that they can avoid it and just work around it with AI.

brcmthrowaway

The OP seems to be shilling this tool nonstop. Modelrift just seems to be an LLM wrapper.. the original models can handle OpenSCAD just fine.

tamimio

Should try different models and harnesses, CC and opus5 aren’t the best combo. Also, I expect soon to have models that verify the mesh and even render it not just generating the code. So we got this for mechanical engineering, cupperhead and similar for electronics, usual agents for software, what is next?

fallat

such a garbage post for various reasons, but the ending is the kicker after making SEVERAL observations that cadquery is pretty much better overall, they say "ehhh we're going to stay with openscad".

ricardobeat

Thanks, Claude!

hingler36

Comparing OpenSCAD to CadQuery is super apples-to-oranges in the first place. CQ works directly with NURBS curves via the Cascade engine, and OS functions by performing CSG operations. Both certainly have their place, but for engineering applications NURBS are decidedly more powerful.

darkteflon

I’ve been playing in this space as a moderately experienced hobbyist - mainly for 3D printing functional parts. I’m using a single repo for all my models and Claude with build123d (plus a VS Code viz extension). Over time, I’ve built up a little folder of useful skills and I’m generally getting satisfactory results. That said, I have had my eye on CadQuery and am looking for an excuse to try it. For those in the know: how does it compare to build123d?

Jemm

I've been making Rapidcam.app. The file schema is open source and json format designed for AI to be able to write directly. The format is also designed for version control. The focus is 2.5D parametric mostly for 3 or 4 axis CNC.

syntaxing

Has anyone have good success using AI generated CAD parts? I’ve been trying but it’s always 95% there, but with all hardware, you need 100% right. It’s often quicker and cheaper for me to do it by hand (but I was a mechanical design engineer for about a decade prior)

YuechenLi

Oh, on this topic, I've actually built a geometric CAD kernel with GPT that I think is pretty promising and in my biased opinion, a bit more advanced than OpenSCAD or CadQuery's OCCT backend, anybody here want to clone the repo to give it a try with their Claude/GPT to see they do better on my thing? https://github.com/yuechen-li-dev/Aetheris/ It's still kinda buggy but they are actively getting fixed, but Claude and GPTs seemed to work better with my current design than other CAD stacks. Still haven't added threads yet, so can't speak for T3, but generally speaking the generated parts are pretty good quality, and I also got fillets/chamfers working so that's a plus.

rao-v

I tried modifying the provided CadQuery skill to work with build123d with an agent (surprisingly gemini pretty good at this) and it produced very good results (possibly better than the reported CadQuery/OpenSCAD results after taking a little care to avoid leaking the benchmark pass/fail criteria). Worth a try if you are interested in this space. I like build123d simply because it can export proper STEP (like CadQuery) but has a nicely python friendly design.

rounce

> Both toolchains shipped. The difference is in how they fail. Yuck, this style of LLM output is awful, it’s horrible to read and does a lot to say absolutely nothing.

awestroke

Replicad is another frontend to the same cad kernel. But can run in browser or via bun or whichever js runtime you want. I've found llms to be better at writing typescript than python code for things like this. Using Replicad, I've successfully iterated on a 3d printed enclosure for my electronics project. It's fantastic to get everything parameter based and version controlled from revision 1.