A sufficiently detailed spec is code

measurablefunc

I agree with the overall structure of the argument but I like to think of specifications like polynomial equations defining some set of zeroes. Specifications are not really code but a good specification will cut out a definable subset of expected behaviors that can then be further refined with an executable implementation. For example, if a specification calls for a lock-free queue then there are any number of potential implementations w/ different trade-offs that I would not expect to be in the specification.

notepad0x90

I agree to this, with the caveat that a standard is not a spec. E.g.: The C or C++ standards, they're somewhat detailed, but even if they were to be a lot more detailed, becoming 'code' would defeat the purpose (if 'code' means a deterministic turing machine?), because it won't allow for logic that is dependent on the implementer ("implementation defined behavior" and "undefined behavior" in C parlance). whereas a specification's whole point is to enforce conformance of implementations to specific parameters.

ranyume

I tried myself to make a language over an agent's prompt. This programing language is interpreted in real time, and parts of it are deterministic and parts are processed by an LLM. It's possible, but I think that it's hard to code anything in such a language. This is because when we think of code we make associations that the LLM doesn't make and we handle data that the LLM might ignore entirely. Worse, the LLM understands certain words differently than us and the LLM has limited expressions because of it's limits in true reasoning (LLMs can only express a limited number of ideas, thus a limited number of correct outputs).

charcircuit

This articles ignores that AI agents have intelligence which means that they can figure out unspecified parts of the spec on their own. There is a lot of the design of software that I don't care about and I'm fine letting AI pick a reasonable approach.

rdevilla

I think it's only a matter of time before people start trying to optimize model performance and token usage by creating their own more technical dialect of English (LLMSpeak or something). It will reduce both ambiguity and token usage by using a highly compressed vocabulary, where very precise concepts are packed into single words (monads are just monoids in the category of endofunctors, what's the problem?). Grammatically, expect things like the Oxford comma to emerge that reduce ambiguity and rounds of back-and-forth clarification with the agent. The uninitiated can continue trying to clumsily refer to the same concepts, but with 100x the tokens, as they lack the same level of precision in their prompting. Anyone wanting to maximize their LLM productivity will start speaking in this unambiguous, highly information-dense dialect that optimizes their token usage and LLM spend...

HoldOnAMinute

I have a lot of fun making requirements documents for Claude. I use an iterative process until Claude can not suggest any more improvements or clarifications. "Is this it?" "NOPE" https://www.youtube.com/watch?v=TYM4QKMg12o

macinjosh

IMHO, LLMs are better at Python and SQL than Haskell because Python and SQL syntax mirrors more aspects of human language. Whereas Haskell syntax reads more like a math equation. These are Large _Language_ Models so naturally intelligence learned from non-code sources transfers better to more human like programming languages. Math equations assume the reader has context not included in the written down part for what the symbols mean.

adampunk

Just waterfall harder

hintymad

> A sufficiently detailed spec is code This is exactly the argument in Brooks' No Silver Bullet. I still believe that it holds. However, my observation is that many people don't really need that level of details. When one prompts an AI to "write me a to-do list app", what they really mean is that "write me a to-do list app that is better that I have imagined so far", which does not really require detailed spec.

adi_kurian

This won't age well, or my comment won't age well. We'll see!

randyrand

True. That's why I only write assembly. Imagine a piece of software deciding register spill for you! Unhinged!

pkoird

Meh, it's the age old distinction between Formal vs Informal language. Simply put: Formal language = No ambiguities. Once you remove all ambiguous information from an informal spec, that, whatever remains, automatically becomes a formal description.

Panzerschrek

I am developing my own programming language, but I have no specification written for it. When people tell me that I need a specification, I reply that I already have one - the source code of the language compiler.

ulrikrasmussen

A corollary of this statement is that code without a spec is not code. No /s, I think that is true - code without a spec certainly does something, but it is, by the absence of a detailed spec, undefined behavior.

sjeiuhvdiidi

Absolute nonsense. A sufficiently detailed "spec" is the code. What is wrong with you people ? Pure nonsense, all they have to offer.

ramoz

The reference spec is a terrible spec. There are also many alternative definitions as to what a spec actually is. You show me the 70K starred'd GitHub spec library and all the jargon that comes with it + 30 different commands it installs. I'll say, "Yeah, that's probably not the way." However, constructing proper context for steering, creating a tactical plan, and saying "go" is a different thing. There are two ways to do AI today. 1. You dog walk it, you throw a ball exactly where you want it, knowing the AI will fetch and come back. (You own the code. You tell the AI to go do very explicit tasks. You can rely on this) 2. You whisper to the dog to go walk itself and cross your fingers hoping it comes back; if you can do this for one dog, you can do this for ten and you can quickly become the greatest dog walker overnight. (You're trying different spec libraries. You're trying GasTown. You're failing, you're faltering); however, you are continuously optimizing every day. - a consistent narrative is leaning into planning / plan modes in really putting thoughtful effort into what's about to happen. This Deepmind paper[1] has aged well. Paradoxically, the more capable AI gets, the more important specification becomes (or costly lack of it becomes), and the more time you spend planning and iterating on intent before you let an agent act. Tokens and code might be cheap, but we are moving closer and closer to where we let AI operate overnight or where we want agents to operate within our actual environments in real time. The tokens or actions in these instances, have higher costs. I say all this because we are very clearly moving into high automation and high agency with AI. Thus, an increasingly important question around this is still worth finding answers to: can a clear statement of intent, combined with growing agent capability and tighter feedback loops, outperform the traditional write-then-review cycle? We are far from a real answer however. [1] https://deepmind.google/blog/specification-gaming-the-flip-s...