Local LLMs perform better when you teach them to ask before they answer

froh

I'm positively surprised such a little guidance makes such a difference. is it also useful with the smaller (and cheaper) cloud models?

thinkingemote

From the article: "When tasked with coding, writing, editing, or summarizing, ask the user up to three targeted clarifying questions. Proceed with the task once you've received answers and understand the prompt fully. If the task is a simple factual question or conversational message, respond directly."

riknos314

I started using similar approaches in the sonnet 3.5 era and found them incredibly useful at the time. The frontier lab models have gotten significantly better about their guesses over time, but I still sometimes turn to the technique if my own ideation is only about 80% of the way there, as the LLM's questioning can help me identify the blind spots that need more consideration.

kh_hk

Isn't this akin to including all the (missing) keywords from the prompt? YMMV but to me we have found the less optimized way of using LLMs

tana_shahh

Absolutely True not only for Local LLMs but for cloud ones too. Clarifying the intention, the type of output we want improves the model's response multiple folds.

shlewis

This is true even with the SOTA models. Making LLMs ask questions and giving answers is always a good idea. Almost every prompt I write ends with something like this: Unless undoubtedly clear, every decision and action must come from mutual agreement.

halJordan

This is not new knowledge at all. In fact it was discovered before, and is the direct precursor of, Chain of Thought/Thinking models which are now the norm. What's most interesting and surprising is watching all latecomers rediscover optimizations from years ago. Some people really do need to do things the hard way ig.