Show HN: I built a 2-min quiz that shows you how bad you are at estimating
I've gotten to the point in my career where I now make strategic decisions often (hiring, firing, choosing what equipment to go with, etc.), as well as in my personal life where I need to strongly weigh my options for a big purchase or investment. I found a not-so-surprising parallel between the two as these decisions "resolved." Am I making good decisions or am I getting lucky? Did some research, read some books, and realized I should get in the habit of tracking my decision process. That quickly turned into the idea that formed Convexly. The landing page is a 10-question calibration quiz where you assign a confidence level to statements drawn from a rotating pool of 100 (working on making the pool larger) and you get a Brier score back instantly. No signup required, and you can share your scores right away. If you find it interesting, you can create a free account where you can track your decisions with probability estimates, resolve them over time, and get calibration curves that show if you are over/underconfident. From what I've seen so far, users are overconfident when they say they're between 70-90% sure about something. For the math: Beta-PERT distributions for the payoff modeling, Kelly criterion for the position sizing, signal detection theory for separating skill from randomness. On the coding side: FastAPI with NumPy/SciPy, frontend in Next.js and Supabase. So far this has been a solo project of mine. If you want to see all the features use code SHOWHN for 30 days of full access, no credit card required. Curious if anything about your score surprised you after taking the quiz.
Discussion Highlights (20 comments)
Evgeniuz
There’s a bias, I think. When I saw the title that is about how bad I’m at estimating, I’ve leaned towards counterintuitive answers. This got me quite a high score. I think test set should also include intuitive facts (or maybe I was just lucky).
convexly
Interesting data from the quiz so far: 160+ quiz takers! The average is 0.239 (barely better than a coin flip at 0.25), but almost everyone indicates they are confident in their answers.
convexly
Update: 400+ quiz takers now... insane. Best Brier score so far is 0.007 (nearly perfect calibration). The worst came in at 0.600. Average is 0.230, still just better than a coin flip. Where did you land?
iamtedd
Why do I need to sign up to get the results? Why couldn't it just be on the page?
reltnek
I think this might be conflating confidence with accuracy. I tried leaving the slider the the middle (nominally the least confident position) and it gave a score of 0.25 and diagnosed it as 'overconfident'.
testycool
I thought it was interesting, but don't appreciate having to give you my email to see full results. I unsubscribe from mails that aren't useful to me day-to-day because they're distracting. Other than that it seems like a cool idea. I'd recommend slightly bigger fonts. I often have this issue with Gemini. Brier Score: 0.216 (lower is better) Diagnosis: Overconfident
suralind
Did it twice: once had 0.177, 2nd time got 0.280. Note sure what to make of this, I guess I should always leave it on 50/50?
convolvatron
I didn't find the questions very representative about estimation. that is maybe if happen to know many of random root facts about the world under which they were based, then their application might be a revenant question about ability to estimate. I really felt more like I was making uneducated guesses (0.155). I suppose I was expecting more ping pong balls in airplanes
zupa-hu
It is very disappointing that you can't see what you got right or wrong without giving out your email. I'm not even sure if one would learn from the email or whatever the calibration result is. I'm happy for you if it works but I sure feel cheated. I hope others also feel it's against the spirit of a Show HN. But maybe it's just me.
fred_is_fred
Is it down? The start and skip button both dont work and I see this error in my console. Manifest fetch from https://www.convexly.app/manifest.json failed, code 403
lorenzohess
Maybe I don't know enough about "calibration" in a technical sense, but it seems like this quiz cant really distinguish between factual knowledge and calibration skill? Is this type of quiz reproducible for individuals and across various cross-sections of the population? Are there studies on this? Is the quiz based on these studies?
addisonl
> Question: A fair die rolling a 6 twice in a row is more likely than rolling 1-2-3-4-5-6 in sequence Two 6s in a row is 1/36 chance (1/6)^2 1-2-3-4-5-6 is a 1/46656 chance (1/6)^6 Website is claiming they are the same probability: > Same probability: 1/46,656 — Both outcomes have exactly the same probability: (1/6)^6 = 1/46,656. This illustrates the representativeness heuristic — random-looking sequences feel more probable than ordered ones. Website's "answer" is wrong: was the question supposed to be rolling a 6 six times in a row?
Hnus
Why is it asking for email?
gcanyon
Wait, so roughly is it rewarding being confident when correct, and penalizing being confident when wrong? Meaning that the highest score is only achievable if you answer fully confident true or false, and get all 10 correct? If so, isn't that conflating knowledge with over/under confidence?
loloquwowndueo
“You averaged 97% confidence but were right 80% of the time.” Heck yeah.
rahimnathwani
This reminds me of: https://taketest.xyz/confidence-calibration The same site also has something with a fixed confidence level: https://taketest.xyz/ci-calibration
sonofhans
I’ve taken the quiz but not been compelled to sign up. The site feels manipulative, e.g., the “show me all the questions” link is tiny and hidden between two larger boxes, and even then it only shows 2 questions with a signup CTA. Maybe that’s best practice growth hacking these days, but to me it’s a manipulative turnoff. If you’d given me all the questions and answers simply then I would signed up for more, especially with the discount code. Otherwise, how am I supposed to even know what I’m signup up for? Every interaction I’ve had with the site so far is a sales attempt, so mostly I expect more of those.
Havoc
I'd consider removing some questions that are bound to be country specific. e.g. The one about time spent in front of a red light. >0.188 Slightly above avg - yay
convexly
Update at 2 hours: 1350+ quiz takers! 50% overconfident, 40% well-calibrated, and 10% underconfident. The average score is around 0.228, with the best score still at 0.007 (nearly perfect). The pattern so far is people are most overconfident in the 70-90% range, but are right closer to ~55% of the time.
convexly
Made a few changes based on feedback from this thread: full results now shown immediately with no email gate, changed the UX to include true/false/uncertain buttons + a confidence slider, I cleaned up the quiz result page, and fixed the die probability question. Thanks for all the honest feedback!