Decision Question Designer

Design decision questions and score your own calibrated probabilities.

Decision Question Designer — Build yes/no, choice and rating questions, view calibrated probabilities and Brier scoring.

How to use Decision Question Designer

  1. Type your decision question and pick yes/no, multiple-choice or rating type.
  2. Edit option labels and enter your probability for each option.
  3. Use Split evenly or Normalize to 100 percent to balance the probabilities.
  4. Read the bars and leading answer in the probability visualizer.
  5. When the answer is known, pick the actual outcome and press Log outcome.
  6. Watch your mean Brier score and calibration error improve over time.
  7. Load a preset template any time you need a starting point.

Features

  • Yes/no, multiple-choice and 1-to-5 or 1-to-10 rating question builder
  • User-entered probabilities with bar and gauge visualization
  • One-click normalize to 100 percent and even split
  • Outcome log with per-call and mean Brier scores
  • Calibration error tracking across all logged outcomes
  • Calibration explainer with a worked 70 percent rain example
  • Four preset question templates
  • Copy summary and download JSON export
  • Fully client-side, no model, no network calls

Frequently asked questions

Is it free?

Yes, unlimited.

Login to use this free online tool. Create a free account to get started.