Skip to content
BroadcastwellStart free
Menu

QuestionMeasurement

What is the smallest useful answer record?

Score 0
Sairam SivakumarReaderBroadcastwell team, Founder

Bring a receipt. Answers that carry a published receipt are easier for everyone to check.

Here is my proposed record for reviewing one answer:

  • Exact question
  • Engine and experience
  • Capture time
  • Locale
  • Account context where relevant, without private identifiers
  • Full answer, where you have permission to share it
  • Cited URLs
  • Validity decision
  • Reviewer note

This is a proposal, not a claim that every source exposes every field. Unknown fields should stay unknown.

What would you remove or add? Share a blank template if you have one that others can reuse without exposing private information.

Source: broadcastwell.com/methodology

3 answers

The accepted answer first, then by score, then oldest first.

  1. Answer by Rohan Iyer

    Score 0
    Rohan IyerWritten with AI assistanceReaderBroadcastwell team, AI Engineer

    If I could keep only five fields, I would keep: the exact question, the engine and surface, the capture timestamp, the complete answer text, and the evidence attached to the answer.

    The exact question is the experimental input. The engine and surface tell me which system produced the output. The timestamp matters because model behavior and retrieval can change. The full answer text is the ground truth for later scoring; storing only a score throws away the thing we may need to re-audit. For the fifth field, I would store citations or source URLs together with any resolved named entities, because that lets us separate "the model named the brand" from "the model used the brand's site as evidence."

    If I had room for a sixth field, it would be a run identifier that ties retries, errors, and capture metadata together without exposing private account information.

  2. Answer by clane

    Score 0
    claneReader

    I would keep capture evidence separate from later scoring, so a reviewer can change a coding decision without overwriting the original observation. A compact blank template could be:

    • record_id / attempt_id:
    • exact_question:
    • engine_and_surface:
    • captured_at_utc / locale:
    • session_context / unknown_controls:
    • capture_status: complete, incomplete, or failed
    • permitted_answer_text / cited_urls:
    • scoring_rule_version / reviewer_decision / reason:

    A retry should be a linked new attempt, not a replacement for the first result. Where an answer cannot be shared, record the restriction rather than silently substituting a paraphrase as the original. No account IDs, cookies or tokens belong in this public template.

    AI-assisted design proposal.

  3. Answer by Sairam Sivakumar

    Score 0
    Sairam SivakumarReaderBroadcastwell team, Founder

    clane, linking each retry to its original attempt and keeping scoring separate would preserve the history needed for a later correction. The sharing restriction is also part of the evidence record; a paraphrase should never masquerade as the captured answer.

    One useful addition is a reference to the retained source file and a checksum, with access restricted when necessary. That can show whether the file changed, though it does not establish that the capture itself is authentic. A reviewer should still be able to trace the record to its actual collection method and permitted evidence.

    Source: broadcastwell.com/methodology