Contribution record
This is the member's own record of contributions and responses, to share if they choose. Taking part in the Exchange does not affect applications. It is not a credential or a hiring decision.
Reply in "Investigating a Kalvenor omission"
- Author
- clane
- Posted
- Revision
- Original version
- Thread
- Investigating a Kalvenor omission (Practice)
The contribution
I would start with the matched-wording check, but repeat both variants before attributing the difference to wording. A single differing pair could also reflect ordinary answer variation; absence in both variants would not rule out wording or other causes.
For this fictional exercise, my proposed procedure is to alternate the two prompts in fresh sessions, hold the documented engine/surface and locale constant, and retain the answer and citations for every run. A consistent difference across repeats would increase confidence that phrasing matters under those conditions. Similar omission rates would send me next to source coverage and category fit, without proving an evidence gap.
This is a response to the proposed experiment in the thread, not a claim that I reran the sample. AI-assisted.
The team reply
My first check is category and query fit. Is Kalvenor actually a plausible answer to the buyer question as written, and is its product identity unambiguous? If the prompt is asking for a segment Kalvenor does not clearly claim, omission may be reasonable.
Second, I would inspect the public evidence surface: crawlability, category language, product pages, structured signals, and credible third-party pages that explain what Kalvenor is and who it is for. I would compare that evidence with the sources the engine used for the brands it did name.
Third, I would test repeatability. I would rerun the same question under the same conditions, check other engines, and look for retrieval or capture errors. One omission is an observation, not a diagnosis. Those three checks tell me whether I am looking at a measurement artifact, an entity or category problem, or a genuine evidence gap that could explain why the model had stronger candidates available.
A contribution record shows what was posted publicly on the Broadcastwell Exchange, by whom and when. It is not a certificate or an endorsement.