Audio, Speech & Video Temporal Grounding practice drill
10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.
Sample scenarios
PROMPT
Transcribe the attached 12-second insurance-call audio clip verbatim.
OBSERVATION
At 00:07, the policyholder clearly says, "I can't authorize the deductible change until Friday," with a brief glottal stop on can't.
MODEL A
I can authorize the deductible change until Friday.
MODEL B
I can't authorize the deductible change until Friday.
Why does Model A receive a major ASR severity deduction despite having only a single-word error?
Low WER tolerance: verbatim ASR tasks automatically fail any transcript whenever overall word error rate exceeds 2%.
Dropped negation: turning can't into can reverses the speaker's legal authorization and meaning.
Contraction expansion rule: Model A violated style guidelines by failing to expand spoken contractions into cannot.
Timestamp omission error: Model A did not bracket the glottal stop at 00:07 with an inline acoustic hesitation tag.
Show the answer
B. Dropped negation: turning can't into can reverses the speaker's legal authorization and meaning.
In ASR RLHF evaluation, not all single-word errors carry equal weight. Dropping a negation (can't → can) flips the polarity of a binding authorization statement, making it a critical semantic failure even though the raw word-error count is only one word.
Principle: ASR Evaluation: Negation Drops & Polarity-Flipping Errors
PROMPT
Transcribe the attached radiology dictation clip accurately.
OBSERVATION
The physician dictates: "Small polyp noted in the terminal ileum; no fracture of the right ilium." Both words sound nearly identical phonetically (/ˈɪliəm/), but ileum is the small intestine and ilium is the pelvic bone.
AI RESPONSE
Small polyp noted in the terminal ilium; no fracture of the right ileum.
What ASR failure must an evaluator flag here?
Diarization mismatch: the model merged the radiologist's clinical impression with a background nurse's note.
Punctuation normalization error: clinical dictation guidelines forbid semicolons between two anatomical clauses.
Acoustic clipping artifact: rapid dictation speed truncated the final syllable of both Latin medical terms.
Homophone swap: it inverted ileum (intestine) and ilium (pelvic bone), corrupting clinical sense.
Show the answer
D. Homophone swap: it inverted ileum (intestine) and ilium (pelvic bone), corrupting clinical sense.
Homophones sound identical acoustically, so accurate ASR depends on domain and syntactic context. Swapping ileum (where a polyp occurs) and ilium (the pelvic bone that can fracture) produces a major clinical accuracy error that human evaluators must catch and penalize.
Principle: ASR Evaluation: Context-Dependent Homophones & Domain Terms
PROMPT
Produce a speaker-diarized transcript and summarize who requested the refund in this two-person support call.
OBSERVATION
Speaker 1 (Customer) asks at 00:18, "Please cancel my annual plan and refund the charge." Speaker 2 (Support Agent) replies at 00:24, "I've initiated the cancellation, and the refund will post in five business days."
AI RESPONSE
[Speaker 2]: Please cancel my annual plan and refund the charge. [Speaker 1]: I've initiated the cancellation... Summary: The support agent requested a refund from the customer.
What exact audio evaluation defect occurred?
Speaker diarization swap: inverted speaker labels reversed who requested versus processed the refund.
Acoustic silence hallucination: the model inserted phantom words during the 6-second pause between 00:18 and 00:24.
Prosody misattribution: the model mistook polite agent intonation for an urgent customer escalation request.
Verbatim filler omission: the model stripped vocal pauses from Speaker 1, shifting the turn boundary by six seconds.
Show the answer
A. Speaker diarization swap: inverted speaker labels reversed who requested versus processed the refund.
Even when the transcribed words are 100% accurate, swapping speaker identities (Speaker 1 vs Speaker 2) corrupts who said what and propagates directly into downstream summarization errors. Diarization attribution accuracy is a core axis in multi-speaker audio RLHF.
Principle: ASR Evaluation: Speaker Diarization & Role Attribution