Multimodal Vision, Audio & Video Evaluation Practice Test
Auditing AI vision, OCR, chart/UI screenshot reasoning, ASR audio transcription, prosody/tone, and video temporal grounding for multimodal RLHF assessments.
10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.
Timed mock assessment
40 scenarios in 48 minutes, drawn from every topic. No answers shown until the end, then a full score breakdown, your weakest topics and a review of what you missed.
Prices are in US dollars, plus tax where it applies. Stripe, our payment processor and the seller of record, adds tax at checkout.
All sales are final. If something is really wrong, such as a duplicate charge or a purchase that does not unlock what you bought, write to support@annotationprep.com and we will put it right.
One-time payment. Nothing renews.
You have an assessment ticket for this practice test. Start the mock assessment to use it.
Transcribe the attached 12-second insurance-call audio clip verbatim.
OBSERVATION
At 00:07, the policyholder clearly says, "I can't authorize the deductible change until Friday," with a brief glottal stop on can't.
MODEL A
I can authorize the deductible change until Friday.
MODEL B
I can't authorize the deductible change until Friday.
Why does Model A receive a major ASR severity deduction despite having only a single-word error?
Low WER tolerance: verbatim ASR tasks automatically fail any transcript whenever overall word error rate exceeds 2%.
Dropped negation: turning can't into can reverses the speaker's legal authorization and meaning.
Contraction expansion rule: Model A violated style guidelines by failing to expand spoken contractions into cannot.
Timestamp omission error: Model A did not bracket the glottal stop at 00:07 with an inline acoustic hesitation tag.
Show the answer
B. Dropped negation: turning can't into can reverses the speaker's legal authorization and meaning.
In ASR RLHF evaluation, not all single-word errors carry equal weight. Dropping a negation (can't → can) flips the polarity of a binding authorization statement, making it a critical semantic failure even though the raw word-error count is only one word.
Principle: ASR Evaluation: Negation Drops & Polarity-Flipping Errors
PROMPT
Compare Q3 and Q4 uptime from the attached bar chart.
OBSERVATION
The chart's y-axis starts at 96.0% and ends at 100.0%. The Q3 bar reaches 97.0% (25% of plot height) and the Q4 bar reaches 99.0% (75% of plot height).
AI CLAIM
Q4 uptime tripled compared to Q3, jumping from 97.0% to 99.0%.
What multimodal chart-reading error should you flag?
Unit conversion error: uptime percentages must be normalized into downtime minutes before comparing two quarters.
Bar-width distortion: the model confused categorical bar spacing with a continuous quarterly time-series slope.
Truncated y-axis misread: it compared pixel bar heights (3x taller) instead of real values (+2.0 percentage points).
Baseline inversion error: the model read the bar values downward from the 100.0% ceiling rather than upward from the 96.0% floor.
Show the answer
C. Truncated y-axis misread: it compared pixel bar heights (3x taller) instead of real values (+2.0 percentage points).
Because the y-axis starts at 96.0% rather than 0%, a 2.0 percentage-point increase (97.0% to 99.0%) looks three times taller visually. Vision-language models frequently mistake pixel height ratios for value ratios; evaluators must flag this as a major chart-groundedness failure.
Principle: Chart Grounding: Truncated Y-Axis vs. Pixel-Height Bias
PROMPT
Transcribe the attached radiology dictation clip accurately.
OBSERVATION
The physician dictates: "Small polyp noted in the terminal ileum; no fracture of the right ilium." Both words sound nearly identical phonetically (/ˈɪliəm/), but ileum is the small intestine and ilium is the pelvic bone.
AI RESPONSE
Small polyp noted in the terminal ilium; no fracture of the right ileum.
What ASR failure must an evaluator flag here?
Diarization mismatch: the model merged the radiologist's clinical impression with a background nurse's note.
Punctuation normalization error: clinical dictation guidelines forbid semicolons between two anatomical clauses.
Acoustic clipping artifact: rapid dictation speed truncated the final syllable of both Latin medical terms.
Homophone swap: it inverted ileum (intestine) and ilium (pelvic bone), corrupting clinical sense.
Show the answer
D. Homophone swap: it inverted ileum (intestine) and ilium (pelvic bone), corrupting clinical sense.
Homophones sound identical acoustically, so accurate ASR depends on domain and syntactic context. Swapping ileum (where a polyp occurs) and ilium (the pelvic bone that can fracture) produces a major clinical accuracy error that human evaluators must catch and penalize.
Principle: ASR Evaluation: Context-Dependent Homophones & Domain Terms
PROMPT
Which region accounts for the second-largest share of annual revenue in the attached pie chart?
OBSERVATION
The pie chart shows four unlabeled slices clockwise from 12 o'clock: Dark Blue (44%), Teal (28%), Amber (18%), and Slate Gray (10%). The legend lists: Dark Blue = North America, Slate Gray = APAC, Teal = EMEA, Amber = LATAM.
MODEL A
APAC is the second-largest region at 28%, following North America at 44%.
MODEL B
EMEA is the second-largest region at 28%, following North America at 44%.
What caused Model A's error?
Clockwise ordering bug: Model A read the pie slices counterclockwise starting from the smallest Slate Gray wedge.
Sum-to-100 hallucination: Model A redistributed 18% from LATAM into APAC because pie charts lack explicit gridlines.
Percentage estimation drift: Model A misread the 18% Amber wedge as 28% due to perspective skew in the rendered chart.
Legend color-mapping swap: Model A matched the second legend entry (APAC) to the second-largest slice (Teal / EMEA).
Show the answer
D. Legend color-mapping swap: Model A matched the second legend entry (APAC) to the second-largest slice (Teal / EMEA).
A classic vision-language failure on charts is pairing legend text by list position instead of binding each label to its actual color swatch. Here Slate Gray (APAC) appeared second in the legend, so Model A wrongly assigned it to the 28% Teal slice belonging to EMEA.
Produce a speaker-diarized transcript and summarize who requested the refund in this two-person support call.
OBSERVATION
Speaker 1 (Customer) asks at 00:18, "Please cancel my annual plan and refund the charge." Speaker 2 (Support Agent) replies at 00:24, "I've initiated the cancellation, and the refund will post in five business days."
AI RESPONSE
[Speaker 2]: Please cancel my annual plan and refund the charge. [Speaker 1]: I've initiated the cancellation... Summary: The support agent requested a refund from the customer.
What exact audio evaluation defect occurred?
Speaker diarization swap: inverted speaker labels reversed who requested versus processed the refund.
Acoustic silence hallucination: the model inserted phantom words during the 6-second pause between 00:18 and 00:24.
Prosody misattribution: the model mistook polite agent intonation for an urgent customer escalation request.
Verbatim filler omission: the model stripped vocal pauses from Speaker 1, shifting the turn boundary by six seconds.
Show the answer
A. Speaker diarization swap: inverted speaker labels reversed who requested versus processed the refund.
Even when the transcribed words are 100% accurate, swapping speaker identities (Speaker 1 vs Speaker 2) corrupts who said what and propagates directly into downstream summarization errors. Diarization attribution accuracy is a core axis in multi-speaker audio RLHF.
Principle: ASR Evaluation: Speaker Diarization & Role Attribution