AnnotationPrep

Outlier Multimodal Vision & OCR Evaluation Test practice test

28 practice questions for this assessment, each with the grading rationale and underlying rule.

Start practising

Multimodal vision, audio & video eval: practice drill

10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.

Timed mock assessment

28 scenarios in 34 minutes, drawn from every topic. No answers shown until the end, then a full score breakdown, your weakest topics and a review of what you missed.

Practice tests that prepare you for this assessment

Practise by topic

About this assessment

Platform
Outlier AI
Availability
Remote / Global

Exact question counts, time limits, and project rubrics vary by platform cohort and qualification tier. Always read the platform's live project instructions carefully.

Sample scenarios

PROMPT

Explain the exact sequence of events in this 15-second kitchen video clip.

OBSERVATION

At 00:02 a cat jumps onto the counter; at 00:05 the cat's paw nudges a glass pitcher off the edge; at 00:06 the pitcher shatters on the tile floor; at 00:09 a woman enters holding a broom.

AI RESPONSE

After the woman walks in with a broom, the glass pitcher shatters on the floor, startling a cat onto the kitchen counter.

What video-understanding failure does this response demonstrate?

  1. Spatial occlusion error: the model failed to detect the tile floor beneath the counter edge.
  2. Temporal-order & causality reversal: the model inverted the clip's chronology, turning the aftermath into the cause.
  3. Fine-grained object hallucination: the model confused a plastic watering can with a glass kitchen pitcher.
  4. Framerate subsampling tolerance: swapping events inside a 15-second clip is minor if every entity is named correctly.
Show the answer

B. Temporal-order & causality reversal: the model inverted the clip's chronology, turning the aftermath into the cause.

Models that process video as a 'bag of frames' without strong temporal encoding frequently name the right objects and actions while scrambling their chronological order. Reversing cause and effect (claiming the broom and shatter preceded the cat jumping) is a major temporal-grounding failure.

Principle: Video Grounding: Chronological Ordering & Temporal Causality

PROMPT

Compare Q3 and Q4 uptime from the attached bar chart.

OBSERVATION

The chart's y-axis starts at 96.0% and ends at 100.0%. The Q3 bar reaches 97.0% (25% of plot height) and the Q4 bar reaches 99.0% (75% of plot height).

AI CLAIM

Q4 uptime tripled compared to Q3, jumping from 97.0% to 99.0%.

What multimodal chart-reading error should you flag?

  1. Unit conversion error: uptime percentages must be normalized into downtime minutes before comparing two quarters.
  2. Bar-width distortion: the model confused categorical bar spacing with a continuous quarterly time-series slope.
  3. Truncated y-axis misread: it compared pixel bar heights (3x taller) instead of real values (+2.0 percentage points).
  4. Baseline inversion error: the model read the bar values downward from the 100.0% ceiling rather than upward from the 96.0% floor.
Show the answer

C. Truncated y-axis misread: it compared pixel bar heights (3x taller) instead of real values (+2.0 percentage points).

Because the y-axis starts at 96.0% rather than 0%, a 2.0 percentage-point increase (97.0% to 99.0%) looks three times taller visually. Vision-language models frequently mistake pixel height ratios for value ratios; evaluators must flag this as a major chart-groundedness failure.

Principle: Chart Grounding: Truncated Y-Axis vs. Pixel-Height Bias

PROMPT

Give the exact timestamp range (MM:SS–MM:SS) when the presenter demonstrates the keyboard shortcut Ctrl+Shift+P in this 4-minute software tutorial.

OBSERVATION

At 00:45–00:52 the presenter mentions the command palette verbally without touching the keyboard. The actual on-screen Ctrl+Shift+P keystroke overlay and command palette popup occur from 02:14 to 02:19.

MODEL A

The presenter demonstrates Ctrl+Shift+P from 00:45 to 00:52.

MODEL B

The presenter mentions it briefly at 00:45, and demonstrates the Ctrl+Shift+P shortcut on screen from 02:14 to 02:19.

Why does Model A fail timestamp grounding?

  1. Format violation: timestamps in video QA must always use decimal seconds (134.0s) instead of MM:SS notation.
  2. Audio-only refusal: Model A ignored the spoken explanation entirely because keyboard shortcuts require silent OCR.
  3. Minor drift only: citing 00:45–00:52 is close enough because the topic was introduced at that earlier mark.
  4. Event-onset misgrounding: it cited the verbal mention instead of the on-screen demo (02:14–02:19).
Show the answer

D. Event-onset misgrounding: it cited the verbal mention instead of the on-screen demo (02:14–02:19).

Timestamp grounding requires aligning the user's requested visual/behavioral event—not merely the first transcript keyword hit—to the exact playback window. Model A anchored lazily to the audio mention at 00:45, missing the actual on-screen demonstration nearly 90 seconds later.

Principle: Video Grounding: Precise Timestamp Localization

Taking this assessment soon?

Tell us what skills or question formats you want more drills on.

Other Rubric Design & Prompting assessments