AnnotationPrep

Vision, OCR, Charts & UI Screenshot Audits practice questions

20 questions from the Multimodal vision, audio & video eval practice test.

Start practising

Vision, OCR, Charts & UI Screenshot Audits practice drill

10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.

Sample scenarios

PROMPT

Compare Q3 and Q4 uptime from the attached bar chart.

OBSERVATION

The chart's y-axis starts at 96.0% and ends at 100.0%. The Q3 bar reaches 97.0% (25% of plot height) and the Q4 bar reaches 99.0% (75% of plot height).

AI CLAIM

Q4 uptime tripled compared to Q3, jumping from 97.0% to 99.0%.

What multimodal chart-reading error should you flag?

  1. Unit conversion error: uptime percentages must be normalized into downtime minutes before comparing two quarters.
  2. Bar-width distortion: the model confused categorical bar spacing with a continuous quarterly time-series slope.
  3. Truncated y-axis misread: it compared pixel bar heights (3x taller) instead of real values (+2.0 percentage points).
  4. Baseline inversion error: the model read the bar values downward from the 100.0% ceiling rather than upward from the 96.0% floor.
Show the answer

C. Truncated y-axis misread: it compared pixel bar heights (3x taller) instead of real values (+2.0 percentage points).

Because the y-axis starts at 96.0% rather than 0%, a 2.0 percentage-point increase (97.0% to 99.0%) looks three times taller visually. Vision-language models frequently mistake pixel height ratios for value ratios; evaluators must flag this as a major chart-groundedness failure.

Principle: Chart Grounding: Truncated Y-Axis vs. Pixel-Height Bias

PROMPT

Which region accounts for the second-largest share of annual revenue in the attached pie chart?

OBSERVATION

The pie chart shows four unlabeled slices clockwise from 12 o'clock: Dark Blue (44%), Teal (28%), Amber (18%), and Slate Gray (10%). The legend lists: Dark Blue = North America, Slate Gray = APAC, Teal = EMEA, Amber = LATAM.

MODEL A

APAC is the second-largest region at 28%, following North America at 44%.

MODEL B

EMEA is the second-largest region at 28%, following North America at 44%.

What caused Model A's error?

  1. Clockwise ordering bug: Model A read the pie slices counterclockwise starting from the smallest Slate Gray wedge.
  2. Sum-to-100 hallucination: Model A redistributed 18% from LATAM into APAC because pie charts lack explicit gridlines.
  3. Percentage estimation drift: Model A misread the 18% Amber wedge as 28% due to perspective skew in the rendered chart.
  4. Legend color-mapping swap: Model A matched the second legend entry (APAC) to the second-largest slice (Teal / EMEA).
Show the answer

D. Legend color-mapping swap: Model A matched the second legend entry (APAC) to the second-largest slice (Teal / EMEA).

A classic vision-language failure on charts is pairing legend text by list position instead of binding each label to its actual color swatch. Here Slate Gray (APAC) appeared second in the legend, so Model A wrongly assigned it to the 28% Teal slice belonging to EMEA.

Principle: Chart Grounding: Color-Legend Binding Errors

PROMPT

Extract the exact grand total charged on the attached restaurant receipt image.

OBSERVATION

The thermal receipt prints: Subtotal $48.50, Sales Tax $3.88, Auto-Gratuity (18%) $8.73, Total $61.11. A faint crease runs through the decimal point in 61.11, though the subtotal, tax, and gratuity are crystal clear.

AI RESPONSE

The grand total charged on the receipt is $6,111.

How should an evaluator grade this OCR output?

  1. Major OCR failure: it dropped the decimal point without cross-checking line items that sum to $61.11.
  2. Acceptable transcription: thermal creases justify reporting literal unbroken digits whenever punctuation pixels appear faded.
  3. Minor formatting flaw only: omitting a decimal point in a receipt total does not affect truthfulness if all four digits match.
  4. Partial credit: the model should have summed only Subtotal + Sales Tax ($52.38) and excluded optional auto-gratuity lines.
Show the answer

A. Major OCR failure: it dropped the decimal point without cross-checking line items that sum to $61.11.

A dropped decimal changes $61.11 into $6,111—a 100x magnitude error that fails basic arithmetic consistency with the legible Subtotal ($48.50), Tax ($3.88), and Gratuity ($8.73). Strong multimodal evaluation expects both accurate OCR and arithmetic cross-checking against visible context.

Principle: OCR Auditing: Decimal Integrity & Arithmetic Cross-Checking

Other topics