Glance

Abstract

A frozen open vision-language model (Qwen3-VL-4B, Apache-2.0, on a laptop) is asked typed questions about an image, and the answer is read from the logits of a single forward pass, not generated. For yes/no and pick-one questions this works with no setup. On photographs taken after every model’s release, with labels nobody on this project made, the open model is level with Gemini 3.1 Pro, Claude Opus 5 and GPT-5.6, and with the cheapest current models of OpenAI and Google. It gives the same answers as the same model writing JSON, faster (1.5 times on a full-size photograph, 2.4 to 6.1 times on small images and many questions), with probabilities. A yes/no about a full-size photograph takes 1.1 s, faster than 5 of the 6 hosted models, and is somewhat cheaper than the cheapest hosted model; no image leaves the machine. A 2B model of the same family does as well on yes/no. Ratings are the hard case: zero-shot every model gets the order right (rank agreement 0.93, within one level on 0.99 of images) and the exact level often wrong; the cheapest Google model leads there (0.763 against 0.669), and the open model needs 16 unlabeled images of the rubric to come level (0.758) and 32 labels to lead. Glance is the calibration and measurement harness around that readout; the readout itself is shared with other training-free tools and is not claimed as new.

1What it is

You hand a frozen open vision-language model an image and a typed question: is this true (yes/no), which one (pick one of a list), or where on this rubric (a rating). The model runs once. Nothing is generated: the answer is read from the scores the model gives to the allowed answers at the position where it would start writing, and those scores become probabilities. Several questions about one image share the cost of reading the image.

photo: a bold jumping spider, iNaturalist 402011619 (CC BY, antimatterbee)

"Is the main subject a spider or other arachnid?"  ->  P(yes) = 1.00        0.8 s
"Is the main subject a fish?"                      ->  P(yes) = 0.00        0.7 s
"What kind of organism is it?"  (10 options)       ->  arachnid, p = 1.00   1.3 s

A real logged example from the open 4B model on a laptop, with a second job sharing the GPU. Most answers are this decisive; the uncertain ones are where the probabilities earn their keep.

2Does it work? Every system, the same items

The same questions went to the open 4B model (read with Glance, and also writing its answer as JSON), to three hosted flagships and to each provider’s cheapest current vision model. Three tests, all zero-shot: yes/no, 131 fresh commons questions; pick one of 13, 65 fresh commons photos; rating, exact level of 4, 1,000 lab images. The photographs were taken after every model’s release and labelled by people outside this project (section 2.1).

Yes/no and pick-one: every system is level on accuracy. The open model answers a yes/no in 1.1 s, faster than 5 of the 6 hosted models; it costs a fraction of the flagships and is somewhat cheaper than the cheapest hosted model. Most of its time on a full-size photograph goes into reading the image, not into the answer. Ratings: there is no clear best model. Gemini 3.1 Flash-Lite, the cheapest Google model, leads on accuracy, each provider’s cheap model beats its own flagship, and the open 4B model sits between them. Section 4 explains why ratings behave differently and what fixes them.

Table 1. The numbers behind Figure 1. Accuracy with 95% bootstrap intervals on the items EVERY system answered (the test half of the Commons set: 131 yes/no questions, 65 pick-one photographs; section 2.1 gives the open model on all items); bold rows are the open model. The open model’s rating is read in one pass at the JSON answer position. Hosted speed is wall time per call from one laptop; hosted cost is the provider’s bill where it was logged (“est.” is a list-price upper estimate for a run that predates cost logging). Open-model speed is measured with the GPU otherwise idle; its cost is those seconds at an on-demand cloud GPU price. Cells marked “tonight” await a clean timing. No few-shot prompt was tried for any written row.

2.1 The photographs: nothing any model could have seen

Wikimedia Commons photographs taken after 15 August 2026, labelled by their uploaders’ structured “depicts” statements, and iNaturalist observations uploaded on the day of the test, labelled by community identification. No labels were made by us or by any model.

Table 2. All items of each photo set (the frontier models answered the test half), 95% bootstrap intervals, uncalibrated decisions, nothing fitted.

2.2 Two other views of the same numbers

3Why read the answer instead of writing it?

The obvious alternative is to let the same model write a JSON answer. On yes/no and pick-one the two give identical answers, item for item, so reading changes nothing about what the model knows. What it changes is everything around the answer: no tokens are generated, so it is faster and cheaper: about 1.5 times on one question about a full-size photograph (reading the image dominates), 2 to 3 times on small images, and more as questions per image grow; the answer arrives as probabilities, so you can threshold, abstain or rank (accuracy on the most confident 80% of answers is 0.975 on the iNaturalist set); and a read answer can be fitted, which section 4 needs.

Table 3. The open model writing against reading, cost per 1,000 images on a rented GPU assumed no faster than the laptop; self-hosted cost is GPU time, so the saving is the measured time saving. For comparison, the frontier calls on these tasks measured $0.10 to $7.64 per 1,000 answers.

4Ratings: the hard case

A rating asks for a level on a rubric written in words, here five synthetic four-level scales (blur, exposure, JPEG, noise, resolution) and the same 1,000 held-out images for every system. Zero-shot, every model gets the order right and the exact level often wrong.

The open model is exactly right on 0.56 of images but within one level on 0.987, and 97% of its errors are one step, in a direction that is constant per rubric: half a level too harsh on blur, never the worst level on JPEG. It sees the severity; it does not know where the rubric’s author drew the lines, and neither do the hosted models, which score 0.55 to 0.76. Read in one pass, the open model gives the same ratings as when it writes them (98% identical, 0.669 against 0.672).

4.1 What closes the gap

A written pick has nothing to fit. A read answer is a set of logits, so a few images of your own rubric can remove the constant offset, with or without labels. Sixteen unlabeled images bring the open model level with the best hosted model; 32 labels put it ahead of everything we measured.

Table 4. Exact-level accuracy on the rating test of Table 1 (the last row uses the full test split).

5Does it depend on the model?

The same questions and readouts, not a word changed, on other sizes of the same family and on a model from a different family (different vision tower, different language model).

Table 5. Yes/no and pick-one on the two fresh photo sets, all items, uncalibrated. Even the 2B model is level on yes/no; size shows on the harder pick-one.

Table 6. Ratings, the same 1,000 images: exact-level accuracy. Zero-shot quality belongs to the model; once a few dozen labels exist the models land within a few points of each other, so the fitted recipe carries across sizes and families. “Within one” is for the four-pass read.

6What is, and is not, new

Reading answer-token logits from a frozen generative model, with several questions sharing one image prefix, is what Simple Jev, jev-visual and LitJev also do. For yes/no and pick-one the forward pass here is not new, and the accuracy belongs to the open model. What this project adds is a harness on top: a fresh-photograph comparison with paid frontier calls, the same model writing against reading, self-calibration from unlabeled images and labeled fitting for rating levels (glance fit), and measured dollars and milliseconds. It trains no weights, unlike YOFO, Laya Vision or OpenJev v2.

7Limits and misses

• not supported A content-free prior (blank and noise images) was expected to help zero-shot ratings. It took exact accuracy from 0.558 to 0.400: for an image rubric there is no content-free image.
• not supported Our four-pass rating readout was expected to match the same model’s written answer zero-shot. It trailed it by ten points: a readout selected with a calibration in the loop is good to fit and poor zero-shot. supported The registered fix, one pass read at the JSON answer position, matches the written answer (0.669) and reaches 0.758 with 16 unlabeled images.
• not supported We expected each provider’s cheapest model to score at or below its flagship on ratings. Every one beats its flagship, and Gemini 3.1 Flash-Lite (0.763, $0.12 to $0.33 per 1,000 among the cheap models) is nine points ahead of the open model zero-shot. The open model needs 16 unlabeled images to draw level and 32 labels to lead.
• not supported On KADID-10k (23 distortion types, five levels, human scores) every registered target was missed: 0.527 exact with labels, 0.33 zero-shot.
• supported A fitted readout on the model’s hidden state reaches 0.965 from one pass, against 0.867 for the token readout: the model represents severity almost perfectly. It needs on the order of a hundred labels.
• known Hand-built image features beat the VLM on low-level artifacts when labels are plentiful (0.979). A calibration fitted on one rubric does not transfer to another. Two model families and two sizes measured so far (Tables 5 and 6); that is not “any model”.

8Reproduce

uv sync && uv run glance doctor
uv run python tools/fetch_fresh_inat.py          # 200 photographs, about ten API calls
uv run glance eval --suite inat_choice --suite inat_yesno --model vlm
uv run python tools/make_results_zeroshot.py     # every table on this page

The lab notebook records each hypothesis before its experiment, each verdict after, and two errata. Frontier model outputs are never stored; only whether each answer was right.