Visual Jev: Accurate and Efficient Decisions from Shared Visual Context
Authors: Guanxu Yu, Yuhang Yao
$P(c_k \mid I, X_{\text{context}}, Q) = \frac{\exp(s_k / \tau)}{\sum_{j=1}^K \exp(s_j / \tau)}, \quad s_k = \mathbf{z}_{t_k}$
Key technical innovations in Visual Jev:
1. **Shared Visual Prefix Caching**: Encodes the image and shared context once into Key-Value tensors pinned in GPU VRAM, allowing multiple suffix questions to attend over the same visual memory.
2. **Batched Suffix Forward Pass**: Evaluates all candidate questions in parallel across batch dimensions without cross-suffix interference.
3. **Direct Logit Probability Extraction**: Reads normalized candidate likelihoods from the unmasked vocabulary head at the final choice token, bypassing text generation loops.
4. **Answer-Supervised Post-Training**: Calibrates decision logits with cross-entropy loss, raising equal-weight macro accuracy from 70.6% to 76.1% across MMBench, SEED-Bench, and RealWorldQA.
Frequently Asked Questions
How fast is Visual Jev compared to serial VLM execution?
Visual Jev runs 8.9 times faster in warm amortized time for batches of 32 questions per image, cutting total runtime from 1,930 milliseconds to 84 milliseconds on an RTX 4090.
Does Visual Jev generate conversational text explanations?
No. Visual Jev outputs calibrated probability distributions and structured categorical choices without generating intermediate prose.