{"id":503,"date":"2025-08-28T12:00:18","date_gmt":"2025-08-28T18:12:54","guid":{"rendered":"https:\/\/blog.agentsarchitects.ai\/?p=503"},"modified":"2026-07-14T05:40:01","modified_gmt":"2026-07-14T05:40:01","slug":"thinking-with-visual-primitives-the-architectural-shift-shaping-the-future-of-vision-agents","status":"publish","type":"post","link":"https:\/\/blog.agentsarchitects.ai\/index.php\/2025\/08\/28\/thinking-with-visual-primitives-the-architectural-shift-shaping-the-future-of-vision-agents\/","title":{"rendered":"Thinking with Visual Primitives: The Architectural Shift Shaping the Future of Vision Agents"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"470\" src=\"https:\/\/blog.agentsarchitects.ai\/wp-content\/uploads\/2026\/07\/article1-1024x470.png\" alt=\"\" class=\"wp-image-504\" srcset=\"https:\/\/blog.agentsarchitects.ai\/wp-content\/uploads\/2026\/07\/article1-1024x470.png 1024w, https:\/\/blog.agentsarchitects.ai\/wp-content\/uploads\/2026\/07\/article1-300x138.png 300w, https:\/\/blog.agentsarchitects.ai\/wp-content\/uploads\/2026\/07\/article1-768x353.png 768w, https:\/\/blog.agentsarchitects.ai\/wp-content\/uploads\/2026\/07\/article1-1536x705.png 1536w, https:\/\/blog.agentsarchitects.ai\/wp-content\/uploads\/2026\/07\/article1-2048x940.png 2048w, https:\/\/blog.agentsarchitects.ai\/wp-content\/uploads\/2026\/07\/article1-980x450.png 980w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Thinking with Visual Primitives: The Architectural Shift That Could Define the Next Generation of Vision Agents<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The race to build more capable multimodal AI systems has largely focused on improving perception. Researchers have invested heavily in higher-resolution image processing, dynamic cropping strategies, visual zoom mechanisms, and advanced Chain-of-Thought techniques in an effort to help models &#8220;see&#8221; more effectively.<br>However, a new line of research suggests that perception may not be the primary limitation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The paper&nbsp;<strong>&#8220;Thinking with Visual Primitives&#8221;<\/strong>&nbsp;introduces a different perspective. Instead of focusing on the Perception Gap, the authors identify what they call the&nbsp;<strong>Reference Gap<\/strong>&nbsp;a fundamental challenge that arises when language attempts to reason about complex visual scenes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Understanding the Reference Gap<\/strong><br>In dense visual environments, language alone often struggles to maintain consistent references.<br>Consider:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A team photo containing dozens of people<\/li>\n\n\n\n<li>A maze with multiple intersections<\/li>\n\n\n\n<li>Complex overlapping charts and diagrams<\/li>\n\n\n\n<li>Crowded manufacturing inspection images<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Even when an AI model successfully perceives the visual elements, it may lose track of which object it is referring to during later reasoning steps.<br>According to the authors, this is not a perception problem it is a referencing problem.<br>Natural language was never designed to function as a precise pointer inside a continuous two-dimensional space.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>The Core Innovation: Visual Primitives as Units of Thought<\/strong><br>The central idea behind the research is remarkably simple yet powerful.<br>Instead of using bounding boxes and points only as verification tools after reasoning is completed, the model incorporates them directly into its reasoning process.<br><strong>The reasoning chain alternates between:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Natural language tokens<\/li>\n\n\n\n<li>Bounding boxes<\/li>\n\n\n\n<li>Spatial points<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Example representations include:<br>&lt;|box|&gt;[[x1,y1,x2,y2]]&lt;|\/box|&gt;<br>&lt;|point|&gt;[[x,y]]&lt;|\/point|&gt;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These coordinates exist within a normalized visual space, allowing the model to maintain precise references throughout the reasoning process.<br>This approach closely resembles how humans use pointing gestures when:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Counting objects<\/li>\n\n\n\n<li>Following a path<\/li>\n\n\n\n<li>Tracing diagrams<\/li>\n\n\n\n<li>Solving visual puzzles<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Architecture Highlights<\/strong><br>The proposed system is built on<strong>&nbsp;DeepSeek-V4-Flash,<\/strong>&nbsp;a large Mixture-of-Experts model featuring:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>284B total parameters<\/li>\n\n\n\n<li>Approximately 13B active parameters<\/li>\n\n\n\n<li>Compressed Sparse Attention architecture<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">One particularly interesting aspect is its visual token compression strategy.<br>For an 800\u00d7800 image, the model reportedly retains roughly:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Model&nbsp;&nbsp;&nbsp;<\/strong>&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;<strong>Approximate KV Entries<\/strong><br>DeepSeek-V4-Flash&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;~90<br>Qwen3-VL-235B-A22B&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;~660<br>GPT-5.4&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ~740<br>Gemini-3-Flash&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ~1100<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The paper reports an impressive&nbsp;<strong>7,056\u00d7 pixel-to-KV compression ratio,<\/strong>&nbsp;highlighting significant efficiency gains for large-scale deployment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Training Strategy<\/strong><br>The post-training pipeline uses a specialist-first approach.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Separate experts are trained for:<br><strong>Grounding Tasks<\/strong><br>Responsible for generating and reasoning with bounding boxes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pointing Tasks<\/strong><br>Focused on point-based visual references.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These experts are later merged through:<br>    1. Unified Reinforcement Fine-Tuning (RFT)<br>    2. On-Policy Distillation<br>    3. Reverse-KL Logit Distillation<br>This allows the final model to inherit strengths from both specialized systems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Advanced Reward Modeling<\/strong><br>One of the most technically impressive aspects of the paper is its reward design.<br>Beyond standard formatting and correctness rewards, the researchers introduce task-specific reward mechanisms.<br>Examples include:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Counting Tasks<\/strong><br>A smooth exponential-decay relative error reward that penalizes inaccurate counts proportionally.<br><br><strong>Maze Navigation<\/strong><br>Rewards consider:<br>    1. Exploration progress<br>    2. Coverage completeness<br>    3. Wall violation penalties<br>    4. Path validity<br>    5. Final answer correctness<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Path Tracing<\/strong><br>A bidirectional trajectory reward evaluates:&nbsp;<br>    1. Path accuracy<br>    2. Coverage completeness<br>    3. Deviation penalties<br>This level of reward engineering is often what separates robust systems from easily exploitable ones.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Benchmark Performance<\/strong><br>The model was evaluated on two newly introduced reasoning benchmarks:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>DS_Maze_Navigation<\/strong><br>Reported performance:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Frontier models: ~48\u201351%<\/li>\n\n\n\n<li>Thinking with Visual Primitives: 66.9%<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>DS_Path_Tracing<\/strong><br>Reported performance:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Existing models: 24.5\u201346.5%<\/li>\n\n\n\n<li>Thinking with Visual Primitives: 56.7%<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These results suggest meaningful improvements in tasks requiring persistent visual referencing and topological reasoning.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Important Caveats<\/strong><br>The authors are transparent about several limitations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>In-House Benchmarks<\/strong><br>The primary benchmarks were developed internally and still require broader industry validation and independent replication.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Limited Scope<\/strong><br>The reported scores focus specifically on visual reasoning capabilities and should not be interpreted as indicators of overall model intelligence.<br>This transparency reflects strong evaluation discipline and improves confidence in the reported findings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why This Matters for Enterprise AI<\/strong><br>For teams building AI-powered systems, the implications are significant.<br>Potential applications include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Document Intelligence<\/li>\n\n\n\n<li>Visual Inspection Systems<\/li>\n\n\n\n<li>Robotics Perception<\/li>\n\n\n\n<li>Industrial Quality Control<\/li>\n\n\n\n<li>Dashboard Analysis<\/li>\n\n\n\n<li>Autonomous Agent Workflows<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The research suggests that improving reference tracking may deliver greater performance gains than simply increasing image resolution or extending reasoning chains.<br>Additionally, the efficiency benefits from visual token compression could reduce deployment costs while maintaining strong performance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Current Limitations and Future Research<\/strong><br>The authors acknowledge several open challenges:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Fine-grained visual resolution limitations<\/li>\n\n\n\n<li>Dependence on explicit trigger mechanisms<\/li>\n\n\n\n<li>Limited cross-domain generalization<\/li>\n\n\n\n<li>Point-based reasoning constraints in unfamiliar environments<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These areas are likely to become major research directions for future multimodal systems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Final Thoughts<\/strong><br>&#8220;Thinking with Visual Primitives&#8221; presents a compelling shift in how we think about multimodal reasoning.<br>Rather than asking how AI models can see more, the research asks a more fundamental question:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How can AI models maintain accurate references while thinking about what they see?<\/strong><br>If future research validates these findings across broader benchmarks and real-world environments, visual primitives may become a foundational building block for the next generation of intelligent vision agents.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;<br><strong>References:<\/strong><br>Lu, R., Ma, Y., Chen, X., Luo, L., Wu, Z., Pan, Z., Liu, X., Lin, Y., Li, H., Liu, W., Hao, Z., Gao, X., Nie, S., Wei, Y., Xie, Z., Chen, T., &amp; Zeng, G. (2026). Thinking with Visual Primitives. DeepSeek-AI, Peking University, and Tsinghua University. Technical Report.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Available via Hugging Face:<\/strong><br><a href=\"https:\/\/huggingface.co\/datasets\/NodeLinker\/deepseek-ai-Thinking-with-Visual-Primitives-deleted-repo\">https:\/\/huggingface.co\/datasets\/NodeLinker\/deepseek-ai-Thinking-with-Visual-Primitives-deleted-repo<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Author Note:<\/strong><br>All technical claims, benchmark results, and architectural descriptions discussed in this article are summarized from the cited research paper. Analysis and interpretation reflect the author&#8217;s perspective.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Thinking with Visual Primitives: The Architectural Shift That Could Define the Next Generation of Vision Agents The race to build more capable multimodal AI systems has largely focused on improving perception. Researchers have invested&#46;&#46;&#46;<\/p>\n","protected":false},"author":1,"featured_media":505,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-503","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-agentic-ai"],"_links":{"self":[{"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/posts\/503","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/comments?post=503"}],"version-history":[{"count":2,"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/posts\/503\/revisions"}],"predecessor-version":[{"id":746,"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/posts\/503\/revisions\/746"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/media\/505"}],"wp:attachment":[{"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/media?parent=503"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/categories?post=503"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.agentsarchitects.ai\/index.php\/wp-json\/wp\/v2\/tags?post=503"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}