Computer Use

MiniMax and Together describe why agent workloads change inference priorities: large codebase context, repeated tool turns, multimodality, and long-horizon RL all demand purpose-built environments and kernels.

OSReward finds that VLM judges often approve failed computer-use runs. Its benchmark and open reward models offer a more grounded way to evaluate trajectories without paying frontier-model costs.

Desktop-Delta Bench tests whether computer-use models can verify what changed after an action, exposing a reliability gap hidden by end-task and single-frame scores.

Persistent Codex workflows become more useful with reusable skills, memory, app-aware context, and scheduled thread check-ins—but computer use needs explicit boundaries and stopping rules.

OpenForgeRL trains agents inside their deployment harnesses by proxying model calls into standard RL stacks and isolating rollouts in Kubernetes containers; error recovery remains weak.

Perception agents share the rendered interface with users, turning visual annotations and design rules into agent input and checks. The open-source harness is an early attempt to close computer-use feedback loops.

Experimental on-device agents can play games and adapt interfaces without cloud calls, but real-time use must fit memory, frame-time, and battery budgets. Accessibility is promising, not production-ready.

PalmClaw runs the agent loop, memory, skills, and tools directly on a phone, exposing device capabilities as structured calls instead of GUI gestures. The paper reports faster, more reliable task execution.

A free, code-oriented Chinese curriculum spans model tuning, deployment, agents, alignment, security, and multimodal systems. It is useful as a broad learning map, but remains a work in progress.

Sonnet 5 lands June 30 at $2/$10 per Mtok intro pricing (through Aug 31), nearing Opus 4.8 on agentic coding and computer use. Note the new tokenizer: inputs map to 1.0–1.35x more tokens.

Gemini Spark lands on macOS (US, AI Ultra beta) and gains custom MCP support plus connectors for Tasks, Keep, Canva and Dropbox — Google's assistant now speaks the protocol your agent tooling already uses.

Alibaba's page-agent is an MIT-licensed JS library that embeds a natural-language GUI agent in any webpage via one script tag. It drives the DOM as text — no extension, headless browser, or multimodal model needed.

Google's June roundup: Gemma 4 12B runs locally in 16GB of memory, Gemini 3.5 Flash adds computer use for desktop, mobile, and browser agents, and Nano Banana 2 Lite ships as a cheaper image model.

Chrome DevTools MCP gives coding agents direct access to browser automation, network data, console errors, screenshots, and performance traces, with privacy controls worth setting explicitly.

ego lite lets Codex, Claude Code, and other agents automate logged-in web sessions in isolated browser spaces without taking over your active tabs. It is macOS-only today.