Multimodal Collaborative Agents for Next-Gen Commerce — Nidhi Kaushik Vyas, Google DeepMind
For fuzzy requests, an agent should identify the missing constraint with the most decision value, elicit it in the right modality, then choose a response format suited to the task.
The proposed loop builds a working state from conversation history, personal context, references, hard constraints, soft preferences, confidence, and live variables such as inventory. It then asks the **single highest-utility question**, such as room width before suggesting furniture.
Treat elicitation as an agent policy: identify blockers, avoid repeated questioning, and select **text or visual preference boards** according to whether the user can articulate the constraint. Map each answer into the catalog ontology before retrieval, then choose a response format such as a comparison table or inspiration images.
The proposed loop builds a working state from conversation history, personal context, references, hard constraints, soft preferences, confidence, and live variables such as inventory. It then asks the **single highest-utility question**, such as room width before suggesting furniture. Treat elicitation as an agent policy: identify blockers, avoid repeated questioning, and select **text or visual preference boards** according to whether the user can articulate the constraint. Map each answer into the catalog ontology before retrieval, then choose a response format such as a comparison table or inspiration images. This is a commerce-grounded framework, not reported performance evidence. Merchant ontologies remain important, and agent-to-agent shopping is described as **an early-stage possibility**, with direct user involvement still preferred during discovery.
This turns process-first agent design into a concrete discovery policy: maintain structured state, ask only the question with the greatest expected utility, normalize the answer into a merchant ontology, and then choose retrieval and presentation. It reinforces human-led context gathering while narrowing agent-to-agent commerce to an early possibility rather than a demonstrated replacement for user discovery.