MACRS - Multi-Agent Conversational Recommender

MACRS (Fang et al., 2024) is a conversational recommender system built from several cooperating LLM agents rather than one. What interested me isn’t recommendation as such but how it learns from failure mid-conversation and adapts to the user in real time.

The ideas worth keeping

• Signals. Browsing behaviour, response patterns and engagement all feed the agents’ view of the user.
• Two levels of reflection.
  • Strategy-level: when a recommendation fails (the user disengages, rejects it), work out what went wrong with the dialogue strategy and correct it.
  • Information-level: update the model of the user’s preferences from what they just said.
• A feedback loop: feedback → interpretation → adaptation → better decisions. It copes with dynamic environments and unexpected behaviour, and builds trust over a conversation.
• Simulate the user with another agent. A user-simulator agent holds hidden preferences and talks to the system, which makes it possible to test recovery from failed recommendations without real people.

How it was evaluated

• Success rate: did the conversation end with the right item?
• Hit Ratio@k: did the right item appear in the top k suggestions?
• Average turns: efficiency. Fewer is better, but not at the cost of asking nothing.

MACRS consistently beat single-agent systems, including ones trained with reinforcement learning.

Ablations as a design tool

The part I’d reuse is the ablation study. Remove one layer at a time and see what breaks:

• no information reflection → preferences never update;
• no strategy reflection → the system never learns from a bad turn;
• no multi-agent orchestration → one agent does everything, worse;
• no planner → no reasoning about strategy at all.

Each removal shows that component’s individual value. It’s a cheap way to justify (or delete) a box on your architecture diagram, and it echoes the evaluation advice in Agentic System Design Checklist.

The lesson

Optimise the system, not just the model, especially as token use rises. Reflection as a first-class step is the same move that makes Eureka work.