Runaway incident: Daniela (reservation scenario) entered a tool-calling loop, invoking faq_lookup with the same query dozens of times per second, stuck at 'Performing' in Sidekiq for minutes with 1-of-12 busy. Root cause was two interacting factors: 1. The previous commit removed scenario_agent.register_handoffs( assistant_agent) to prevent ping-pong. In practice, the scenario LLM uses handoff_to_orchestrator as a safety valve when it cannot advance. Without it, the LLM kept calling other available tools (faq_lookup) indefinitely. 2. max_turns was 100. A runaway loop could burn 100 LLM + tool cycles before Sidekiq's timeout fired, which meant real token spend in a single bad turn could blow a day's budget. Both restored/fixed: - max_turns: 100 -> 15. Plenty for normal flows; hard ceiling on any runaway. The LLM simply ran out of turns and had to emit a final response instead of looping further. - scenario -> orchestrator handoff: re-registered. Ping-pong risk is contained by max_turns AND by explicit prompt rules in the scenario instruction forbidding gratuitous handoffs (added to Daniela prompt in earlier commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| app | ||
| config | ||
| lib | ||
| LICENSE | ||
| tasks_railtie.rb | ||