From aa7da915e3e6b41750bee3be875f87c2a4069887 Mon Sep 17 00:00:00 2001 From: Rodribm10 Date: Sun, 19 Apr 2026 11:30:19 -0300 Subject: [PATCH] fix(captain): remove scenario->orchestrator back-handoff (ping-pong) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Problema observado em teste real 2026-04-19 11:24: usuário forneceu suíte+data+hora pra Daniela. Em vez de chamar generate_pix, Daniela chamou handoff_to_jasmine. Jasmine respondeu "Vou te transferir pra Daniela..." — mentira, a conversa ficou parada com a Jasmine. Sequência dentro de UM único run: jasmine.handoff_to_daniela_reservas_agent -> daniela.handoff_to_jasmine (!) -> jasmine responde "vou te transferir..." O prompt da Daniela tem "🚨 NUNCA FAÇA HANDOFF DE VOLTA PRA JASMINE" mas o LLM ignora a proibição quando a ferramenta está registrada. A única solução robusta é não registrar a ferramenta. Historicamente tivemos medo de remover a back-edge porque sem ela a Daniela (quando confusa) ficava em loop chamando faq_lookup — incidente que queimou créditos reais. Esse medo não vale mais: commit f3f8a8d5c adicionou TOOL_LOOP_THRESHOLD=3 + MAX_TURNS_PER_MESSAGE=15 que disparam bot_handoff automático em qualquer loop de tool. A proteção contra runaway existe por OUTRA via agora, então podemos remover a back-edge com segurança. Efeito esperado: - scenario termina a resposta sozinho (sem ping-pong) - scenario confuso/em loop -> rate limit corta -> humano recebe Memory: atualizado feedback_never_touch_captain_without_safety_caps.md refletindo a nova invariante. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../captain/assistant/agent_runner_service.rb | 23 +++++++++++-------- .../assistant/agent_runner_service_spec.rb | 2 +- 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/enterprise/app/services/captain/assistant/agent_runner_service.rb b/enterprise/app/services/captain/assistant/agent_runner_service.rb index c9ac26023..9928f4a05 100644 --- a/enterprise/app/services/captain/assistant/agent_runner_service.rb +++ b/enterprise/app/services/captain/assistant/agent_runner_service.rb @@ -454,17 +454,20 @@ class Captain::Assistant::AgentRunnerService assistant_agent = build_orchestrator_agent_with_memory scenario_agents = @assistant.scenarios.enabled.map(&:agent) - # Bidirectional handoff: orchestrator -> scenarios AND scenarios -> orchestrator. - # Historical note: removing the back-edge looks attractive (prevents ping-pong) - # but in practice the scenario LLM uses handoff_to_orchestrator as a "fallback" - # when it gets confused. Without that fallback, the LLM keeps calling other - # available tools (faq_lookup, etc.) in a loop — observed real-world incident - # where Daniela called faq_lookup dozens of times in a runaway. Keep the edge. - # Ping-pong is instead contained by max_turns in generate_response AND by - # explicit prompt rules in the scenario instruction forbidding gratuitous - # handoffs. + # Unidirectional handoff: orchestrator -> scenarios only. + # + # Historically we also registered scenarios -> orchestrator as a safety + # valve so a confused scenario could escape to Jasmine. In practice this + # caused ping-pong INSIDE a single run: orchestrator hands off to Daniela, + # Daniela immediately hands back, Jasmine responds with "Vou te transferir + # para a Daniela" AFTER the user was already with Daniela. + # + # The runaway-loop fear that originally justified the back-edge (Daniela + # spamming faq_lookup when confused) is now contained by TOOL_LOOP_THRESHOLD + # / MAX_TURNS_PER_MESSAGE in generate_response — any repeated tool call or + # turn exhaustion triggers bot_handoff to a human. So the back-edge is no + # longer a required safety net, and removing it fixes the ping-pong. assistant_agent.register_handoffs(*scenario_agents) if scenario_agents.any? - scenario_agents.each { |scenario_agent| scenario_agent.register_handoffs(assistant_agent) } [assistant_agent] + scenario_agents end diff --git a/spec/enterprise/services/captain/assistant/agent_runner_service_spec.rb b/spec/enterprise/services/captain/assistant/agent_runner_service_spec.rb index 69b8513b7..f434c89a9 100644 --- a/spec/enterprise/services/captain/assistant/agent_runner_service_spec.rb +++ b/spec/enterprise/services/captain/assistant/agent_runner_service_spec.rb @@ -62,7 +62,7 @@ RSpec.describe Captain::Assistant::AgentRunnerService do expect(assistant).to receive(:scenarios).and_return(scenarios_relation) expect(scenario).to receive(:agent).and_return(mock_scenario_agent) expect(mock_agent).to receive(:register_handoffs).with(mock_scenario_agent) - expect(mock_scenario_agent).to receive(:register_handoffs).with(mock_agent) + expect(mock_scenario_agent).not_to receive(:register_handoffs) service.generate_response(message_history: message_history) end