fix(captain): remove scenario->orchestrator back-handoff (ping-pong)
Problema observado em teste real 2026-04-19 11:24:
usuário forneceu suíte+data+hora pra Daniela. Em vez de chamar
generate_pix, Daniela chamou handoff_to_jasmine. Jasmine respondeu
"Vou te transferir pra Daniela..." — mentira, a conversa ficou
parada com a Jasmine.
Sequência dentro de UM único run:
jasmine.handoff_to_daniela_reservas_agent
-> daniela.handoff_to_jasmine (!)
-> jasmine responde "vou te transferir..."
O prompt da Daniela tem "🚨 NUNCA FAÇA HANDOFF DE VOLTA PRA JASMINE"
mas o LLM ignora a proibição quando a ferramenta está registrada.
A única solução robusta é não registrar a ferramenta.
Historicamente tivemos medo de remover a back-edge porque sem ela
a Daniela (quando confusa) ficava em loop chamando faq_lookup —
incidente que queimou créditos reais. Esse medo não vale mais:
commit f3f8a8d5c adicionou TOOL_LOOP_THRESHOLD=3 +
MAX_TURNS_PER_MESSAGE=15 que disparam bot_handoff automático em
qualquer loop de tool. A proteção contra runaway existe por
OUTRA via agora, então podemos remover a back-edge com segurança.
Efeito esperado:
- scenario termina a resposta sozinho (sem ping-pong)
- scenario confuso/em loop -> rate limit corta -> humano recebe
Memory: atualizado feedback_never_touch_captain_without_safety_caps.md
refletindo a nova invariante.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
f3f8a8d5c1
commit
aa7da915e3
@ -454,17 +454,20 @@ class Captain::Assistant::AgentRunnerService
|
||||
assistant_agent = build_orchestrator_agent_with_memory
|
||||
scenario_agents = @assistant.scenarios.enabled.map(&:agent)
|
||||
|
||||
# Bidirectional handoff: orchestrator -> scenarios AND scenarios -> orchestrator.
|
||||
# Historical note: removing the back-edge looks attractive (prevents ping-pong)
|
||||
# but in practice the scenario LLM uses handoff_to_orchestrator as a "fallback"
|
||||
# when it gets confused. Without that fallback, the LLM keeps calling other
|
||||
# available tools (faq_lookup, etc.) in a loop — observed real-world incident
|
||||
# where Daniela called faq_lookup dozens of times in a runaway. Keep the edge.
|
||||
# Ping-pong is instead contained by max_turns in generate_response AND by
|
||||
# explicit prompt rules in the scenario instruction forbidding gratuitous
|
||||
# handoffs.
|
||||
# Unidirectional handoff: orchestrator -> scenarios only.
|
||||
#
|
||||
# Historically we also registered scenarios -> orchestrator as a safety
|
||||
# valve so a confused scenario could escape to Jasmine. In practice this
|
||||
# caused ping-pong INSIDE a single run: orchestrator hands off to Daniela,
|
||||
# Daniela immediately hands back, Jasmine responds with "Vou te transferir
|
||||
# para a Daniela" AFTER the user was already with Daniela.
|
||||
#
|
||||
# The runaway-loop fear that originally justified the back-edge (Daniela
|
||||
# spamming faq_lookup when confused) is now contained by TOOL_LOOP_THRESHOLD
|
||||
# / MAX_TURNS_PER_MESSAGE in generate_response — any repeated tool call or
|
||||
# turn exhaustion triggers bot_handoff to a human. So the back-edge is no
|
||||
# longer a required safety net, and removing it fixes the ping-pong.
|
||||
assistant_agent.register_handoffs(*scenario_agents) if scenario_agents.any?
|
||||
scenario_agents.each { |scenario_agent| scenario_agent.register_handoffs(assistant_agent) }
|
||||
|
||||
[assistant_agent] + scenario_agents
|
||||
end
|
||||
|
||||
@ -62,7 +62,7 @@ RSpec.describe Captain::Assistant::AgentRunnerService do
|
||||
expect(assistant).to receive(:scenarios).and_return(scenarios_relation)
|
||||
expect(scenario).to receive(:agent).and_return(mock_scenario_agent)
|
||||
expect(mock_agent).to receive(:register_handoffs).with(mock_scenario_agent)
|
||||
expect(mock_scenario_agent).to receive(:register_handoffs).with(mock_agent)
|
||||
expect(mock_scenario_agent).not_to receive(:register_handoffs)
|
||||
|
||||
service.generate_response(message_history: message_history)
|
||||
end
|
||||
|
||||
Loading…
Reference in New Issue
Block a user