fix(captain): remove scenario->orchestrator back-handoff (ping-pong)

Problema observado em teste real 2026-04-19 11:24:
usuário forneceu suíte+data+hora pra Daniela. Em vez de chamar
generate_pix, Daniela chamou handoff_to_jasmine. Jasmine respondeu
"Vou te transferir pra Daniela..." — mentira, a conversa ficou
parada com a Jasmine.

Sequência dentro de UM único run:
  jasmine.handoff_to_daniela_reservas_agent
  -> daniela.handoff_to_jasmine (!)
  -> jasmine responde "vou te transferir..."

O prompt da Daniela tem "🚨 NUNCA FAÇA HANDOFF DE VOLTA PRA JASMINE"
mas o LLM ignora a proibição quando a ferramenta está registrada.
A única solução robusta é não registrar a ferramenta.

Historicamente tivemos medo de remover a back-edge porque sem ela
a Daniela (quando confusa) ficava em loop chamando faq_lookup —
incidente que queimou créditos reais. Esse medo não vale mais:
commit f3f8a8d5c adicionou TOOL_LOOP_THRESHOLD=3 +
MAX_TURNS_PER_MESSAGE=15 que disparam bot_handoff automático em
qualquer loop de tool. A proteção contra runaway existe por
OUTRA via agora, então podemos remover a back-edge com segurança.

Efeito esperado:
- scenario termina a resposta sozinho (sem ping-pong)
- scenario confuso/em loop -> rate limit corta -> humano recebe

Memory: atualizado feedback_never_touch_captain_without_safety_caps.md
refletindo a nova invariante.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rodribm10 2026-04-19 11:30:19 -03:00
parent f3f8a8d5c1
commit aa7da915e3
2 changed files with 14 additions and 11 deletions

View File

@ -454,17 +454,20 @@ class Captain::Assistant::AgentRunnerService
assistant_agent = build_orchestrator_agent_with_memory
scenario_agents = @assistant.scenarios.enabled.map(&:agent)
# Bidirectional handoff: orchestrator -> scenarios AND scenarios -> orchestrator.
# Historical note: removing the back-edge looks attractive (prevents ping-pong)
# but in practice the scenario LLM uses handoff_to_orchestrator as a "fallback"
# when it gets confused. Without that fallback, the LLM keeps calling other
# available tools (faq_lookup, etc.) in a loop — observed real-world incident
# where Daniela called faq_lookup dozens of times in a runaway. Keep the edge.
# Ping-pong is instead contained by max_turns in generate_response AND by
# explicit prompt rules in the scenario instruction forbidding gratuitous
# handoffs.
# Unidirectional handoff: orchestrator -> scenarios only.
#
# Historically we also registered scenarios -> orchestrator as a safety
# valve so a confused scenario could escape to Jasmine. In practice this
# caused ping-pong INSIDE a single run: orchestrator hands off to Daniela,
# Daniela immediately hands back, Jasmine responds with "Vou te transferir
# para a Daniela" AFTER the user was already with Daniela.
#
# The runaway-loop fear that originally justified the back-edge (Daniela
# spamming faq_lookup when confused) is now contained by TOOL_LOOP_THRESHOLD
# / MAX_TURNS_PER_MESSAGE in generate_response — any repeated tool call or
# turn exhaustion triggers bot_handoff to a human. So the back-edge is no
# longer a required safety net, and removing it fixes the ping-pong.
assistant_agent.register_handoffs(*scenario_agents) if scenario_agents.any?
scenario_agents.each { |scenario_agent| scenario_agent.register_handoffs(assistant_agent) }
[assistant_agent] + scenario_agents
end

View File

@ -62,7 +62,7 @@ RSpec.describe Captain::Assistant::AgentRunnerService do
expect(assistant).to receive(:scenarios).and_return(scenarios_relation)
expect(scenario).to receive(:agent).and_return(mock_scenario_agent)
expect(mock_agent).to receive(:register_handoffs).with(mock_scenario_agent)
expect(mock_scenario_agent).to receive(:register_handoffs).with(mock_agent)
expect(mock_scenario_agent).not_to receive(:register_handoffs)
service.generate_response(message_history: message_history)
end