7f56cd92f8
There were customer reported issues with FAQs which were generated in a different langauge than what they were expecting. The reason behind this was that the language of the account was not considered in the prompt provided. If the language of the content was say Spanish, and the account locale was english. The output was not predicable. The output depends on the model and the execution time. This PR would update the prompt to behave consistently with the account locale. Even though the content provided is in a different language, it would generate FAQs in the account locale. Changes: - Updated the prompt to include a detailed expectation of the FAQs quality along with the language - Added specs for the services where the prompt generator is called. Tested the prompt using Phoenix playground across GPT 5, GPT 4.1, GPT 4.0. The reasoning setting for GPT 5 needs to be low so that it doesn't generate random questions like "What was this updated?"
48 lines
1.1 KiB
Ruby
48 lines
1.1 KiB
Ruby
class Captain::Llm::FaqGeneratorService < Llm::BaseOpenAiService
|
|
def initialize(content, language = 'english')
|
|
super()
|
|
@language = language
|
|
@content = content
|
|
end
|
|
|
|
def generate
|
|
response = @client.chat(parameters: chat_parameters)
|
|
parse_response(response)
|
|
rescue OpenAI::Error => e
|
|
Rails.logger.error "OpenAI API Error: #{e.message}"
|
|
[]
|
|
end
|
|
|
|
private
|
|
|
|
attr_reader :content, :language
|
|
|
|
def chat_parameters
|
|
prompt = Captain::Llm::SystemPromptsService.faq_generator(language)
|
|
{
|
|
model: @model,
|
|
response_format: { type: 'json_object' },
|
|
messages: [
|
|
{
|
|
role: 'system',
|
|
content: prompt
|
|
},
|
|
{
|
|
role: 'user',
|
|
content: content
|
|
}
|
|
]
|
|
}
|
|
end
|
|
|
|
def parse_response(response)
|
|
content = response.dig('choices', 0, 'message', 'content')
|
|
return [] if content.nil?
|
|
|
|
JSON.parse(content.strip).fetch('faqs', [])
|
|
rescue JSON::ParserError => e
|
|
Rails.logger.error "Error in parsing GPT processed response: #{e.message}"
|
|
[]
|
|
end
|
|
end
|