be8205657e
- Fixed Firecrawl webhook payloads to ensure proper data handling and delivery. - Removed unused Robin AI code to improve codebase cleanliness and maintainability. - Implement authentication for the Firecrawl endpoint to improve security. A key is generated to secure the webhook URLs from FireCrawl. --------- Co-authored-by: Pranav <pranavrajs@gmail.com>
21 lines
519 B
Ruby
21 lines
519 B
Ruby
class Captain::Tools::FirecrawlParserJob < ApplicationJob
|
|
queue_as :low
|
|
|
|
def perform(assistant_id:, payload:)
|
|
assistant = Captain::Assistant.find(assistant_id)
|
|
metadata = payload[:metadata]
|
|
|
|
document = assistant.documents.find_or_initialize_by(
|
|
external_link: metadata['url']
|
|
)
|
|
|
|
document.update!(
|
|
content: payload[:markdown],
|
|
name: metadata['title'],
|
|
status: :available
|
|
)
|
|
rescue StandardError => e
|
|
raise "Failed to parse FireCrawl data: #{e.message}"
|
|
end
|
|
end
|