# Pull Request Template
## Description
Playground now uses v2. It was only wired to use v1. Traces get `source:
playground` on langfuse when playground has been used.
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration.
locally and specs
<img width="1806" height="1276" alt="image"
src="https://github.com/user-attachments/assets/41ef4eb3-52b1-4b8e-9a4f-e8510c90cb39"
/>
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
# Pull Request Template
## Linear links:
-
https://linear.app/chatwoot/issue/CW-4479/if-image-is-sent-by-the-customer-send-it-to-openai
## Description
This pull request adds “Captain image support” to Chatwoot. It
introduces multimodal message handling so that when a customer sends an
image, Captain can forward the file to OpenAI’s vision endpoint,
generate a caption/analysis
## Type of change
Please delete options that are not relevant.
- [x] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
<img width="891" alt="image"
src="https://github.com/user-attachments/assets/c7cc98ed-cc44-4865-a53a-83d129e2fe2c"
/>
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Pranav <pranav@chatwoot.com>
This PR introduces the concept of a tool registry. The implementation is
straightforward: you can define a tool by creating a class with a
function name. The function name gets registered in the registry and can
be referenced during LLM calls. When the LLM invokes a tool using the
registered name, the registry locates and executes the appropriate tool.
If the LLM calls an unregistered tool, the registry returns an error
indicating that the tool is not defined.
Show captain messages under the name of the assistant which generated
the message.
- Add support for `Captain::Assistant` sender type
- Add push_event_data for captain_assistants
- Add activity message handler for captain_assistants
- Update UI to show captain messages under the name of the assistant
- Fix the issue where openAI errors when image is sent
- Add support for custom name of the assistant
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
This PR adds service to automate account abuse detection. Currently
based on the signup name and URL, could potentially add more context
such as usage analysis, message metadata etc.
This PR introduces a review step for generated FAQs, allowing a human to
validate and approve them before use in customer interactions. While
hallucinations are minimal, this step ensures accurate and reliable FAQs
for Captain to use during LLM calls when responding to customers.
- Added a status field for the FAQ
- Allow the filter on the UI.
<img width="1072" alt="Screenshot 2025-01-15 at 6 39 26 PM"
src="https://github.com/user-attachments/assets/81dfc038-31e9-40e6-8a09-586ebc4e8384"
/>
Migration Guide: https://chwt.app/v4/migration
This PR imports all the work related to Captain into the EE codebase. Captain represents the AI-based features in Chatwoot and includes the following key components:
- Assistant: An assistant has a persona, the product it would be trained on. At the moment, the data at which it is trained is from websites. Future integrations on Notion documents, PDF etc. This PR enables connecting an assistant to an inbox. The assistant would run the conversation every time before transferring it to an agent.
- Copilot for Agents: When an agent is supporting a customer, we will be able to offer additional help to lookup some data or fetch information from integrations etc via copilot.
- Conversation FAQ generator: When a conversation is resolved, the Captain integration would identify questions which were not in the knowledge base.
- CRM memory: Learns from the conversations and identifies important information about the contact.
---------
Co-authored-by: Vishnu Narayanan <vishnu@chatwoot.com>
Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
