e84875ca04
382 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
gabrieljablonski
|
e84875ca04 | test: update outgoing message spec to reflect nil sender from WhatsApp | ||
|
Gabriel Jablonski
|
4483b7457a
|
test: fix ci (#203)
* fix: update merge method to deep_merge for scheduled message metadata and adjust error handling in WhatsApp service specs * fix: update error expectation syntax in WhatsappZapiService specs * fix: update due_for_sending expectation to compare message IDs |
||
|
Cayo P. R. Oliveira
|
f9d1146cb0
|
feat: mensagens agendadas (#198)
* feat: Adds model for scheduling messages * feat: Implement scheduled message handling and processing jobs * feat: Add ScheduledMessagesController and associated specs for managing scheduled messages * refactor: Simplify scheduled message job specs and improve metadata handling * feat: Add ScheduledMessagePolicy for managing access to scheduled messages * feat: Add routes for managing scheduled messages * feat: Add scheduled message event handling and broadcasting * feat: Add JSON views for scheduled messages creation, destruction, updating, and indexing * feat: Update scheduled message status and dispatch update event after message creation * feat: Ensure scheduled message updates trigger dispatch event * feat: Add mutation types for managing scheduled messages * feat: Add additionalAttributes prop to Message component and provider * feat: Implement scheduled message handling in ActionCable and Vuex store * feat: Add unit tests for scheduled messages actions and mutations * feat: implement scheduled messages functionality - Added support for scheduling messages in the conversation dashboard. - Introduced new components: ScheduledMessageModal and ScheduledMessages for managing scheduled messages. - Enhanced ReplyBottomPanel to include scheduling options. - Updated Base.vue to handle scheduled message styling. - Integrated Vuex store module for managing scheduled messages state. - Added necessary translations for scheduled messages in English and Portuguese. * feat: add pagination to scheduled messages index and update tests accordingly * chore: update scheduled messages specs for future time validation and response status * chore: enhance scheduled messages API with pagination and add skeleton loader component * feat: add create_scheduled_message action to automation rule attributes * feat: implement create_scheduled_message action and enhance attachment handling * feat: add scheduled message functionality with UI components and localization * test: enhance scheduledMessages mutations tests with meta handling and structure * chore: update label to display file name upon successful upload in AutomationFileInput component * feat: add initialAttachment prop to ScheduledMessageModal and update ReplyBox to pass attachment * chore: prepend_mod_with to ScheduledMessagesController for better module handling * fix: attachment visibility in ScheduledMessageItem component * chore: enhance ScheduledMessage model with validations and reduce controller load * refactor: simplify ScheduledMessagesAPI methods by removing unnecessary instance variable * chore: update event emission for scheduled message creation in ReplyBox and ScheduledMessageModal * refactor: update status configuration to use label keys * chore: update date formatting in ScheduledMessageItem component * refactor: collapse logic to checkOverflow and update related functionality * chore: add author indication for current user in scheduled messages * chore: enhance scheduled message metadata with author information and localization * fix: send message shortcut * chore: handle errors in scheduled message submission * chore: update scheduled message modal to use combined date and time input * chore: refactor scheduled messages handling to remove pagination and update related tests * fix: ensure scheduled messages update status and dispatch on failure * fix: update scheduled message due date logic and simplify sending checks * refactor: rename build_message method for send_message * fix: update scheduled message creation time and improve test reliability * chore: ignore unnecessary check * chore: add scheduled message metadata handling in message builder, add scheduled message factorie and update specs * refactor: use scheduled message factorie creation in specs * chore: streamline error handling in scheduled message job and remove dispatch logic * fix: change scheduled_messages association to destroy dependent records * refactor: remove unused attributes from scheduled message payload builder * chore: update scheduled message retrieval to use conversation association * chore: correct cron format for scheduled messages job * chore: remove migration for author_type in scheduled_messages * feat: enhance scheduled messages management with delete confirmation and error handling * chore: set cron poll interval to 10 seconds for improved scheduling precision * feat: include additional_attributes in message JSON response * feat: enhance scheduled message validation and localization support * chore: update scheduled message display * Merge branch 'main' into Cayo-Oliveira/CU-86aenh268/Mensagens-agendadas * feat: add scheduled message indicators and validation for message length * fix: remove unnecessary condition from line-clamp class binding * feat: update scheduled messages localization and enhance content validation * feat: update scheduled messages order, enhance scheduledAt computation, and add message association * fix: reorder condition for Facebook channel message length computation * fix: change detection for attachments in scheduled messages * fix: remove unnecessary colon from close-on-backdrop-click prop in ScheduledMessageModal * chore: add error handling for scheduled message deletion and update localization for delete failure * fix: enforce minimum delay of 1 minute for scheduled messages and update validation * fix: remove unused private property and improve locale formatting for scheduled messages * fix: adjust positioning of DropdownBody in ReplyBottomPanel and clean up schema foreign keys * docs: add scheduled messages management APIs and payload definitions --------- Co-authored-by: gabrieljablonski <contact@gabrieljablonski.com> |
||
|
Gabriel Jablonski
|
b1aaf58097
|
fix(whatsapp): update conversation lookup to handle multiple contact inboxes in single conversation mode (#197)
* fix(whatsapp): update conversation lookup to handle multiple contact inboxes in single conversation mode * fix(whatsapp): update conversation retrieval logic to ensure correct conversation is reopened |
||
|
Gabriel Jablonski
|
b5d3250a2a
|
feat(baileys): add reply context handling (#196) | ||
|
Gabriel Jablonski
|
77c90a69ca
|
feat(whatsapp): delete messages on baileys/zapi providers (#194)
* feat(baileys): implement message deletion functionality * feat(zapi): add message deletion functionality and corresponding tests * feat(whatsapp): update message deletion logic for provider compatibility * feat(whatsapp): enhance message deletion logic to handle missing phone numbers |
||
|
Gabriel Jablonski
|
ec8366aabd
|
fix(whatsapp): fix conversation duplication on race condition (#193)
* fix(whatsapp): update message source ID handling and improve Redis key formatting * fix(whatsapp): prevent updating contact_inbox on identifier conflict * fix(whatsapp): refactor conversation routing tests to use shared examples for better clarity and maintainability |
||
|
Muhsin Keloth
|
990201f697 |
fix: Remove phone_number_id param from WhatsApp media retrieval for incoming messages (#13319)
Fixes https://github.com/chatwoot/chatwoot/issues/13317 Fixes an issue where WhatsApp attachment messages (images, audio, video, documents) were failing to download. Messages were being created but without attachments. The `phone_number_id` parameter was being passed to the `GET /<MEDIA_ID>` endpoint when downloading incoming media. According to Meta's documentation: > "Note that `phone_number_id` is optional. If included, the request will only be processed if the business phone number ID included in the query matches the ID of the business phone number **that the media was uploaded on**." For incoming messages, media is uploaded by the customer, not by the business phone number. Passing the business's `phone_number_id` causes validation to fail with error: `Param phone_number_id is not a valid whatsapp business phone number id ID` This PR removes the `phone_number_id` parameter from the media URL request for incoming messages. |
||
|
gabrieljablonski
|
f23fbb97f5 | test: comment out test for audio attachment due to transcoding requirement | ||
|
Gabriel Jablonski
|
b199c2c786
|
fix: whatsapp race condition (#190)
* fix: whatsapp race condition * fix: prevent race conditions in WhatsApp message handling * fix: improve WhatsApp contact lock handling to prevent race conditions |
||
|
Gabriel Jablonski
|
502f7a470f
|
test: fix ci error (#189) | ||
|
gabrieljablonski
|
6ab1898992 | Merge branch 'main' into chore/merge-upstream-4.10 | ||
|
Muhsin Keloth
|
96b5780ea7
|
fix: Respect survey label rules for WhatsApp CSAT template (#13285)
Ensure CSAT survey label rules are evaluated once in CsatSurveyService before any channel-specific sending (including WhatsApp/Twilio templates), remove the duplicated rule check from the template builder, and cover the blocking-label scenario in service specs while simplifying the template specs accordingly. Co-authored-by: Sojan Jose <sojan@pepalo.com> |
||
|
Gabriel Jablonski
|
24d348cdd3
|
feat: whastapp cloud provider voice notes (#186) | ||
|
Gabriel Jablonski
|
f54d113571
|
fix: whatsapp race condition (#185)
* feat: include external created at in whatsapp cloud message * fix: whatsapp providers race condition * refactor: remove redundant comments and streamline message processing lock acquisition * fix: scope message source key by inbox.id to prevent race condition * fix: scope message source key by inbox.id to prevent race condition |
||
|
Muhsin Keloth
|
c483034a07
|
feat: Add support for sending CSAT surveys via templates (Whatsapp Twilio) (#13143)
Fixes https://linear.app/chatwoot/issue/CW-6189/support-for-sending-csat-surveys-via-approved-whatsapp --------- Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com> Co-authored-by: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com> Co-authored-by: iamsivin <iamsivin@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Shivam Mishra
|
7b51939f07
|
fix: country_code should be checked against the contact (#13186)
|
||
|
Vinay Keerthi
|
59cbf57e20
|
feat: Advanced Search Backend (#12917)
## Description
Implements comprehensive search functionality with advanced filtering
capabilities for Chatwoot (Linear: CW-5956).
This PR adds:
1. **Time-based filtering** for contacts and conversations (SQL-based
search)
2. **Advanced message search** with multiple filters
(OpenSearch/Elasticsearch-based)
- **`from` filter**: Filter messages by sender (format: `contact:42` or
`agent:5`)
- **`inbox_id` filter**: Filter messages by specific inbox
- **Time range filters**: Filter messages using `since` and `until`
parameters (Unix timestamps in seconds)
- **90-day limit enforcement**: Automatically limits searches to the
last 90 days to prevent performance issues
The implementation extends the existing `Enterprise::SearchService`
module for advanced features and adds time filtering to the base
`SearchService` for SQL-based searches.
## API Documentation
### Base URL
All search endpoints follow this pattern:
```
GET /api/v1/accounts/{account_id}/search/{resource}
```
### Authentication
All requests require authentication headers:
```
api_access_token: YOUR_ACCESS_TOKEN
```
---
## 1. Search All Resources
**Endpoint:** `GET /api/v1/accounts/{account_id}/search`
Returns results from all searchable resources (contacts, conversations,
messages, articles).
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp (contacts/conversations only) | No
|
| `until` | integer | Unix timestamp (contacts/conversations only) | No
|
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search?q=customer" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [...],
"conversations": [...],
"messages": [...],
"articles": [...]
}
}
```
---
## 2. Search Contacts
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/contacts`
Search contacts by name, email, phone number, or identifier with
optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active in the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search contacts active between 30 and 7 days ago:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"contacts": [
{
"id": 42,
"email": "john@example.com",
"name": "John Doe",
"phone_number": "+1234567890",
"identifier": "user_123",
"additional_attributes": {},
"created_at": 1701234567
}
]
}
}
```
---
## 3. Search Conversations
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/conversations`
Search conversations by display ID, contact name, email, phone number,
or identifier with optional time filtering.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `since` | integer | Unix timestamp - filter by last_activity_at | No |
| `until` | integer | Unix timestamp - filter by last_activity_at | No |
### Example Requests
**Basic search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations active in the last 24 hours:**
```bash
SINCE=$(date -v-1d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search conversations from last month:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"conversations": [
{
"id": 123,
"display_id": 45,
"inbox_id": 1,
"status": "open",
"messages": [...],
"meta": {...}
}
]
}
}
```
---
## 4. Search Messages (Advanced)
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/messages`
Advanced message search with multiple filters powered by
OpenSearch/Elasticsearch.
### Prerequisites
- OpenSearch/Elasticsearch must be running (`OPENSEARCH_URL` env var
configured)
- Account must have `advanced_search` feature flag enabled
- Messages must be indexed in OpenSearch
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
| `from` | string | Filter by sender: `contact:{id}` or `agent:{id}` |
No |
| `inbox_id` | integer | Filter by specific inbox ID | No |
| `since` | integer | Unix timestamp - searches from this time (max 90
days ago) | No |
| `until` | integer | Unix timestamp - searches until this time | No |
### Important Notes
- **90-Day Limit**: If `since` is not provided, searches default to the
last 90 days
- If `since` exceeds 90 days, returns `422` error: "Search is limited to
the last 90 days"
- All time filters use message `created_at` timestamp
### Example Requests
**Basic message search:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific contact:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from a specific agent:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=agent:5" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages in a specific inbox:**
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&inbox_id=3" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages from the last 7 days:**
```bash
SINCE=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Search messages between specific dates:**
```bash
SINCE=$(date -v-30d +%s)
UNTIL=$(date -v-7d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}&until=${UNTIL}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Combine all filters:**
```bash
SINCE=$(date -v-14d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42&inbox_id=3&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
**Attempt to search beyond 90 days (returns error):**
```bash
SINCE=$(date -v-120d +%s)
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response (Success)
```json
{
"payload": {
"messages": [
{
"id": 789,
"content": "I need a refund for my purchase",
"message_type": "incoming",
"created_at": 1701234567,
"conversation_id": 123,
"inbox_id": 3,
"sender": {
"id": 42,
"type": "contact"
}
}
]
}
}
```
### Example Response (90-day limit exceeded)
```json
{
"error": "Search is limited to the last 90 days"
}
```
**Status Code:** `422 Unprocessable Entity`
---
## 5. Search Articles
**Endpoint:** `GET /api/v1/accounts/{account_id}/search/articles`
Search help center articles by title or content.
### Parameters
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `q` | string | Search query | Yes |
| `page` | integer | Page number (15 items per page) | No |
### Example Request
```bash
curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/articles?q=installation" \
-H "api_access_token: YOUR_ACCESS_TOKEN"
```
### Example Response
```json
{
"payload": {
"articles": [
{
"id": 456,
"title": "Installation Guide",
"slug": "installation-guide",
"portal_slug": "help",
"account_id": 1,
"category_name": "Getting Started",
"status": "published",
"updated_at": 1701234567
}
]
}
}
```
---
## Technical Implementation
### SQL-Based Search (Contacts, Conversations, Articles)
- Uses PostgreSQL `ILIKE` queries by default
- Optional GIN index support via `search_with_gin` feature flag for
better performance
- Time filtering uses `last_activity_at` for contacts/conversations
- Returns paginated results (15 per page)
### Advanced Search (Messages)
- Powered by OpenSearch/Elasticsearch via Searchkick gem
- Requires `OPENSEARCH_URL` environment variable
- Requires `advanced_search` account feature flag
- Enforces 90-day lookback limit via
`Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS`
- Validates inbox access permissions before filtering
- Returns paginated results (15 per page)
---
## Type of change
- [x] New feature (non-breaking change which adds functionality)
- [x] Enhancement (improves existing functionality)
---
## How Has This Been Tested?
### Unit Tests
- **Contact Search Tests**: 3 new test cases for time filtering
(`since`, `until`, combined)
- **Conversation Search Tests**: 3 new test cases for time filtering
- **Message Search Tests**: 10+ test cases covering:
- Individual filters (`from`, `inbox_id`, time range)
- Combined filters
- Permission validation for inbox access
- Feature flag checks
- 90-day limit enforcement
- Error handling for exceeded time limits
### Test Commands
```bash
# Run all search controller tests
bundle exec rspec spec/controllers/api/v1/accounts/search_controller_spec.rb
# Run search service tests (includes enterprise specs)
bundle exec rspec spec/services/search_service_spec.rb
```
### Manual Testing Setup
A rake task is provided to create 50,000 test messages across multiple
inboxes:
```bash
# 1. Create test data
bundle exec rake search:setup_test_data
# 2. Start OpenSearch
mise elasticsearch-start
# 3. Reindex messages
rails runner "Message.search_index.import Message.all"
# 4. Enable feature flag
rails runner "Account.first.enable_features('advanced_search')"
# 5. Test via API or Rails console
```
---
## Checklist
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [x] I have made corresponding changes to the documentation (this PR
description)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---
## Additional Notes
### Requirements
- **OpenSearch/Elasticsearch**: Required for advanced message search
- Set `OPENSEARCH_URL` environment variable
- Example: `export OPENSEARCH_URL=http://localhost:9200`
- **Feature Flags**:
- `advanced_search`: Account-level flag for message advanced search
- `search_with_gin` (optional): Account-level flag for GIN-based SQL
search
### Performance Considerations
- 90-day limit prevents expensive long-range queries on large datasets
- GIN indexes recommended for high-volume search on SQL-based resources
- OpenSearch/Elasticsearch provides faster full-text search for messages
### Breaking Changes
- None. All new parameters are optional and backward compatible.
### Frontend Integration
- Frontend PR tracking advanced search UI will consume these endpoints
- Time range pickers should convert JavaScript `Date` to Unix timestamps
(seconds)
- Date conversion: `Math.floor(date.getTime() / 1000)`
### Error Handling
- Invalid `from` parameter format is silently ignored (filter not
applied)
- Time range exceeding 90 days returns `422` with error message
- Missing `q` parameter returns `422` (existing behavior)
- Unauthorized inbox access is filtered out (no error, just excluded
from results)
---------
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
|
||
|
Muhsin Keloth
|
3e5b2979eb
|
feat: Add support for sending CSAT surveys via templates (Whatsapp Cloud) (#12787)
This PR enables sending CSAT surveys on WhatsApp using approved WhatsApp message templates, ensuring survey delivery even after the 24-hour session window. The system now automatically creates, updates, and monitors WhatsApp CSAT templates without manual intervention. <img width="1664" height="1792" alt="approved" src="https://github.com/user-attachments/assets/c6efd61e-1d01-4738-abb6-0afc0dace975" /> #### Why this change Previously, WhatsApp CSAT messages failed outside the 24-hour customer window. With this update: - CSAT surveys are delivered reliably using WhatsApp templates - Template creation happens automatically in the background - Users can modify survey content and recreate templates easily - Clear UI states show template approval status #### Screens & States <details> <summary>Default — No template configured yet</summary> <img width="1662" height="1788" alt="default" src="https://github.com/user-attachments/assets/ed26d71b-cf7c-4a26-a2af-da88772c847c" /> </details> <details> <summary>Pending — Template submitted, awaiting Meta approval</summary> <img width="1658" height="1816" alt="pending" src="https://github.com/user-attachments/assets/923b789b-d91b-4364-905d-e56a2b65331a" /> </details> <details> <summary>Approved — Survey will be sent when conversation resolves</summary> <img width="1664" height="1792" alt="approved" src="https://github.com/user-attachments/assets/c6efd61e-1d01-4738-abb6-0afc0dace975" /> </details> <details> <summary>Rejected — Template rejected by Meta</summary> <img width="1672" height="1776" alt="rejected" src="https://github.com/user-attachments/assets/f69a9b0e-be27-4e67-a993-7b8149502c4f" /> </details> <details> <summary>Not Found — Template missing in Meta Platform</summary> <img width="1660" height="1784" alt="not-exist" src="https://github.com/user-attachments/assets/a2a4b4f7-b01a-4424-8fcb-3ed84256e057" /> </details> <details> <summary>Edit Template — Delete & recreate template on change</summary> <img width="2342" height="1778" alt="edit-survey" src="https://github.com/user-attachments/assets/0f999285-0341-4226-84e9-31f0c6446924" /> </details> #### Test Cases **1. First-time CSAT setup on WhatsApp inbox** - Enable CSAT - Enter message + button text - Save - Expected: Template created automatically, UI shows pending state **2. CSAT toggle without changing text** - Existing approved template - Toggle CSAT OFF → ON (no text change) - Expected: No confirmation alert, no template recreation **3. Editing only survey rules** - Modify labels or rule conditions only - Expected: No confirmation alert, template remains unchanged **4. Template text change** - Change survey message or button text - Save - Expected: - Confirmation dialog shown - On confirm → previous template deleted, new one created - On cancel → revert to previous values **5. Language change** - Change template language (e.g., en → es) - Expected: Confirmation dialog + new template on confirm **6. Sending survey** - Template approved → always send template - Template pending → send free-form within 24 hours only - Template rejected/missing → fallback to free-form (if within window) - Outside 24 hours & no approved template → activity log only **7. Non-WhatsApp inbox** - Enable CSAT for email/web inbox - Expected: No template logic triggered Fixes https://linear.app/chatwoot/issue/CW-6188/support-for-sending-csat-surveys-via-approved-whatsapp --------- Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com> Co-authored-by: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com> Co-authored-by: iamsivin <iamsivin@gmail.com> |
||
|
Muhsin Keloth
|
bd698cb12c
|
feat: Add call-to-action template support for Twilio (#13179)
Fixes https://linear.app/chatwoot/issue/CW-6228/add-call-to-action-template-support-for-twilio-whatsapp-templates Adds support for Twilio WhatsApp call-to-action templates, enabling customers to use URL button templates with variable inputs. <img width="2982" height="1388" alt="CleanShot 2026-01-05 at 16 25 55@2x" src="https://github.com/user-attachments/assets/7cf332f5-3f3e-4ffb-a461-71c60a0156c8" /> |
||
|
Pranav
|
79381a4c5f
|
fix: Add code_block method to WhatsApp and Instagram markdown renderers (#13166)
Problem: SystemStackError: stack level too deep occurred when rendering
messages with indented content (4+ spaces) for WhatsApp, Instagram, and
Facebook channels.
Root Cause: CommonMarker::Renderer#code_block contains a self-recursive
placeholder that must be overridden:
```
def code_block(node)
code_block(node) # calls itself infinitely
end
```
WhatsAppRenderer and InstagramRenderer were missing this override,
causing infinite recursion when markdown with 4-space indentation
(interpreted as code blocks) was rendered.
Fix: Added code_block method to both renderers that outputs the node
content as plain text:
```
def code_block(node)
out(node.string_content)
end
```
Fix https://linear.app/chatwoot/issue/CW-6217/systemstackerror-stack-level-too-deep-systemstackerror
|
||
|
gabrieljablonski
|
549214e96d | Merge branch main into chore/merge-upstream | ||
|
Sojan Jose
|
7314c279ee
|
test(leadsquared): make ApiError specs reload-safe (#13098)
- fix the flaky lead-squared spec |
||
|
Mazen Khalil
|
ca5e112a8c
|
feat: TikTok channel (#12741)
fixes: #11834 This pull request introduces TikTok channel integration, enabling users to connect and manage TikTok business accounts similarly to other supported social channels. The changes span backend API endpoints, authentication helpers, webhook handling, configuration, and frontend components to support TikTok as a first-class channel. **Key Notes** * This integration is only compatible with TikTok Business Accounts * Special permissions are required to access the TikTok [Business Messaging API](https://business-api.tiktok.com/portal/docs?id=1832183871604753). * The Business Messaging API is region-restricted and is currently unavailable to users in the EU. * Only TEXT, IMAGE, and POST_SHARE messages are currently supported due to limitations in the TikTok Business Messaging API * A message will be successfully sent only if it contains text alone or one image attachment. Messages with multiple attachments or those combining text and attachments will fail and receive a descriptive error status. * Messages sent directly from the TikTok App will be synced into the system * Initiating a new conversation from the system is not permitted due to limitations from the TikTok Business Messaging API. **Backend: TikTok Channel Integration** * Added `Api::V1::Accounts::Tiktok::AuthorizationsController` to handle TikTok OAuth authorization initiation, returning the TikTok authorization URL. * Implemented `Tiktok::CallbacksController` to handle TikTok OAuth callback, process authorization results, create or update channel/inbox, and handle errors or denied scopes. * Added `Webhooks::TiktokController` to receive and verify TikTok webhook events, including signature verification and event dispatching. * Created `Tiktok::IntegrationHelper` module for JWT-based token generation and verification for secure TikTok OAuth state management. **Configuration and Feature Flags** * Added TikTok app credentials (`TIKTOK_APP_ID`, `TIKTOK_APP_SECRET`) to allowed configs and app config, and registered TikTok as a feature in the super admin features YAML. [[1]](diffhunk://#diff-5e46e1d248631a1147521477d84a54f8ba6846ea21c61eca5f70042d960467f4R43) [[2]](diffhunk://#diff-8bf37a019cab1dedea458c437bd93e34af1d6e22b1672b1d43ef6eaa4dcb7732R69) [[3]](diffhunk://#diff-123164bea29f3c096b0d018702b090d5ae670760c729141bd4169a36f5f5c1caR74-R79) **Frontend: TikTok Channel UI and Messaging Support** * Added `TiktokChannel` API client for frontend TikTok authorization requests. * Updated channel icon mappings and tests to include TikTok (`Channel::Tiktok`). [[1]](diffhunk://#diff-b852739ed45def61218d581d0de1ba73f213f55570aa5eec52aaa08f380d0e16R16) [[2]](diffhunk://#diff-3cd3ae32e94ef85f1f2c4435abf0775cc0614fb37ee25d97945cd51573ef199eR64-R69) * Enabled TikTok as a supported channel in contact forms, channel widgets, and feature toggles. [[1]](diffhunk://#diff-ec59c85e1403aaed1a7de35971fe16b7033d5cd763be590903ebf8f1ca25a010R47) [[2]](diffhunk://#diff-ec59c85e1403aaed1a7de35971fe16b7033d5cd763be590903ebf8f1ca25a010R69) [[3]](diffhunk://#diff-725b90ca7e3a6837ec8291e9f57094f6a46b3ee00e598d16564f77f32cf354b0R26-R29) [[4]](diffhunk://#diff-725b90ca7e3a6837ec8291e9f57094f6a46b3ee00e598d16564f77f32cf354b0R51-R54) [[5]](diffhunk://#diff-725b90ca7e3a6837ec8291e9f57094f6a46b3ee00e598d16564f77f32cf354b0R68) * Updated message meta logic to support TikTok-specific message statuses (sent, delivered, read). [[1]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696R23) [[2]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696L63-R65) [[3]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696L81-R84) [[4]](diffhunk://#diff-e41239cf8dda36c1bd1066dbb17588ae8868e56289072c74b3a6d7ef5abdd696L103-R107) * Added support for embedded message attachments (e.g., TikTok embeds) with a new `EmbedBubble` component and updated message rendering logic. [[1]](diffhunk://#diff-c3d701caf27d9c31e200c6143c11a11b9d8826f78aa2ce5aa107470e6fdb9d7fR31) [[2]](diffhunk://#diff-047859f9368a46d6d20177df7d6d623768488ecc38a5b1e284f958fad49add68R1-R19) [[3]](diffhunk://#diff-c3d701caf27d9c31e200c6143c11a11b9d8826f78aa2ce5aa107470e6fdb9d7fR316) [[4]](diffhunk://#diff-cbc85e7c4c8d56f2a847d0b01cd48ef36e5f87b43023bff0520fdfc707283085R52) * Adjusted reply policy and UI messaging for TikTok's 48-hour reply window. [[1]](diffhunk://#diff-0d691f6a983bd89502f91253ecf22e871314545d1e3d3b106fbfc76bf6d8e1c7R208-R210) [[2]](diffhunk://#diff-0d691f6a983bd89502f91253ecf22e871314545d1e3d3b106fbfc76bf6d8e1c7R224-R226) These changes collectively enable end-to-end TikTok channel support, from configuration and OAuth flow to webhook processing and frontend message handling. ------------ # TikTok App Setup & Configuration 1. Grant access to the Business Messaging API ([Documentation](https://business-api.tiktok.com/portal/docs?id=1832184145137922)) 2. Set the app authorization redirect URL to `https://FRONTEND_URL/tiktok/callback` 3. Update the installation config with TikTok App ID and Secret 4. Create a Business Messaging Webhook configuration and set the callback url to `https://FRONTEND_URL/webhooks/tiktok` ([Documentation](https://business-api.tiktok.com/portal/docs?id=1832190670631937)) . You can do this by calling `Tiktok::AuthClient.update_webhook_callback` from rails console once you finish Tiktok channel configuration in super admin ( will be automated in future ) 5. Enable TikTok channel feature in an account --------- Co-authored-by: Sojan Jose <sojan@pepalo.com> Co-authored-by: iamsivin <iamsivin@gmail.com> |
||
|
Gabriel Jablonski
|
774c168d94
|
fix: z-api read message (#165)
* fix: z-api read messages * fix: use threads for parallel requests * feat: use jobs instead * fix: refactor ZapiReadMessageJob to use service method for sending read messages |
||
|
Vinay Keerthi
|
a8ed074bf0
|
fix: Preserve double newlines in text-based messaging channels (#13055)
## Summary
Fixes the issue where double newlines (paragraph breaks) were collapsing
to single newlines in text-based messaging channels (Telegram, WhatsApp,
Instagram, Facebook, LINE, SMS).
### Root Cause
The `preserve_multiple_newlines` method only preserved 3+ consecutive
newlines using the regex `/\n{3,}/`. When users pressed Enter twice
(creating a paragraph break with 2 newlines), CommonMarker would parse
this as separate paragraphs, which then collapsed to a single newline in
the output.
This caused:
- ❌ Normal Enter: Double newlines collapsed to single newline
- ✅ Shift+Enter: Worked (created hard breaks)
### Fix
Changed the regex from `/\n{3,}/` to `/\n{2,}/` to preserve 2+
consecutive newlines. This prevents CommonMarker from collapsing
paragraph breaks.
Now:
- ✅ Single newline (`\n`) → Single newline (handled by softbreak)
- ✅ Double newline (`\n\n`) → Double newline (preserved with
placeholders)
- ✅ Triple+ newlines → Preserved as before
### Test Coverage
Added comprehensive tests for:
- Single newlines preservation
- Double newlines (paragraph breaks) preservation
- Multiple consecutive newlines
- Newlines with varying amounts of whitespace between them (1 space, 3
spaces, 5 spaces, tabs)
All 66 tests passing.
### Impact
This fix affects all text-based messaging channels that use the markdown
renderer:
- Telegram
- WhatsApp
- Instagram
- Facebook
- LINE
- SMS
- Twilio SMS (when configured for WhatsApp)
Fixes
https://linear.app/chatwoot/issue/CW-6135/double-newline-is-breaking
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
|
||
|
Muhsin Keloth
|
2bd8e76886
|
feat: Add backend changes for whatsapp csat template (#12984)
This PR add the backend changes for the feature [sending CSAT surveys via WhatsApp message templates ](https://github.com/chatwoot/chatwoot/pull/12787) --------- Co-authored-by: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com> |
||
|
Vinay Keerthi
|
89d02e2c92
|
fix: Preserve multiple newlines with whitespace in text-based messaging channels (#13044)
## Description Fixes an issue where multiple newlines with whitespace between them (e.g., `\n \n \n`) were being collapsed to single newlines in text-based messaging channels (Telegram, WhatsApp, Instagram, Facebook, Line, SMS). The frontend was sending messages with spaces/tabs between newlines, and the markdown renderer was treating these as paragraph content, collapsing them during rendering. ### Changes: 1. Added whitespace normalization in `render_telegram_html`, `render_whatsapp`, `render_instagram`, `render_line`, and `render_plain_text` methods 2. Strips whitespace from whitespace-only lines before markdown processing 3. Added comprehensive regression tests for all affected channels ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) ## How Has This Been Tested? 1. **Unit Tests**: Added 7 new specs testing multiple newlines with whitespace between them for all text-based channels 2. **Manual Testing**: Verified with actual frontend payload containing `\n \n \n` patterns 3. **Regression Testing**: All existing 63 specs pass ### Test Results: - ✅ All 63 markdown renderer specs pass (56 original + 7 new) - ✅ All 12 Telegram channel specs pass - ✅ All 27 WhatsApp + Instagram specs pass - ✅ Verified with real-world payload: 18 newlines preserved (previously collapsed to 1) ### Test Command: ```bash RAILS_ENV=test bundle exec rspec spec/services/messages/markdown_renderer_service_spec.rb RAILS_ENV=test bundle exec rspec spec/models/channel/telegram_spec.rb ``` ## Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes |
||
|
Vinay Keerthi
|
f2eaa845dc
|
fix: Preserve multiple consecutive newlines in text-based messaging channels (#13032)
## Description This PR fixes an issue where multiple consecutive newlines (blank lines for visual spacing) were being collapsed in text-based messaging channels like WhatsApp, Instagram, and SMS. When users send messages via API with intentional spacing using multiple newlines (e.g., `\n\n\n\n`), the markdown renderer was following standard Markdown spec and collapsing them into single blank lines. While this is correct for document formatting, messaging platforms like WhatsApp and Instagram support and preserve multiple blank lines for visual spacing. The fix adds preprocessing to preserve multiple consecutive newlines (3+) by converting them to placeholder tokens before CommonMarker processing, then restoring the exact number of newlines in the final output. ## Changes - Added `preserve_multiple_newlines` and `restore_multiple_newlines` helper methods to `MarkdownRendererService` - Updated `render_whatsapp` to preserve multiple consecutive newlines - Updated `render_instagram` to preserve multiple consecutive newlines - Updated `render_plain_text` (affects SMS, Twilio SMS, Twitter) to preserve multiple consecutive newlines - Updated `render_line` to preserve multiple consecutive newlines - HTML-based renderers (Email, Telegram, WebWidget) remain unchanged as they handle spacing via HTML tags ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality not to work as expected) - [ ] This change requires a documentation update ## How Has This Been Tested? Added comprehensive test coverage: - 3 new tests for multi-newline preservation across WhatsApp, Instagram, and SMS channels - All 56 tests passing (up from 53) Testing scenarios: - Single newlines preserved: `"Line 1\nLine 2"` remains `"Line 1\nLine 2"` - Multiple newlines preserved: `"Para 1\n\n\n\nPara 2"` remains `"Para 1\n\n\n\nPara 2"` - Standard paragraph breaks (2 newlines) work as before - Markdown formatting (bold, italic, links) continues to work correctly - Backward compatibility maintained for all channels ## Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] Any dependent changes have been merged and published in downstream modules |
||
|
Vinay Keerthi
|
141dfc3321
|
fix: preserve newlines and formatting in Twilio WhatsApp messages (#13022)
## Description This PR fixes an issue where Twilio WhatsApp messages were losing newlines and markdown formatting. The problem had two root causes: 1. Text-based renderers (WhatsApp, Instagram, SMS) were converting newlines to spaces when processing plain text without markdown list markers 2. Twilio WhatsApp channels were incorrectly using the plain text renderer instead of the WhatsApp renderer, stripping all markdown formatting The fix updates the markdown rendering system to: - Preserve newlines by overriding the `softbreak` method in WhatsApp, Instagram, and PlainText renderers - Detect Twilio WhatsApp channels (via the `medium` field) and route them to use the WhatsApp renderer - Maintain backward compatibility with existing code ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality not to work as expected) - [ ] This change requires a documentation update ## How Has This Been Tested? Added comprehensive test coverage: - 3 new tests for newline preservation in WhatsApp, Instagram, and SMS channels - 4 new tests for Twilio WhatsApp specific behavior (medium detection, formatting preservation, backward compatibility) - All 53 tests passing (up from 50) Manual testing verified: - Twilio WhatsApp messages with plain text preserve newlines - Twilio WhatsApp messages with markdown preserve formatting (bold, italic, links) - Regular WhatsApp, Instagram, and SMS channels continue to work correctly - Backward compatibility maintained when channel parameter is not provided ## Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] Any dependent changes have been merged and published in downstream modules |
||
|
Sivin Varghese
|
399c91adaa
|
feat: Standardize rich editor across all channels (#12600)
# Pull Request Template ## Description This PR includes, 1. **Channel-specific formatting and menu options** for the rich reply editor. 2. **Removal of the plain reply editor** and full **standardization** on the rich reply editor across all channels. 3. **Fix for multiple canned responses insertion:** * **Before:** The plain editor only allowed inserting canned responses at the beginning of a message, making it impossible to combine multiple canned responses in a single reply. This caused inconsistent behavior across the app. * **Solution:** Replaced the plain reply editor with the rich (ProseMirror) editor to ensure a unified experience. Agents can now insert multiple canned responses at any cursor position. 4. **Floating editor menu** for the reply box to improve accessibility and overall user experience. 5. **New Strikethrough formatting option** added to the editor menu. --- **Editor repo PR**: https://github.com/chatwoot/prosemirror-schema/pull/36 Fixes https://github.com/chatwoot/chatwoot/issues/12517, [CW-5924](https://linear.app/chatwoot/issue/CW-5924/standardize-the-editor), [CW-5679](https://linear.app/chatwoot/issue/CW-5679/allow-inserting-multiple-canned-responses-in-a-single-message) ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) ## How Has This Been Tested? ### Screenshot **Dark** <img width="850" height="345" alt="image" src="https://github.com/user-attachments/assets/47748e6c-380f-44a3-9e3b-c27e0c830bd0" /> **Light** <img width="850" height="345" alt="image" src="https://github.com/user-attachments/assets/6746cf32-bf63-4280-a5bd-bbd42c3cbe84" /> ## Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged and published in downstream modules --------- Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com> Co-authored-by: Pranav <pranav@chatwoot.com> Co-authored-by: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com> |
||
|
Gabriel Jablonski
|
797bde6566
|
feat: implement CSAT message handling in WhatsApp services (#161) | ||
|
Gabriel Jablonski
|
c73ef7f1a2
|
fix: enhance filename extraction for document messages with captions (#159)
* fix: enhance filename extraction for document messages with captions * test: add specs for filename extraction from document messages with and without captions * fix: update media URL stubbing for multiple message types in filename extraction tests |
||
|
Vishnu Narayanan
|
a8e9acfae9
|
fix: shopify and leadsquared specs in ci (#12926)
fix: shopify and leadsquared specs in ci |
||
|
Gabriel Jablonski
|
b03dfdb751
|
Chore/merge upstream 4.8.0 (#150)
* chore: Hide "Learn More" button in feature spotlight for self-hosted (#12675) * feat: single query for reporting event stats (#12664) This PR collapses multiple queries fetching stats from a single table to a single query ```sql SELECT user_id as user_id, COUNT(CASE WHEN name = 'conversation_resolved' THEN 1 END) as resolved_count, AVG(CASE WHEN name = 'conversation_resolved' THEN value END) as avg_resolution_time, AVG(CASE WHEN name = 'first_response' THEN value END) as avg_first_response_time, AVG(CASE WHEN name = 'reply_time' THEN value END) as avg_reply_time FROM "reporting_events" WHERE "reporting_events"."account_id" = <account_id> AND "reporting_events"."created_at" >= '2025-09-14 18:30:00' AND "reporting_events"."created_at" < '2025-10-14 18:29:59' GROUP BY "reporting_events"."user_id"; ``` ### Why this works? Here's why this optimization is faster based on PostgreSQL internals: - Single Table Scan vs Multiple Scans: Earlier we did 4 sequential scans (or 4 index scans) of the same data, with the same where clause, now in a single scan all 4 `CASE` expressions are evaluated in a single pass. - Shared Buffer Cache Efficiency: PostgreSQL's shared buffer cache stores recently accessed pages, with this, pages are loaded once and re-used for all aggregation, earlier with separate queries we were forced to re-read all from the disk each time - Reduced planning and network overhead (4 vs 1 query) ### How is it tested 1. The specs all pass without making any changes 2. Verified the reports side by side after generating from report seeder #### How to test Generate seed data using the following command ```bash ACCOUNT_ID=1 ENABLE_ACCOUNT_SEEDING=true bundle exec rake db:seed:reports_data ``` Once done download the reports, checkout to this branch and download the reports again and compare them * chore: Update translations (#12625) * chore: Migrate mailers from the worker to jobs (#12331) Previously, email replies were handled inside workers. There was no execution logs. This meant if emails silently failed (as reported by a customer), we had no way to trace where the issue happened, the only assumption was “no error = mail sent.” By moving email handling into jobs, we now have proper execution logs for each attempt. This makes it easier to debug delivery issues and would have better visibility when investigating customer reports. Fixes https://linear.app/chatwoot/issue/CW-5538/emails-are-not-sentdelivered-to-the-contact --------- Co-authored-by: Sojan Jose <sojan@pepalo.com> Co-authored-by: Shivam Mishra <scm.mymail@gmail.com> * chore(deps-dev): bump vite from 5.4.20 to 5.4.21 (#12700) Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.4.20 to 5.4.21. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/vitejs/vite/releases">vite's releases</a>.</em></p> <blockquote> <h2>v5.4.21</h2> <p>Please refer to <a href="https://github.com/vitejs/vite/blob/v5.4.21/packages/vite/CHANGELOG.md">CHANGELOG.md</a> for details.</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/vitejs/vite/blob/v5.4.21/packages/vite/CHANGELOG.md">vite's changelog</a>.</em></p> <blockquote> <h2><!-- raw HTML omitted -->5.4.21 (2025-10-20)<!-- raw HTML omitted --></h2> <ul> <li>fix(dev): trim trailing slash before <code>server.fs.deny</code> check (<a href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20968">#20968</a>) (<a href="https://github.com/vitejs/vite/tree/HEAD/packages/vite/issues/20970">#20970</a>) (<a href=" |
||
|
Gabriel Jablonski
|
14f43a6bc5
|
fix(zapi): contact race condition (#149) | ||
|
Vishnu Narayanan
|
08b9134486
|
feat: speed up circleci and github actions (#12849)
# 🚀 Speed up CI/CD test execution with parallelization ## TL;DR - **Problem**: CI tests took 36-42 minutes per commit, blocking developer workflow - **Solution**: Implemented 16-way parallelization + optimized slow tests + fixed Docker builds - **Impact**: **1,358 hours/month saved** (7.7 FTE) across GitHub Actions + CircleCI - GitHub Actions tests: 36m → 7m (82% faster) - Backend tests: 28m → 4m (87% faster with 16-way parallelization) - CircleCI tests: 42m → 7m (83% faster) - Docker builds: 34m → 5m (85% faster) - **Result**: 5-6x faster feedback loops, 100% success rate on recent runs --- ## Problem CI test runs were taking **36-42 minutes per commit push** (GitHub Actions: 36m avg, CircleCI: 42m P95), creating a significant bottleneck in the development workflow. ## Solution This PR comprehensively restructures both CI test pipelines to leverage 16-way parallelization and optimize test execution, reducing test runtime from **36-42 minutes to ~7 minutes** - an **82% improvement**. --- ## 📊 Real Performance Data (Both CI Systems) ### GitHub Actions #### Before (develop branch - 5 recent runs) ``` Individual runs: 35m 29s | 36m 1s | 40m 0s | 36m 4s | 34m 18s Average: 36m 22s ``` #### After (feat/speed_up_ci branch - 9 successful runs) ``` Individual runs: 6m 39s | 7m 2s | 6m 53s | 6m 26s | 6m 52s | 6m 42s | 6m 45s | 6m 40s | 6m 37s Average: 6m 44s Range: 6m 26s - 7m 2s ``` **Improvement**: ⚡ **81.5% faster** (29m 38s saved per run) #### Backend Tests Specific Impact With 16-way parallelization, backend tests show dramatic improvement: - **Before**: 27m 52s (sequential execution) - **After**: 3m 44s (longest of 16 parallel runners) - Average across runners: 2m 30s - Range: 1m 52s - 3m 44s - **Improvement**: ⚡ **86.6% faster** (24m 8s saved) --- ### CircleCI #### Before (develop branch - CircleCI Insights) ``` Duration (P95): 41m 44s Runs: 70 (last 30 days) Success Rate: 84% ``` #### After (feat/speed_up_ci branch - Last 2 pipeline runs) ``` Run 1 (1h ago): 7m 7s ├─ lint: 4m 12s ├─ frontend-tests: 5m 36s ├─ backend-tests: 6m 23s ├─ coverage: 20s └─ build: 1s Run 2 (2h ago): 7m 21s ├─ lint: 3m 47s ├─ frontend-tests: 5m 4s ├─ backend-tests: 6m 33s ├─ coverage: 19s └─ build: 1s Average: 7m 14s Success Rate: 100% ✅ ``` **Improvement**: ⚡ **82.7% faster** (34m 30s saved per run) --- ## 🐳 Related Work: Docker Build Optimization As part of the broader CI/CD optimization effort, Docker build performance was improved separately in **PR #12859**. ### Docker Build Fix (Merged Separately) **Problem**: Multi-architecture Docker builds (amd64/arm64) were taking ~34 minutes due to cache thrashing **Solution**: Added separate cache scopes per platform in `.github/workflows/test_docker_build.yml`: ```yaml cache-from: type=gha,scope=${{ matrix.platform }} cache-to: type=gha,mode=max,scope=${{ matrix.platform }} ``` **Results** (measured from November 2025 data): - **Before**: 34.2 minutes/run average (15,547 minutes across 454 runs) - **After**: 5 minutes/run - **Improvement**: 85% faster, 29.2 minutes saved per run - **Frequency**: 25.2 runs/day - **Monthly savings**: **369 hours** (46 developer-days) This prevents different architectures from invalidating each other's caches and contributes 27% of total CI/CD time savings. --- ## 🎯 Key Findings ### Both CI Systems Now Perform Similarly - **CircleCI**: 7m 14s average - **GitHub Actions**: 6m 44s average - **Difference**: Only 30 seconds apart (remarkably consistent!) ### Combined Performance - **Average improvement across both systems**: **82.1% faster** - **Time saved per commit**: ~32 minutes - **Developer feedback loop**: 36-42 minutes → ~7 minutes ### Success Rate Improvement - **CircleCI**: 84% → 100% (on feat/speed_up_ci branch) - **GitHub Actions**: 100% (all 9 recent runs successful) - Fixed all test isolation issues that caused intermittent failures ### Impact at Scale (Based on Real November 2025 Data) - **CI runs per day**: **30.8 average** for tests, **25.2** for Docker builds - Measured from GitHub Actions Usage Metrics (18 days) - Weekdays: 38-54 runs/day - Peak: up to 68 runs in a single day - **This PR (test suite only)**: - **Daily time saved**: **15.3 hours** (GitHub Actions + CircleCI) - **Monthly time saved**: **458 hours** (57 developer-days) on GitHub Actions - Additional **531 hours** (66 developer-days) on CircleCI - **Combined with Docker optimization** (PR #12859): **1,358 hours/month** (see Summary) - **Developer experience**: 5-6x faster iteration cycles --- ## Code Changes ### 1. **Backend Test Parallelization (16x)** Both CI systems now use 16-way parallelization with identical round-robin test distribution: ```bash # Distribute tests evenly across 16 runners SPEC_FILES=($(find spec -name '*_spec.rb' | sort)) for i in "${!SPEC_FILES[@]}"; do if [ $(( i % 16 )) -eq $RUNNER_INDEX ]; then TESTS="$TESTS ${SPEC_FILES[$i]}" fi done ``` **Why round-robin over timing-based?** - CircleCI's timing-based splitting grouped similar tests together - This caused race conditions with OAuth callback tests (Linear, Shopify, Notion) - Round-robin ensures even distribution and better test isolation - Both CI systems now behave identically ### 2. **Frontend Test Optimization** Enabled Vitest thread parallelization in `vite.config.ts`: ```typescript pool: 'threads', poolOptions: { threads: { singleThread: false, }, }, ``` ### 3. **CI Architecture Restructuring** Split monolithic CI jobs into parallel stages: - **Lint** (backend + frontend) - runs independently for fast feedback - **Frontend tests** - runs in parallel with backend - **Backend tests** - 16-way parallelized across runners - **Coverage** - aggregates results from test jobs - **Build** (CircleCI only) - final job for GitHub status check compatibility ### 4. **Critical Test Optimization** **report_builder_spec.rb**: Changed `before` to `before_all` - Reduced execution time from **19 minutes to 1.2 minutes** (16x speedup) - Setup now runs once instead of 21 times - Single biggest performance improvement after parallelization --- ## Test Stability Fixes (10 spec files) Parallelization exposed latent test isolation issues that were fixed: ### Object Identity Comparisons (6 files) Tests were comparing Ruby object instances instead of IDs: - `spec/models/integrations/hook_spec.rb` - Use `.pluck(:id)` for comparisons - `spec/enterprise/models/captain/scenario_spec.rb` - Compare IDs instead of objects - `spec/models/notification_spec.rb` - Compare IDs for sort order validation - `spec/models/account_spec.rb` - Compare IDs in scope queries - `spec/services/widget/token_service_spec.rb` - Compare class names instead of class objects - `spec/models/concerns/avatarable_shared.rb` - Use `respond_to` checks for ActiveStorage ### Database Query Caching - `spec/jobs/delete_object_job_spec.rb` - Added `.reload` to force fresh database queries ### Test Expectations Timing - `spec/jobs/mutex_application_job_spec.rb` - Removed flaky unlock expectation after error block - Related to original PR #8770 - Expectation after error block never executes in parallel environments ### Timezone Handling - `spec/mailers/account_notification_mailer_spec.rb` - Fixed date parsing at timezone boundaries - Changed test time from 23:59:59Z to 12:00:00Z ### Test Setup - `spec/builders/v2/report_builder_spec.rb` - Optimized with `before_all` --- ## CircleCI GitHub Integration Fix ### Problem GitHub PR checks were stuck on "Waiting for status" even when all CircleCI jobs passed. GitHub was expecting a job named `build` but the workflow only had a workflow named "build". ### Solution Added an explicit `build` job that runs after all other jobs: ```yaml build: steps: - run: name: Legacy build aggregator command: echo "All main jobs passed" requires: - lint - coverage ``` This ensures GitHub's required status checks work correctly. --- ## ✅ Testing & Validation - ✅ **GitHub Actions**: 9 successful runs, consistent 6m 26s - 7m 2s runtime - ✅ **CircleCI**: 2 successful runs, consistent 7m 7s - 7m 21s runtime - ✅ Both CI systems produce identical, consistent results - ✅ GitHub PR status checks complete correctly - ✅ Success rate improved from 84% to 100% (recent runs) - ✅ No test regressions introduced - ✅ All flaky tests fixed (callback controllers, mutex jobs, etc.) --- ## 🎉 Summary This PR delivers an **82% improvement** in test execution time across both CI systems: - **GitHub Actions tests**: 36m → 7m (81.5% faster) - Backend tests specifically: 28m → 4m (86.6% faster) - **CircleCI tests**: 42m → 7m (82.7% faster) - **Developer feedback loop**: 5-6x faster - **Test stability**: 84% → 100% success rate ### 📊 Total CI/CD Impact (All Optimizations) Based on real November 2025 data, combining this PR with Docker build optimization (PR #12859): **Monthly Time Savings**: **1,358 hours/month** = **170 developer-days/month** = **7.7 FTE** | System | Runs/Day | Before | After | Savings | Monthly Impact | |--------|----------|---------|--------|---------|----------------| | **GitHub Actions Tests** | 30.8 | 36.5m | 6.7m | 29.8m/run | 458 hrs (34%) | | **GitHub Actions Docker** | 25.2 | 34.2m | 5.0m | 29.2m/run | 369 hrs (27%) | | **CircleCI Tests** | 30.8 | 41.7m | 7.2m | 34.5m/run | 531 hrs (39%) | *Data source: GitHub Actions Usage Metrics (November 2025, 18 days), CircleCI Insights (30 days)* The combined optimizations save the equivalent of **nearly 8 full-time developers** worth of CI waiting time every month, significantly improving developer velocity and reducing CI costs. All test isolation issues exposed by parallelization have been fixed, ensuring reliable and consistent results across both CI platforms. woot woot !!! --------- |
||
|
Sojan Jose
|
5f2b2f4221
|
feat: APIs to assign agents_bots as assignee in conversations (#12836)
## Summary - add an assignee_agent_bot_id column as an initital step to prototype this before fully switching to polymorphic assignee - update assignment APIs and conversation list / show endpoints to reflect assignee as agent bot - ensure webhook payloads contains agent bot assignee [Codex Task](https://chatgpt.com/codex/tasks/task_e_6912833377e48326b6641b9eee32d50f) --------- Co-authored-by: Pranav <pranav@chatwoot.com> |
||
|
Tanmay Deep Sharma
|
c9823d9409
|
feat: Assignment service (v2) (#12320)
## Linear Link ## Description This PR introduces a new robust auto-assignment system for conversations in Chatwoot. The system replaces the existing round-robin assignment with a more sophisticated service-based architecture that supports multiple assignment strategies, rate limiting, and Enterprise features like capacity-based assignment and balanced distribution. ## Type of change - [ ] New feature (non-breaking change which adds functionality) ## How Has This Been Tested? - Unit test cases - Test conversations getting assigned on status change to open - Test the job directly via rails console ## Checklist: - [ ] My code follows the style guidelines of this project - [ ] I have performed a self-review of my code - [ ] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged and published in downstream modules <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Adds a new service-based auto-assignment system with scheduled jobs, rate limiting, enterprise capacity/balanced selection, and wiring via inbox/handler; includes Redis helpers and comprehensive tests. > > - **Auto-assignment v2 (core services)**: > - Add `AutoAssignment::AssignmentService` with bulk assignment, configurable conversation priority, RR selection, and per-agent rate limiting via `AutoAssignment::RateLimiter`. > - Add `AutoAssignment::RoundRobinSelector` for agent selection. > - **Jobs & scheduling**: > - Add `AutoAssignment::AssignmentJob` (per-inbox bulk assign; env-based limit) and `AutoAssignment::PeriodicAssignmentJob` (batch over accounts/inboxes). > - Schedule periodic run in `config/schedule.yml` (`periodic_assignment_job`). > - **Model/concerns wiring**: > - Include `InboxAgentAvailability` in `Inbox`; add `Inbox#auto_assignment_v2_enabled?`. > - Update `AutoAssignmentHandler` to trigger v2 job when `auto_assignment_v2_enabled?`, else fallback to legacy. > - **Enterprise extensions**: > - Add `Enterprise::InboxAgentAvailability` (capacity-aware filtering) and `Enterprise::Concerns::Inbox` association `inbox_capacity_limits`. > - Extend service via `Enterprise::AutoAssignment::AssignmentService` (policy-driven config, capacity filtering, exclusion rules) and add selectors/services: `BalancedSelector`, `CapacityService`. > - **Infrastructure**: > - Enhance `Redis::Alfred` with `expire`, key scan/count, and extended ZSET helpers (`zadd`, `zcount`, `zcard`, `zrangebyscore`). > - **Tests**: > - Add specs for jobs, core service, rate limiter, RR selector, and enterprise features (capacity, balanced selection, exclusions). > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 0ebe187c8aea73765b0122a44b18d6f465c2477f. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com> Co-authored-by: Shivam Mishra <scm.mymail@gmail.com> |
||
|
Gabriel Jablonski
|
c041090675
|
feat(baileys): upgrade to v7 (#143)
* feat(baileys): upgrade to v7 (#139) * feat(baileys): v7 * chore(zapi): match baileys changes * test: fix send on whatsapp spec * chore: simplify logic and minor fixes * fix: ensure contact avatar updates when profile picture changes * chore: add beta release workflow * feat: enhance contact update logic to handle conflicts in inbox updates (#140) * feat: update GitHub Actions workflow to trigger on release events and adjust GIT_REF handling * fix: prevent unique constraint error when LID contact inbox already exists (#141) * fix: prevent unique constraint error when LID contact inbox already exists * feat(baileys): handle ephemeral messages |
||
|
Shivam Mishra
|
615e81731c
|
refactor: strategy pattern for mailbox conversation finding (#12766)
Co-authored-by: Pranav <pranav@chatwoot.com> Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com> |
||
|
Gabriel Jablonski
|
63a2c70667
|
fix(zapi): first connection issues, and merge existing contact on received message (#138)
* fix(zapi): pairing issues on first connection * fix(zapi): handle existing contact when receiving new message * refactor: simplify logic |
||
|
Tanmay Deep Sharma
|
90352b3a20
|
fix: Remove the same account validation for whatsapp channels (#12811)
## Description Modified the phone number validation in Whatsapp::ChannelCreationService to check for duplicate phone numbers across ALL accounts, not just within the current account. ## Type of change - [ ] Bug fix (non-breaking change which fixes an issue) ## How Has This Been Tested? - Added test coverage for cross-account phone number validation - Using actual UI flow <img width="1493" height="532" alt="image" src="https://github.com/user-attachments/assets/67d2bb99-2eb9-4115-8d56-449e4785e0d8" /> ## Checklist: - [ ] My code follows the style guidelines of this project - [ ] I have performed a self-review of my code - [ ] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged and published in downstream modules |
||
|
Pranav
|
5491ca2470
|
feat: Differentiate bot and user in the summary (#12801)
While generating the summary, use the appropriate sender type for the message. |
||
|
Sojan Jose
|
f89d9a4401
|
feat: Bulk delete for contacts (#12778)
Introduces a new bulk action `delete` for contacts ref: https://github.com/chatwoot/chatwoot/pull/12763 ## Screens <img width="1492" height="973" alt="Screenshot 2025-10-31 at 6 27 21 PM" src="https://github.com/user-attachments/assets/30dab1bb-2c2c-4168-9800-44e0eb5f8e3a" /> <img width="1492" height="985" alt="Screenshot 2025-10-31 at 6 27 32 PM" src="https://github.com/user-attachments/assets/5be610c4-b19e-4614-a164-103b22337382" /> |
||
|
Pranav
|
40c75941f5
|
fix: Avoid introducing new attributes in search (#12791)
Fix `Limit of total fields [1000] has been exceeded` https://linear.app/chatwoot/issue/CW-5861/searchkickimporterror-type-=-illegal-argument-exception-reason-=-limit#comment-6b6e41bd |
||
|
Gabriel Jablonski
|
7a3544ba99
|
chore: remove zapi feature flag (#132) | ||
|
Sojan Jose
|
159c810117
|
feat: Bulk actions for contacts (#12763)
Introduces APIs and UI for bulk actions in contacts table. The initial action available will be assign labels Fixes: #8536 #12253 ## Screens <img width="1350" height="747" alt="Screenshot 2025-10-29 at 4 05 08 PM" src="https://github.com/user-attachments/assets/0792dff5-0371-4b2e-bdfb-cd32db773402" /> <img width="1345" height="717" alt="Screenshot 2025-10-29 at 4 05 19 PM" src="https://github.com/user-attachments/assets/ae510404-c6de-4c15-a720-f6d10cdac25b" /> --------- Co-authored-by: Muhsin <muhsinkeramam@gmail.com> Co-authored-by: iamsivin <iamsivin@gmail.com> Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com> |
||
|
Muhsin Keloth
|
26ea87a6cb
|
fix: Extend phone number normalization to Twilio WhatsApp (#12655)
### Problem WhatsApp Cloud channels already handle Brazil/Argentina phone number format mismatches (PRs #12492, #11173), but Twilio WhatsApp channels were creating duplicate contacts when: - Template sent to new format: `whatsapp:+5541988887777` (13 digits) - User responds from old format: `whatsapp:+554188887777` (12 digits) ### Solution The solution extends the existing phone number normalization infrastructure to support both WhatsApp providers while handling their different payload formats: ### Provider Format Differences - **WhatsApp Cloud**: `wa_id: "919745786257"` (clean number) - **Twilio WhatsApp**: `From: "whatsapp:+919745786257"` (prefixed format) ### Test Coverage #### Brazil Phone Number Tests **Case 1: New Format (13 digits with "9")** - **Test 1**: No existing contact → Creates new contact with original format - **Test 2**: Contact exists in same format → Appends to existing conversation **Case 2: Old Format (12 digits without "9")** - **Test 3**: Contact exists in old format → Appends to existing conversation - **Test 4** *(Critical)*: Contact exists in new format, message in old format → Finds existing contact, prevents duplicate - **Test 5**: No contact exists → Creates new contact with incoming format #### Argentina Phone Number Tests **Case 3: With "9" after country code** - **Test 6**: No existing contact → Creates new contact - **Test 7**: Contact exists in normalized format → Uses existing contact **Case 4: Without "9" after country code** - **Test 8**: Contact exists in same format → Appends to existing - **Test 9**: No contact exists → Creates new contact Fixes https://linear.app/chatwoot/issue/CW-5565/inconsistencies-for-mobile-numbersargentina-brazil-and-mexico-numbers |
||
|
Gabriel Jablonski
|
a8777f5b04
|
fix(baileys): update LID contact phone number if not present (#129)
* fix(baileys): update LID contact phone number if not present * fix(baileys): move try_update_contact_avatar call to update_contact_information method * refactor(baileys): consolidate contact update logic in update_contact_information method |