Commit Graph

200 Commits

Author SHA1 Message Date
Sojan Jose
390cd756e8
chore: Reorganize the installation config settings (#8794)
- Reorganizing installation config settings to move more configurations into UI from environment variables
- Changes to installation config to support premium plans in the enterprise edition
- Fixes the broken premium indicator in account/show and accounts/edit page
2024-01-31 16:48:42 +04:00
Shivam Mishra
3760f206e8
fix: mutex timeout and error handling (#8770)
Fixes the follow cases 
- The ensure block released the lock even on LockAcquisitionError
- Custom timeout was not allowed

This also refactored the with_lock method, now the key has to be constructed in the parent function itself

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-01-24 14:18:21 +04:00
Surabhi Suman
4b40c61201
feat: Support Regex validation for custom attributes (#7856)
This allows a user to add/update a custom regex and a cue while defining custom attributes(Only applicable for type- text).
While adding/editing custom attributes, the values are validated against the attribute definition regex, and if it is incorrect, a cue message or default error message is shown and restricts invalid values from being saved.

Fixes: #6866
2024-01-23 18:01:57 +04:00
Sojan Jose
bc04d81a5a
fix: Handle Net::IMAP::InvalidResponseError Exception bad response type "ESMTP" (#8755) 2024-01-22 15:32:44 +05:30
Pranav Raj S
29106ebc4b
fix: raise_lock_acquisition_error if the job cannot set the lock (#8744)
Consider a scenario where two jobs are concurrently accessing a job with a lock mechanism. If the second job accesses lock_manager.locked? before the first job called lock_manager.lock(lock_key), it would return a false positive. At this point, both jobs can be executed freely.

To address this issue, the following change ensures that only the current thread that has set the lock can execute. Otherwise, a LockAcquisitionError will be thrown.

Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2024-01-19 14:31:41 +04:00
Pranav Raj S
a579684b34
fix: Update the message-id check to use messageId parsing from mail gem (#8743)
- Update the message ID parsing logic to use the mail gem
- Update the code to improve readability
2024-01-19 14:29:04 +04:00
Muhsin Keloth
0ac015ce7a
feat: Add a job to clear notifications that were created before 1 month (#8732) 2024-01-19 13:03:23 +05:30
Muhsin Keloth
19474e0074
chore: Improve active job error logs for deserialization error (#8742)
- Improve active job error logs for deserialization error 

Co-authored-by: Sojan <sojan@pepalo.com>
2024-01-18 19:27:18 +04:00
Pranav Raj S
1f4d860d9d
fix: Use channel.inbox instead of inbox (#8734) 2024-01-18 00:46:25 -08:00
Pranav Raj S
fdbb3bf4b1
fix: Optimize email fetching logic to fix bandwidth exceeded issues (#8730)
The Inboxes::FetchImapEmailsJob is designed to fetch the entire email object and check if its message id is already in the database. However, this process is resource-intensive and time-consuming, as fetching the full email object takes a significant amount of time.

On average, fetching 100 emails can take approximately 3-4 minutes to complete depending on the IMAP server. Since we are not using server flags to identify which emails have already been fetched (to avoid compatibility issues with flags in different email services), we have to fetch all previously available emails. Currently we fetch all the messages that were created from yesterday. This becomes problematic with services like Gmail, which throttle requests based on bandwidth usage.

To address this issue, I have updated the logic as follows:

Fetch the sequence IDs of all the mails in the "Inbox" that were created from yesterday.
Use the FETCH command to fetch only the message-ids using BODY.PEEK[HEADER.FIELDS (MESSAGE-ID)] with the sequence IDs we got in the previous step. This is a faster operation with lower bandwidth usage, as it only returns the sequence ID and message ID.
Check if the message IDs are already present in the database for the selected inbox.
If not present, fetch the entire email object using the FETCH command and create the message.
If the message ID is already present, ignore it and move on to the next message-id.
I have also addressed the issue of duplicate emails appearing in conversations when two fetch email jobs occur simultaneously. I have added a lock for the channel to ensure that the job gracefully exits without waiting for the current job to complete.

Fixes #7247
Fixes #6082
Fixes #8314

Co-authored-by: Sojan <sojan@pepalo.com>
2024-01-18 11:45:16 +04:00
Muhsin Keloth
c899cc825d
fix: Handle Contact import MalformedCSVError (#8706) 2024-01-18 13:05:58 +05:30
Muhsin Keloth
818424259f
chore: Get all notification API improvments (#8549)
Co-authored-by: Sojan Jose <sojan@chatwoot.com>
2024-01-17 09:02:18 +05:30
Shivam Mishra
10076c6a3e
feat: set lock timeout to 1 second (#8661) 2024-01-12 08:28:07 +05:30
Muhsin Keloth
68062216e4
feat: Add a job to reopen snoozed notifications (#8545) 2024-01-05 15:13:22 +05:30
Pranav Raj S
00eb5b152a
chore(migration): Re-run database migration job for caching. (#8581)
A support request that came to Chatwoot Cloud revealed that the job was timed prematurely. The default timeout for Sidekiq queues was set as 25 minutes in sidekiq.yml file. The cache was not created properly for the accounts with more than 100k conversations.

This change removes the cache logic in the previous migration and creates a new migration with a new job which processes conversations in batch.
2023-12-18 17:29:02 -08:00
Sojan Jose
0b2f539ad0
chore: Update self-hosted billing redirect (#8580) 2023-12-18 16:34:47 -08:00
Pranav Raj S
890515edfd
feat(perf): Cache labels on the conversation model (#8527) 2023-12-11 18:27:55 -08:00
Muhsin Keloth
80ff5e2d0a
chore: Change the RemoveMessageNotifications job implementation. (#8521) 2023-12-11 08:27:30 +05:30
Nithin David Thomas
f002870c6a
feat: Super admin design improvements (#8517)
- Revamp the super admin design
- Introduce a new settings page for support and billing settings
- Move the access tokens into users, agent bots and platform app show pages


Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: Pranav Raj S <pranav@chatwoot.com>
2023-12-08 19:40:35 -08:00
Muhsin Keloth
db33b7d1dc
feat: Clear all previous notifications if a new notification is added to a conversation (#8490) 2023-12-06 14:03:43 +05:30
Muhsin Keloth
76711d95ff
chore: Change primary actor to Conversation for all the notification types. (#8435) 2023-12-06 10:43:09 +05:30
Muhsin Keloth
6c8dacfa0d
feat: Facebook delivery reports (#8136) 2023-11-20 12:22:45 +05:30
Muhsin Keloth
af7631d9f1
chore: Update MarkMessagesAsReadJob to accept delivered status (#8319) 2023-11-08 13:44:57 -08:00
Vishnu Narayanan
677888bcde
feat: Add job to remove stale contact inboxes (#8096) 2023-11-06 20:24:30 -08:00
Muhsin Keloth
b4d20689b7
feat: Mark the messages as failed if the API channel webhooks fail for any reason. (#8277) 2023-11-04 12:26:28 +05:30
Muhsin Keloth
24fbab94c3
chore: Refactor MarkMessagesAsReadJob based on conversation id and time stamp (#8217)
- Mark all messages as read by providing the conversation ID and timestamp.
- For Instagram, ensure all previous messages that weren't marked as failed are now marked as read. This is because the read events are only triggered for the most recent message and not for any previous ones.
2023-10-27 15:21:39 -07:00
Muhsin Keloth
61e03fa33a
feat: Bandwidth sms channel delivery reports (#8198)
Fixes: https://linear.app/chatwoot/issue/CW-2566/delivery-report-for-bandwidth-sms
2023-10-27 14:07:15 -07:00
Pranav Raj S
6c4b92f7f6
fix: Revert the changes for SyncCustomFilterCountJob (#8238) 2023-10-27 10:45:48 -07:00
Muhsin Keloth
78ce8a4652
feat: Add support for Instagram delivery reports (#8125) 2023-10-18 23:42:34 -07:00
Muhsin Keloth
980013abae
feat: Add delivery reports for live chat (#8092) 2023-10-12 20:48:20 +05:30
Muhsin Keloth
26e8877cd9
feat: Support link unfurling for all the channels within the same connected channel account. (#8033) 2023-10-08 17:55:03 +05:30
Sojan Jose
6a73953003
chore: Add delay before running dataimport job (#8039)
- We have observed some failures for data import jobs in the cloud due to race conditions with job executions and active storage file uploading. This PR adds delays and retries to accommodate that.
2023-10-03 22:18:57 -07:00
Muhsin Keloth
24fe3805d8
feat: Slack link unfurling (#7940)
Co-authored-by: Pranav Raj S <pranav@chatwoot.com>
2023-09-29 19:35:56 +05:30
Shivam Mishra
336584c95a
feat: mutex for InstagramEventsJob [CW-2447] (#7828) 2023-09-04 15:02:13 +05:30
Shivam Mishra
2acf09b3eb
refactor: use conversation_id instead of sender_id (#7831) 2023-08-31 16:10:37 +07:00
Shivam Mishra
5598b4b27e
feat: implement mutex for SlackSendJob (#7783) 2023-08-25 11:58:29 +07:00
Sojan Jose
64ae9f625a
chore: Persist emojis in Contact Import (#7803)
The previous fix would remove emojis in contact data. This change ensures they are persisited

ref: #7787
2023-08-24 14:04:52 -07:00
Sojan Jose
acb7debd3f
chore: Contact import improvements (#7787)
- Ensure existing contact information is updated on data import
- Refactor the existing job to make it more readable
- Fixes issues with import files in the wrong encoding
fixes: #7307
2023-08-23 23:24:47 -07:00
Shivam Mishra
44f6a9ec3c
feat: update lock timeout and retry configuration (#7785) 2023-08-24 12:59:47 +07:00
Shivam Mishra
26ef21a243
feat: locking and retry in FB message parsing (#7701) 2023-08-23 09:48:17 +07:00
Pranav Raj S
9ddd428935
feat: Update the slack integration-flow to allow users to select the channel (#7637) 2023-07-28 14:50:30 -07:00
Pranav Raj S
022f4f899f
Revert "feat: Support Azure single-tenant application using the Graph… (#7436) 2023-06-29 16:50:18 -07:00
Tejaswini Chile
d05c953eef
feat: Support Azure single-tenant application using the Graph API (#6728) (#6878) 2023-06-28 08:13:08 +05:30
Vishnu Narayanan
f963e00731
fix: capture user and ip details on Inbox delete (#7395)
Fixes: https://linear.app/chatwoot/issue/CW-1772/ip-address-and-user-details-are-missing-in-some-of-the-logs

Co-authored-by: Sojan <sojan@pepalo.com>
2023-06-27 19:49:24 +05:30
Tejaswini Chile
212d7caab1
fix: log User is authenticated but not connected (#7394) 2023-06-26 16:07:40 +05:30
Tejaswini Chile
9d0de04f7c
fix: set custom filter count in redis (#7164) 2023-06-19 16:10:03 +05:30
Tejaswini Chile
23ca6d56f9
feat: Contact Exports (#7258) 2023-06-13 09:18:43 +05:30
Tejaswini Chile
f64f2138db
fix: Update mail check for html_part (#7273) 2023-06-09 16:08:40 +05:30
Tejaswini Chile
d25e7fd54e
fix: add email logging for source-id email-id and message-id (#7262)
* fix: mail logger for source-id email-id and message-id

* added mail provided to the tracking

* change the logger verb from info to error

* app/jobs/inboxes/fetch_imap_emails_job.rb

* codeclimate fix

* remove extra filter

* fixes: specs
2023-06-07 18:24:37 +05:30
Sojan Jose
ffc6364690
chore: Improve Whatsapp Templates Sync (#7210)
- update the templates updated at, even if the API request fails ( to prevent jobs from stacking up in case of API failures upstream )
- sequence the job in batches of 25 requests per minutes schedule ( in case API response time is high, also not to send too many requests in a single batch )
- move the sync job re-rerun to 3 hours ( since we are updating the updated at even in case of failures )(prev 15 minutes )

Fixes: https://linear.app/chatwoot/issue/CW-1590
2023-05-29 22:23:22 +05:30
Tejaswini Chile
8e79cf72e1
fix: issue with slack job (#7179) 2023-05-24 20:22:34 +05:30
Tejaswini Chile
590e4e9c1c
Fix: prevent IMAPBadResponse exception from sending the authorization mail (#7154) 2023-05-22 13:51:56 +05:30
Tejaswini Chile
0e903d2365
chore: Add delay for slack job when message has attachments (#7107) 2023-05-22 13:51:14 +05:30
Clairton Rodrigo Heinzen
a25179c342
fix: Instagram events job NoMethodError: undefined method for nil:NilClass (#7105)
- Sometimes tag entry does not have messaging or standby and cause a error
2023-05-18 15:22:01 +05:30
Tejaswini Chile
0f776a173c
Fix: Mail ate compare with nil date (#7084)
fixes: https://chatwoot-p3.sentry.io/issues/4185144468/?project=6382945
2023-05-15 21:42:22 +05:30
Sojan Jose
d320605556
chore: Fix Activerecord querry cancelled when fetching email (#7083)
limiting the query to 2 days to ensure that it runs timebound.

fixes: https://linear.app/chatwoot/issue/CW-1747/activerecordquerycanceled-pgquerycanceled-error-canceling-statement
2023-05-15 21:23:17 +05:30
Tejaswini Chile
b994706265
fix: Email date attribute not being set (#7081)
Fixes: https://linear.app/chatwoot/issue/CW-1738/typeerror-no-implicit-conversion-of-nil-into-string-typeerror
2023-05-15 20:19:03 +05:30
Tejaswini Chile
65e91f6a6f
fix: Duplicate instagram conversations for echo messages (#7078) 2023-05-15 16:25:15 +05:30
Tejaswini Chile
354010a6e1
chore: fetch mails with multiple attachments (#7030) 2023-05-14 10:02:36 +05:30
Sojan Jose
85e57c2e94
chore: Reorganize Sidekiq Queues (#6976)
- Rearrange and reprioritize current sidekiq queues
- Trim the unnecessary queues

ref: https://linear.app/chatwoot/issue/CW-1480/chore-run-all-sidekiq-jobs-async
2023-05-04 15:44:16 +05:30
Sojan Jose
cf91e9eb58
chore: Use find_each instead of .all.each (#6975)
- Enable the rubocop Rails/FindEach
- Replace the .all.each with .find_each

This should let us avoid potential memory usage.
Motivation from the speedshop newsletter by Nate Berkopec

ref: https://www.rubyinrails.com/2017/11/16/use-find-each-instead-of-all-each-in-rails/
ref: https://linear.app/chatwoot/issue/CW-1480/chore-run-all-sidekiq-jobs-async
2023-04-25 09:32:35 +05:30
Pranav Raj S
c5c36af529
fix: Remove fallback phone_number search in WhatsApp event processing (#6904) 2023-04-18 18:30:01 -07:00
Tejaswini Chile
905e77048f
chore: Contact import improvements (CW-1362) (#6747)
Fixes: https://linear.app/chatwoot/issue/CW-1362/csv-imports-are-not-working-properly
Fixes: #3462

---------

Co-authored-by: Sojan <sojan@pepalo.com>
2023-04-18 00:40:55 +05:30
Pranav Raj S
463c09184c
fix: Disable processing events if account is suspended (#6849) 2023-04-07 12:01:03 -07:00
Tejaswini Chile
6a0ca35de4
Feat: detect language of the message content (#6660) 2023-04-04 08:57:27 +05:30
Tejaswini Chile
d2d6d271c3
CW-1399 Phone number import (#6815) 2023-04-03 13:53:59 +05:30
Tejaswini Chile
21da03fe5b
Fix: Consider bot message on slack integration (#6793) 2023-03-31 18:56:51 +05:30
Sojan Jose
a4c9a2b2f2
chore: Support telegram edited_message events (#6785)
- Add support for telegram edited_message events: Update the user message back in the Chatwoot dashboard when updated by the contact on telegram
2023-03-29 18:48:16 +05:30
Pranav Raj S
eb7070d946
feat(poc): Disable widget based on country (#6658) 2023-03-14 09:09:57 -07:00
Shivam Mishra
a6405ea339
fix: migration script to run on all reporting events (#6590)
* fix: migration script to run on all reporting events

* fix: don't update user_id if it is already present

* refactor: create a new migration

* feat: update schema

* feat: ignore events with bot handoff

* feat: prefetch conversations with handoff events

* Revert "feat: update schema"

This reverts commit 25ed2856e62655f5f1db14fd0cffad3a69d0b1fb.

* feat: update schema

* refactor: separate method get_conversations_with_bot_handoffs

* refactor: cognitive complexity

* refactor: early return if last_bot_reply is blank

* feat: add async_database_migration queue

* feat: update queue priority
2023-03-02 19:03:31 +05:30
Sojan Jose
d4e7eaecce
feat: New APIs for search (#6564)
- Adding new API endpoints for search
- Migrations to add appropriate indexes
2023-02-28 22:00:36 +05:30
Shivam Mishra
06ffaa90fc
fix: bots included in time to response metrics (#6409)
* feat: ignore bots in avg_first_response_time

* feat: ignore bots in avg_first_response count

* feat: add bot handoff event

* feat: add handoff event listener and reporting event

* fix: ignore agent bot in first response

* refactor: calculate first_response with last handoff

* refactor: method defn order

* test: new reporting events

* feat: Revert "feat: ignore bots in avg_first_response count"

This reverts commit de1977c219a2e7a9180dd02272244fe3b3f7ce89.

* feat: Revert "feat: ignore bots in avg_first_response_time"

This reverts commit bb9171945d5e3b2f6015f4f96dd1b76b3efb6987.

* fix: business hour calculation for first_reply

* fix: event_start_time for first_response

* feat: add migration to recompute first_responses

* refactor: separate mute helpers for conversation

* refactor: rename migration

* refactor: migration script

* fix: migration typo

* fix: typo in query

* feat: update schema.rb

* Revert "feat: update schema.rb"

This reverts commit 353ef355f2d956dd219907bb66982dc90ca5d896.

* feat: update schema

* refactor: update events as a batch job

* fix: ignore the event if value is negative

* feat: don't create a new hand-off if it's already present

* refactor: break the action into smaller chunks

* refactor: update reporting listener spec

Handle the case to ensure extra bot handoffs are not created for a give conversation

* fix: import error

---------

Co-authored-by: Vishnu Narayanan <vishnu@chatwoot.com>
2023-02-25 09:48:48 +05:30
Sojan Jose
5cbfcfbfa0
chore: Limit conversation resolution Job (#6433)
We will be adding a limit to the resolution job so that it is performed at a spaced interval. This will ensure there won't be a sudden spike in resource usage

fixes: chatwoot/product#707
2023-02-13 14:00:52 +05:30
Tejaswini Chile
00cbdaa8ca
Feat: Support for Microsoft Oauth in Email Channel (#6227)
- Adds the backend APIs required for Microsoft Email Channels

Co-authored-by: Pranav Raj S <pranav@chatwoot.com>
Co-authored-by: Sojan <sojan@pepalo.com>
2023-01-17 02:39:05 +05:30
Tejaswini Chile
fc9fc5a661
fix: destroy bulk service (#5921) 2022-11-25 22:46:50 +05:30
Sojan Jose
8ea0660862
chore: Add reauthorization prompt for Whatsapp Channel (#5929)
- Add reauthorization prompt for Whatsapp Channel

fixes: #5782
2022-11-24 14:50:32 +03:00
Sojan Jose
e2059cfc5b
fix: SocketError: getaddrinfo: for imap channels (#5824)
fixes: #5431
2022-11-08 20:23:46 -08:00
Tejaswini Chile
0343acdb7e
fix: ensure contact_inbox if contact exists(#5667)
- Fixing Instagram issue for existing contacts in the inbox
2022-10-19 12:56:39 -07:00
Tejaswini Chile
7419e413f4
fix: Added "None" option in bulk actions assignment menu (#5585)
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav Raj S <pranav@chatwoot.com>
2022-10-11 13:44:35 -07:00
Jordan Brough
59b31615ed
chore: Use "create!" and "save!" bang methods when not checking the result (#5358)
* Use "create!" when not checking for errors on the result
* Use "save!" when not checking the result
2022-09-13 17:40:06 +05:30
Sojan Jose
7f3f6f7129
fix: Account seeder error on staging (#5371)
- Fix for Account seeding error on staging environments
2022-09-01 21:23:57 +05:30
Tejaswini Chile
9c67814724
Fix: Instagram webhook test event (#5317) 2022-08-22 16:10:14 +05:30
Sojan Jose
ecb1f57efe
chore: Improve memory usage in RemoveStaleNotificationsJob (#5271)
The job we introduced recently for removing the stale migrations would time out in instances with a large amount of deleted data.
So improving it for better memory utilization.
2022-08-15 19:58:42 +05:30
Sojan Jose
831a3bcd50
chore: Fix RemoveStaleNotificationJob query (#5242)
Rubocop amended the query in #5236, which led to failure as the Message model already has a default scope.
This PR fixes it.
2022-08-10 15:08:37 +02:00
Sojan Jose
74fdfffe08
fix: Notification page breakages (#5236)
- Remove the cascading foreign key indexes
- Add migration to clean up existing objects

fixes: #4285
2022-08-10 13:46:46 +02:00
Tejaswini Chile
6a4c0a1578
feat: Execute macro actions, for the conversation (#5066) 2022-07-26 12:41:22 +05:30
Sojan Jose
6a6a37a67b
chore: Ability to Disable Gravatars (#5027)
fixes: #3853

- Introduced DISABLE_GRAVATAR Global Config, which will stop chatwoot from making API requests to gravatar
- Cleaned up avatar-related logic and centralized it into the avatarable concern
- Added specs for the missing cases
- Added migration for existing installations to move the avatar to attachment, rather than making the API that results in 404.
2022-07-21 19:27:12 +02:00
Sojan Jose
4187428729
chore: Update dependencies to the latest versions (#5033) 2022-07-15 09:51:59 +07:00
giquieu
827f977a37
feat: Update API for contact avatar (#4719)
Added the ability to update the contact's avatar via API and Dashboard.

- Contact create and update APIs can now accept avatar attachment parameters [form data].
- Contact create and update endpoints can now accept the avatar_url parameter.[json]
- API endpoint to remove a contact avatar.
- Updated Contact create/edit UI components with avatar support

Fixes: #3428
2022-07-12 10:03:16 +02:00
Aswin Dev P.S
13a4e0e6d9
chore: Email improvements. (#4901)
* Update email processing logic.
* Fix sentry issues
2022-07-08 04:13:24 -07:00
Aswin Dev P.S
19c637eb33
chore: Fix sentry issues (#4940)
Fixes: #4810
2022-07-08 15:09:06 +05:30
Sojan Jose
a6c609f43d
feat: Support for Whatsapp Cloud API (#4938)
Ability to configure Whatsapp Cloud API Inboxes

fixes: #4712
2022-07-06 21:45:03 +02:00
Pranav Raj S
b7606e4dd2
feat: Add native support for CSML in agent_bot API (#4913) 2022-06-23 19:17:46 +05:30
Muhsin Keloth
a8c6cd729b
chore: Sync pre-chat fields after custom attribute update (#4692) 2022-06-20 14:16:49 +05:30
Sojan Jose
fdcaed75f6
chore: Move Whatsapp template sync to cron (#4858)
syncing WhatsApp templates job is moved to a cron job for a better user experience. The Templates are synced at 15-minute intervals now.
2022-06-14 23:46:36 +05:30
Pranav Raj S
6b9f4d08ca
fix: Disable marking IMAP connection as invalid for Standard Errors (#4764) 2022-06-02 11:38:58 +05:30
Sojan Jose
ab05dd9b9b
chore: Improve the reauthorization requirement for Email Channel (#4753) 2022-05-26 20:23:00 +05:30
Aswin Dev P.S
8538660bbd
Add first_reply_created event in conversation (#4576) 2022-05-18 21:16:33 -07:00
Sojan Jose
04dfb034cc
chore: Upgrade Exception tracking (#4638)
- Upgrade Sentry Libraries
- Enable provision for account and user info in error tracking
- Add ChatwootExceptionTracker

fixes: #4375
2022-05-09 14:23:19 +05:30
Sojan Jose
8d2b719dc1
chore: Sentry issues (#4623)
Fixes various issues reported on sentry

- Twilio channel creation validation errors
- Room Channel error with nil class
- Webhook Uri exception
2022-05-06 14:50:55 +05:30