iachat/spec
Vinay Keerthi ef54f07d5b
feat: Add company backfill migration for existing contacts (Part 1) (#12657)
## Description

Implements company backfill migration infrastructure for existing
contacts. This is **Part 1 of 2** for the company model production
rollout as described in
[CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production).

Creates jobs and services to associate existing contacts with companies
based on their email domains, filtering out free email providers (gmail,
yahoo, etc.) and disposable addresses.
 

**What's included:**
- Business email detector service with ValidEmail2 (uses
`disposable_domain?` to avoid DNS lookups)
- Per-account batch job to process contacts for one account
- Orchestrator job to iterate all accounts
- Rake task: `bundle exec rake companies:backfill`

~~*NOTE*: I'm using a hard-coded approach to determine if something is a
"business" email by filtering out emails that are usually personal. I've
also added domains that are common to some of our customers' regions.
This should be simpler. I looked into `Valid_Email2` and I couldn't find
anything to dictate whether an email is a personal email or a business
one. I don't think the approach used in the frontend is valid here.~~
UPDATE: Using `email_provider_info` gem instead.


**Pending - Part 2 (separate PR):** Real-time company creation for new
contacts

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

```bash
# Run all new tests
bundle exec rspec spec/enterprise/services/companies/business_email_detector_service_spec.rb \\
                   spec/enterprise/jobs/migration/company_account_batch_job_spec.rb \\
                   spec/enterprise/jobs/migration/company_backfill_job_spec.rb

# Run RuboCop
bundle exec rubocop enterprise/app/services/companies/business_email_detector_service.rb \\
                     enterprise/app/jobs/migration/company_account_batch_job.rb \\
                     enterprise/app/jobs/migration/company_backfill_job.rb \\
                     lib/tasks/companies.rake
```

**Performance optimization:**
- Uses `disposable_domain?` instead of `disposable?` to avoid DNS MX
lookups (discovered via tcpdump analysis - `disposable?` was making
network calls for every email, causing 100x slowdown)

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-11-03 20:03:47 +05:30
..
actions fix: Disable enqueueing Avatar jobs if the URL is invalid (#12035) 2025-07-24 12:56:39 +04:00
assets feat: Move email attachments from links to file attachments (#11304) 2025-04-15 23:43:12 -07:00
builders feat: Always process email content (#12734) 2025-10-30 13:36:39 +05:30
channels fix: Move contact events to account stream rather than individual user stream (#11082) 2025-03-13 17:46:48 -07:00
config feat: move embedding config to a yaml file (#11611) 2025-05-30 16:26:40 +05:30
configs chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
controllers feat: Bulk actions for contacts (#12763) 2025-10-30 15:28:28 +05:30
dispatchers Non blocking event dispatch (#652) 2020-03-29 19:18:30 +05:30
drops feat: Add the support for custom attributes in message variables (#8511) 2023-12-08 14:13:35 -08:00
enterprise feat: Add company backfill migration for existing contacts (Part 1) (#12657) 2025-11-03 20:03:47 +05:30
factories fix: Extend phone number normalization to Twilio WhatsApp (#12655) 2025-10-28 18:16:29 +05:30
finders feat: add SKIP_INCOMING_BCC_PROCESSING as internal config (#12484) 2025-09-22 17:52:56 +05:30
fixtures feat: Add BE changes for captain pdf support for faq generation (#12113) 2025-08-27 20:31:22 +05:30
helpers chore: Refactor UTM params to stay compliant with standards (#12312) 2025-08-29 11:46:52 -07:00
integration Fix url in emails, add frontendURL helper (#19) 2019-08-25 19:59:28 +05:30
jobs feat: Bulk actions for contacts (#12763) 2025-10-30 15:28:28 +05:30
lib feat: Open conversation when agent bot webhook fails (#12379) 2025-10-13 15:59:59 +05:30
listeners feat: Add support for labels in automations (#11658) 2025-09-18 14:17:54 +05:30
mailboxes feat: add SKIP_INCOMING_BCC_PROCESSING as internal config (#12484) 2025-09-22 17:52:56 +05:30
mailers chore: Migrate mailers from the worker to jobs (#12331) 2025-10-21 16:36:37 -07:00
models chore(sidekiq): log ActiveJob class and job_id on dequeue (#12704) 2025-10-22 20:20:37 -07:00
policies chore: Enforce custom role permissions on conversation access (#12583) 2025-10-22 20:23:37 -07:00
presenters fix: Disable automations on auto-reply emails (#12101) 2025-08-05 13:17:06 +05:30
requests/api/v1 feat: MFA (#12290) 2025-09-18 20:19:24 +05:30
services feat: Bulk actions for contacts (#12763) 2025-10-30 15:28:28 +05:30
support feat: Secure external credentials with database encryption (#12648) 2025-10-13 18:05:12 +05:30
coverage_helper.rb ci(circleci): switch coverage reporting to Qlty orb (#12337) 2025-08-31 00:39:34 +05:30
rails_helper.rb fix: resolve mutex conflicts in Instagram webhook specs (#12154) 2025-08-11 23:31:25 +05:30
spec_helper.rb ci(circleci): switch coverage reporting to Qlty orb (#12337) 2025-08-31 00:39:34 +05:30
test_helper.rb ci(circleci): switch coverage reporting to Qlty orb (#12337) 2025-08-31 00:39:34 +05:30