iachat/lib
Vinay Keerthi ef54f07d5b
feat: Add company backfill migration for existing contacts (Part 1) (#12657)
## Description

Implements company backfill migration infrastructure for existing
contacts. This is **Part 1 of 2** for the company model production
rollout as described in
[CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production).

Creates jobs and services to associate existing contacts with companies
based on their email domains, filtering out free email providers (gmail,
yahoo, etc.) and disposable addresses.
 

**What's included:**
- Business email detector service with ValidEmail2 (uses
`disposable_domain?` to avoid DNS lookups)
- Per-account batch job to process contacts for one account
- Orchestrator job to iterate all accounts
- Rake task: `bundle exec rake companies:backfill`

~~*NOTE*: I'm using a hard-coded approach to determine if something is a
"business" email by filtering out emails that are usually personal. I've
also added domains that are common to some of our customers' regions.
This should be simpler. I looked into `Valid_Email2` and I couldn't find
anything to dictate whether an email is a personal email or a business
one. I don't think the approach used in the frontend is valid here.~~
UPDATE: Using `email_provider_info` gem instead.


**Pending - Part 2 (separate PR):** Real-time company creation for new
contacts

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

```bash
# Run all new tests
bundle exec rspec spec/enterprise/services/companies/business_email_detector_service_spec.rb \\
                   spec/enterprise/jobs/migration/company_account_batch_job_spec.rb \\
                   spec/enterprise/jobs/migration/company_backfill_job_spec.rb

# Run RuboCop
bundle exec rubocop enterprise/app/services/companies/business_email_detector_service.rb \\
                     enterprise/app/jobs/migration/company_account_batch_job.rb \\
                     enterprise/app/jobs/migration/company_backfill_job.rb \\
                     lib/tasks/companies.rake
```

**Performance optimization:**
- Uses `disposable_domain?` instead of `disposable?` to avoid DNS MX
lookups (discovered via tcpdump analysis - `disposable?` was making
network calls for every email, causing 100x slowdown)

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-11-03 20:03:47 +05:30
..
action_view/template/handlers feat: Customisable Email Templates (#1095) 2020-08-06 15:21:06 +05:30
assets Initial Commit 2019-08-14 15:18:44 +05:30
custom_exceptions feat: Add BE changes for captain pdf support for faq generation (#12113) 2025-08-27 20:31:22 +05:30
events feat: Add support for realtime-events in copilot-threads and copilot-messages (#11557) 2025-05-22 22:25:05 -07:00
filters fix: Use case sensitive filter for phone_numbers (#12470) 2025-09-19 12:39:17 +05:30
integrations fix: Handle video file types in Slack file shares (#12630) 2025-10-09 21:05:53 +05:30
linear feat: Add user attribution to Linear integration with actor authorization (#11774) 2025-07-01 16:49:26 +05:30
redis feat: integrate LeadSquared CRM (#11284) 2025-04-29 09:14:00 +05:30
seeders feat: Overview heatmap improvements (#12359) 2025-10-13 19:15:57 +05:30
tasks feat: Add company backfill migration for existing contacts (Part 1) (#12657) 2025-11-03 20:03:47 +05:30
test_data chore: Generate test data for bulk insertion (#11229) 2025-05-06 11:13:11 +05:30
webhooks feat: Open conversation when agent bot webhook fails (#12379) 2025-10-13 15:59:59 +05:30
base_markdown_renderer.rb feat: support image height in markdown rendering of messages (#8177) 2023-11-02 13:51:54 -07:00
chatwoot_app.rb feat(ee): Setup advanced, performant message search (#12193) 2025-08-28 10:10:28 +05:30
chatwoot_captcha.rb feat: Add hCaptcha for public forms (#4017) 2022-02-18 20:02:50 +05:30
chatwoot_exception_tracker.rb fix: modify exception tracker to log even if sentry configured (#7563) 2023-07-21 11:58:49 +03:00
chatwoot_hub.rb chore: Add submenu for super admin settings (#11860) 2025-07-15 21:28:39 -07:00
chatwoot_markdown_renderer.rb feat: Adds backend support for rendering tables in articles (#9526) 2024-05-24 08:44:01 +05:30
config_loader.rb chore: Add display manifest to whitelabel settings (#8708) 2024-01-16 09:50:23 +04:00
current.rb Automation enhancement (#4087) 2022-03-21 13:12:27 +05:30
custom_markdown_renderer.rb feat: move embedding config to a yaml file (#11611) 2025-05-30 16:26:40 +05:30
dyte.rb feat: Upgrade Dyte apis to v2 (#10706) 2025-02-19 14:47:48 -08:00
exception_list.rb Fix: prevent IMAPBadResponse exception from sending the authorization mail (#7154) 2023-05-22 13:51:56 +05:30
global_config_service.rb feat: move Slack config to installation settings (#11548) 2025-05-23 01:07:35 -07:00
global_config.rb chore: Add display manifest to whitelabel settings (#8708) 2024-01-16 09:50:23 +04:00
limits.rb feat: Add company model and API with tests (#12548) 2025-10-08 07:53:43 -07:00
linear.rb feat: Add user attribution to Linear integration with actor authorization (#11774) 2025-07-01 16:49:26 +05:30
microsoft_graph_auth.rb Revert "feat: Support Azure single-tenant application using the Graph… (#7436) 2023-06-29 16:50:18 -07:00
online_status_tracker.rb fix: Get online status from db when not present in cache [CW-3233] (#9477) 2024-05-15 21:23:19 -07:00
open_ai_constants.rb feat: Add BE changes for captain pdf support for faq generation (#12113) 2025-08-27 20:31:22 +05:30
regex_helper.rb fix: Escape closing bracket in mention regex (#11877) 2025-07-04 10:35:11 +05:30
test_data.rb chore: Generate test data for bulk insertion (#11229) 2025-05-06 11:13:11 +05:30
url_helper.rb fix: Referer URL validation (#4309) 2022-03-30 14:36:22 +05:30
vapid_service.rb chore: Switch to web-push gem (#6390) 2023-02-03 18:55:22 +05:30