iachat/app/services
Vinay Keerthi f2eaa845dc
fix: Preserve multiple consecutive newlines in text-based messaging channels (#13032)
## Description

This PR fixes an issue where multiple consecutive newlines (blank lines
for visual spacing) were being collapsed in text-based messaging
channels like WhatsApp, Instagram, and SMS.

When users send messages via API with intentional spacing using multiple
newlines (e.g., `\n\n\n\n`), the markdown renderer was following
standard Markdown spec and collapsing them into single blank lines.
While this is correct for document formatting, messaging platforms like
WhatsApp and Instagram support and preserve multiple blank lines for
visual spacing.

The fix adds preprocessing to preserve multiple consecutive newlines
(3+) by converting them to placeholder tokens before CommonMarker
processing, then restoring the exact number of newlines in the final
output.

## Changes

- Added `preserve_multiple_newlines` and `restore_multiple_newlines`
helper methods to `MarkdownRendererService`
- Updated `render_whatsapp` to preserve multiple consecutive newlines
- Updated `render_instagram` to preserve multiple consecutive newlines
- Updated `render_plain_text` (affects SMS, Twilio SMS, Twitter) to
preserve multiple consecutive newlines
- Updated `render_line` to preserve multiple consecutive newlines
- HTML-based renderers (Email, Telegram, WebWidget) remain unchanged as
they handle spacing via HTML tags

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update

## How Has This Been Tested?

Added comprehensive test coverage:
- 3 new tests for multi-newline preservation across WhatsApp, Instagram,
and SMS channels
- All 56 tests passing (up from 53)

Testing scenarios:
- Single newlines preserved: `"Line 1\nLine 2"` remains `"Line 1\nLine
2"`
- Multiple newlines preserved: `"Para 1\n\n\n\nPara 2"` remains `"Para
1\n\n\n\nPara 2"`
- Standard paragraph breaks (2 newlines) work as before
- Markdown formatting (bold, italic, links) continues to work correctly
- Backward compatibility maintained for all channels

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
2025-12-09 17:45:27 +05:30
..
account fix: account email validation during signup (#11307) 2025-05-20 20:45:39 -07:00
auto_assignment feat: Assignment service (v2) (#12320) 2025-11-17 10:08:25 +05:30
automation_rules feat: Add support for labels in automations (#11658) 2025-09-18 14:17:54 +05:30
base chore: Add cache to improve widget performance (#11163) 2025-03-24 16:04:49 -07:00
contacts feat: outbound voice call essentials (#12782) 2025-11-24 17:47:00 -08:00
conversations feat: APIs to assign agents_bots as assignee in conversations (#12836) 2025-11-18 18:20:58 -08:00
crm feat: save timezone from leadsquared API (#11583) 2025-05-28 09:46:59 +05:30
data_import fix: downcase email before finding (#8921) 2024-02-21 18:51:00 +05:30
email chore: Migrate mailers from the worker to jobs (#12331) 2025-10-21 16:36:37 -07:00
email_templates chore: Fix emails being sent with the wrong translations (#2236) 2021-06-08 22:45:01 +05:30
facebook feat: hide CSAT survey URLs from agents in dashboard (#11622) 2025-06-11 23:39:47 +05:30
geocoder fix: ip-lookup database lazy loading for all environments (#8052) 2024-10-22 23:18:30 -07:00
google feat: add Google Email fetch and OAuth token refresh service (#9603) 2024-06-11 14:22:03 +05:30
imap feat: add Google Email fetch and OAuth token refresh service (#9603) 2024-06-11 14:22:03 +05:30
instagram fix: Handle Instagram API error codes properly in message processing (#13002) 2025-12-04 18:53:50 +05:30
internal feat: add job to remove stale contacts and contact_inboxes (#11186) 2025-03-28 12:18:39 +05:30
labels fix: Update associations when a label is updated (#3046) 2021-09-21 10:16:32 +05:30
line fix: Ensure messages go to correct conversation when receive multi user in 1 LINE webhook (#12322) 2025-09-22 17:05:25 +05:30
linear feat: Add activity messages for linear actions (#11654) 2025-06-13 11:57:11 +05:30
liquid feat: Add liquid processing for SMS campaigns (#10981) 2025-06-11 13:16:44 -04:00
llm_formatter feat: Differentiate bot and user in the summary (#12801) 2025-11-05 11:42:21 -08:00
macros feat: Add webhook event support for macros (#11235) 2025-04-02 20:26:55 -07:00
mailbox refactor: strategy pattern for mailbox conversation finding (#12766) 2025-11-10 20:47:18 +05:30
message_templates fix: Disable automations on auto-reply emails (#12101) 2025-08-05 13:17:06 +05:30
messages fix: Preserve multiple consecutive newlines in text-based messaging channels (#13032) 2025-12-09 17:45:27 +05:30
mfa feat: MFA (#12290) 2025-09-18 20:19:24 +05:30
microsoft feat: add Google Email fetch and OAuth token refresh service (#9603) 2024-06-11 14:22:03 +05:30
notification chore: Handle WebPush rate limiting in push notification service (#12184) 2025-08-13 13:32:22 +05:30
sms fix: Error shouldn't halt the campaign for entire audience (#11980) 2025-08-11 12:03:48 +05:30
telegram fix: stream attachment handling in workers (#12870) 2025-12-05 13:02:53 -08:00
twilio fix: Extend phone number normalization to Twilio WhatsApp (#12655) 2025-10-28 18:16:29 +05:30
twitter feat: hide CSAT survey URLs from agents in dashboard (#11622) 2025-06-11 23:39:47 +05:30
whatsapp fix: Prevent WhatsApp campaign duplicate messages by updating status immediately (#12955) 2025-11-26 15:25:49 +05:30
widget feat: MFA (#12290) 2025-09-18 20:19:24 +05:30
account_deletion_service.rb feat: automate account deletion (#11406) 2025-05-23 12:58:13 +05:30
action_service.rb chore: Add open conversation option (#11828) 2025-07-02 10:01:50 +05:30
base_refresh_oauth_token_service.rb feat: add Google Email fetch and OAuth token refresh service (#9603) 2024-06-11 14:22:03 +05:30
base_token_service.rb feat: MFA (#12290) 2025-09-18 20:19:24 +05:30
csat_survey_service.rb fix: Send CSAT survey only when agent can reply in conversation (#11637) 2025-06-11 22:45:32 +05:30
filter_service.rb chore: Ability to filter conversations with priority (#10967) 2025-03-03 16:38:22 +05:30
ip_lookup_service.rb feat(poc): Disable widget based on country (#6658) 2023-03-14 09:09:57 -07:00
search_service.rb feat(ee): Setup advanced, performant message search (#12193) 2025-08-28 10:10:28 +05:30