docs: add Meta Webhook Proxy documentation for improved WhatsApp message delivery

2026-02-16 20:35:45 -03:00 · 2026-02-16 20:35:45 -03:00 · d73ea58517
commit d73ea58517
parent 3c47ea3d43
1 changed files with 229 additions and 0 deletions
--- a/META-WEBHOOK-PROXY.md
+++ b/META-WEBHOOK-PROXY.md
@ -0,0 +1,229 @@
+# Meta Webhook Proxy
+
+## Problem
+
+Some VPS providers silently drop inbound TCP connections from Meta's webhook servers (AS32934) due to overzealous DDoS protection. This causes 15–20% WhatsApp message loss. A reverse proxy on a clean provider (e.g., DigitalOcean) eliminates the drops completely.
+
+```
+Meta (WhatsApp) → proxy.example.com (clean provider) → your Chatwoot instance
+```
+
+## Architecture
+
+The proxy is a single nginx server that routes requests based on the first path segment:
+
+```
+https://proxy.example.com/<upstream_host>/webhooks/whatsapp/%2B<phone>
+```
+
+This is the URL you configure in Meta's App Dashboard as the webhook callback URL. The proxy extracts `<upstream_host>`, checks it against an allowlist, and forwards the request to `https://<upstream_host>/webhooks/whatsapp/%2B<phone>`.
+
+### Multi-tenant
+
+One proxy serves multiple Chatwoot instances. Each upstream is identified by its domain in the URL path — no separate config per tenant beyond adding the host to the allowlist.
+
+## Setup
+
+### Prerequisites
+
+- A server (Ubuntu 22.04/24.04) on a provider with clean Meta connectivity (DigitalOcean, AWS, etc.)
+- A DNS A record pointing your proxy domain to the server IP (e.g., `proxy.example.com → 1.2.3.4`)
+- SSH root access to the server
+
+### 1. Install nginx and certbot
+
+```bash
+ssh root@proxy.example.com
+
+apt-get update
+apt-get install -y nginx certbot python3-certbot-nginx
+```
+
+### 2. Create a temporary HTTP-only config
+
+Certbot needs nginx running to perform the ACME challenge, but the full config references SSL certs that don't exist yet. Start with an HTTP-only config:
+
+```bash
+cat > /etc/nginx/sites-available/cw-proxy << 'EOF'
+server {
+    listen 80;
+    server_name proxy.example.com;
+
+    location /.well-known/acme-challenge/ {
+        root /var/www/html;
+    }
+
+    location / {
+        return 404;
+    }
+}
+EOF
+```
+
+Enable the site and reload:
+
+```bash
+rm -f /etc/nginx/sites-enabled/default
+ln -sf /etc/nginx/sites-available/cw-proxy /etc/nginx/sites-enabled/cw-proxy
+nginx -t && systemctl reload nginx
+```
+
+### 3. Obtain the SSL certificate
+
+```bash
+certbot certonly --webroot -w /var/www/html -d proxy.example.com \
+  --non-interactive --agree-tos -m your-email@example.com
+```
+
+Certbot installs a systemd timer that auto-renews the certificate before it expires.
+
+### 4. Deploy the full proxy config
+
+Replace the temporary config with the full proxy configuration:
+
+```bash
+cat > /etc/nginx/sites-available/cw-proxy << 'EOF'
+# Allowlist of upstream Chatwoot hosts
+# Add new hosts here to enable proxying
+map $upstream_host $upstream_allowed {
+    default                    0;
+    chatwoot.example.com       1;
+    # chatwoot.other.com       1;
+}
+
+server {
+    listen 80;
+    server_name proxy.example.com;
+
+    location /.well-known/acme-challenge/ {
+        root /var/www/html;
+    }
+
+    location / {
+        return 301 https://$host$request_uri;
+    }
+}
+
+server {
+    listen 443 ssl;
+    server_name proxy.example.com;
+
+    resolver 1.1.1.1 8.8.8.8 valid=300s;
+    resolver_timeout 5s;
+
+    ssl_certificate /etc/letsencrypt/live/proxy.example.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/proxy.example.com/privkey.pem;
+
+    # Extract upstream host from first path segment, proxy the rest
+    location ~ ^/([^/]+)(/.*)$ {
+        set $upstream_host $1;
+        set $upstream_path $2;
+
+        # Reject hosts not in the allowlist
+        if ($upstream_allowed = 0) {
+            return 403;
+        }
+
+        proxy_pass https://$upstream_host$upstream_path$is_args$args;
+        proxy_set_header Host $upstream_host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_ssl_server_name on;
+    }
+
+    location / {
+        return 404;
+    }
+}
+EOF
+```
+
+Test and reload:
+
+```bash
+nginx -t && systemctl reload nginx
+```
+
+### 5. Verify
+
+```bash
+# Root path → 404
+curl -s -o /dev/null -w "%{http_code}" https://proxy.example.com/
+# Expected: 404
+
+# Unknown host → 403
+curl -s -o /dev/null -w "%{http_code}" https://proxy.example.com/unknown.host/webhooks/whatsapp/test
+# Expected: 403
+
+# Allowed host → proxied (502 if upstream is unreachable, 200 if live)
+curl -s -o /dev/null -w "%{http_code}" https://proxy.example.com/chatwoot.example.com/webhooks/whatsapp/test
+# Expected: 502 or 200
+```
+
+### 6. Configure Meta webhook URL
+
+In the Meta App Dashboard, set the webhook callback URL to:
+
+```
+https://proxy.example.com/<your-chatwoot-domain>/webhooks/whatsapp/%2B<phone>
+```
+
+For example, if your Chatwoot is at `chatwoot.example.com` and the phone number is `+5511999999999`:
+
+```
+https://proxy.example.com/chatwoot.example.com/webhooks/whatsapp/%2B5511999999999
+```
+
+Meta will send both verification (GET) and delivery (POST) requests to this URL. The proxy passes them through transparently.
+
+## Adding a new upstream
+
+1. SSH into the proxy server
+2. Edit `/etc/nginx/sites-available/cw-proxy`
+3. Add the new host to the `map` block:
+   ```nginx
+   map $upstream_host $upstream_allowed {
+       default                    0;
+       chatwoot.example.com       1;
+       chatwoot.newclient.com     1;   # ← add this line
+   }
+   ```
+4. Test and reload:
+   ```bash
+   nginx -t && systemctl reload nginx
+   ```
+5. Set the Meta webhook callback URL for the new instance to:
+   ```
+   https://proxy.example.com/chatwoot.newclient.com/webhooks/whatsapp/%2B<phone>
+   ```
+
+## Removing an upstream
+
+1. Remove or comment out the host from the `map` block
+2. `nginx -t && systemctl reload nginx`
+3. Update the Meta webhook callback URL to point directly at the Chatwoot instance (or to a different proxy)
+
+## Key nginx directives
+
+| Directive | Purpose |
+|-----------|---------|
+| `map $upstream_host $upstream_allowed` | Allowlist of permitted upstream hosts. Only hosts set to `1` are proxied; all others get 403. |
+| `proxy_ssl_server_name on` | Enables SNI so the TLS handshake uses the correct hostname for the upstream's certificate. |
+| `resolver 1.1.1.1 8.8.8.8 valid=300s` | Required because `proxy_pass` uses a variable (`$upstream_host`), so nginx cannot resolve DNS at config load time. Uses Cloudflare and Google DNS. |
+| `proxy_set_header Host $upstream_host` | Sets the Host header to the upstream domain so reverse proxies (Traefik, etc.) route correctly. |
+
+## Failure modes
+
+All recoverable — Meta retries with exponential backoff for up to 36 hours:
+
+| Failure | What happens | Recovery |
+|---------|-------------|----------|
+| Proxy down | Connection refused | Meta retries |
+| Upstream down | 502 Bad Gateway | Meta retries |
+| SSL expired | TLS handshake error | Meta retries |
+
+## Important notes
+
+- **Do not rate-limit.** Meta sends webhook deliveries from many IPs in AS32934. Bursts of 10+ requests per second are normal.
+- **SSL auto-renewal** is handled by the certbot systemd timer. Verify with `systemctl status certbot.timer`.
+- The `%2B` in the URL is the URL-encoded `+` sign for the phone number's country code.