Overview

Integrate Datadog with Sizemotion to automatically create incidents when your monitors trigger alerts. When a Datadog monitor enters an alert state, it will create an incident in your team workspace, assign it to the on-call engineer (optional), and include all relevant alert metadata.

Key Benefits:
  • Zero configuration needed - paste URL and go
  • Automatic incident creation from alerts
  • Route different monitors to different teams
  • Auto-assign to on-call engineers
  • Prevent duplicate incidents with alert IDs

Prerequisites

Before setting up the integration, ensure you have:

  • API Token - Created from Admin Settings → API Tokens
  • Team ID - Found in Team Settings → Team Information
  • Datadog Account - With permission to create webhooks

Setup Steps

Step 1: Get Your Webhook URL

Each team has a unique webhook URL. To find yours:

  1. Go to Team Settings in your workspace
  2. Find the Team Information section
  3. Copy your Team ID (e.g., 5)
  4. Your webhook URL will be: 
    https://sizemotion.com/api/v1/incidents/webhook/datadog/{TEAM_ID}
Multiple Teams? Each team gets its own unique webhook URL. Set up one webhook per team in Datadog to route alerts correctly.

Step 2: Create API Token

  1. Navigate to Admin Settings → API Tokens
  2. Click Create New Token
  3. Name it something descriptive (e.g., "Datadog Production")
  4. Copy the token immediately - you won't see it again!

Step 3: Configure Datadog Webhook

In your Datadog account:

  1. Go to Integrations → Webhooks
  2. Click New Webhook
  3. Configure the webhook:
    • Name: team-backend (or your team name)
    • URL: Your webhook URL from Step 1
    • Custom Headers: 
      Authorization: Bearer YOUR_API_TOKEN
  4. Leave the payload as default (no customization needed!)
  5. Click Save
💡 Pro Tip: Create one webhook per team with descriptive names like backend, frontend, devops to easily identify them in monitor configurations.

Step 4: Test the Integration

After saving, use Datadog's Test button to send a test payload. You should see a test incident created in your workspace.

Routing Alerts to Teams

Adding Webhook to Monitors

To route specific monitors to your team, edit the monitor's notification message:

{{#is_alert}}
High CPU detected on {{host.name}}

@webhook-team-backend
{{/is_alert}}

{{#is_recovery}}
CPU returned to normal on {{host.name}}
{{/is_recovery}}

The @webhook-team-backend mention triggers your webhook when the monitor alerts.

Multi-Team Alerts

Send the same alert to multiple teams by mentioning multiple webhooks:

{{#is_alert}}
Critical database issue detected!

@webhook-team-backend
@webhook-team-devops
{{/is_alert}}

Priority Mapping

Datadog priorities automatically map to incident severities:

Datadog Priority Incident Severity Description
P1 Sev 1 Critical - Service down
P2 Sev 2 High - Major degradation
P3 Sev 3 Medium - Minor issues
P4 / P5 Sev 4 Low - Informational

Common Scenarios

High CPU Alert (Backend Team)

{{#is_alert}}
⚠️ High CPU usage on {{host.name}}

CPU: {{value}}% (threshold: {{threshold}}%)
Duration: {{last_triggered_at}}

@webhook-team-backend
@slack-alerts
{{/is_alert}}

Database Connection Errors (Backend + DevOps)

{{#is_alert}}
🔴 Database connection failures detected

Service: {{service.name}}
Error Rate: {{value}}%
Monitor: {{monitor.name}}

@webhook-team-backend
@webhook-team-devops
@pagerduty-critical
{{/is_alert}}

Frontend Performance (Frontend Team)

{{#is_alert}}
⏱️ Page load time exceeded threshold

Page: {{url.path}}
Load Time: {{value}}ms (threshold: {{threshold}}ms)
Region: {{region}}

@webhook-team-frontend
{{/is_alert}}

Troubleshooting

Webhook Not Creating Incidents

Check these common issues:

  • Authorization Header: Ensure you copied the API token correctly and included "Bearer " prefix
  • Team ID: Verify the team ID in the URL matches your actual team
  • API Token Permissions: Token must have incidents:create permission
  • URL Format: Should be /api/v1/incidents/webhook/datadog/{team_id}

Duplicate Incidents

If you see duplicate incidents for the same alert:

  • Check if multiple monitors are triggering for the same issue
  • Verify you're not mentioning the webhook multiple times accidentally
  • Our system automatically prevents duplicates using Datadog's alert ID

Testing the Webhook

To test your webhook configuration:

  1. Use Datadog's Test button in the webhook settings
  2. Or manually trigger a monitor temporarily
  3. Check your workspace for the test incident
  4. Look at the incident metadata to see Datadog alert details

Getting Help

If you're still having issues:

  • Check Admin Settings → API Tokens to see if your token was last used
  • Review the API Documentation for technical details
  • Contact your workspace administrator for help
✅ You're all set! Your Datadog alerts will now automatically create incidents in your team workspace. No payload customization needed - just paste the URL and start monitoring.

Next Steps