Datadog

Elementary’s Datadog integration enables streaming audit logs and system logs directly to your Datadog account for centralized log management and monitoring.

​

Overview

When enabled, Elementary automatically streams your workspace’s audit logs (user activity logs and system logs) to Datadog using the Datadog Logs API. This allows you to:

• Centralize all logs in your Datadog dashboard
• Set up custom alerts and monitors on log events
• Correlate Elementary logs with other application logs
• Perform advanced log analysis and search
• Maintain long-term log retention in Datadog

​

Prerequisites

Before configuring log streaming to Datadog, you’ll need:

1. Datadog API Key - Your Datadog API key for authentication
   • You can find or create an API key in your Datadog account under Organization Settings > API Keys
2. Datadog Site (optional) - Your Datadog site region
   • Default: datadoghq.com (US)
   • EU: datadoghq.eu
   • US3: us3.datadoghq.com
   • US5: us5.datadoghq.com
   • AP1: ap1.datadoghq.com

​

Configuring Log Streaming to Datadog

1. Navigate to the Logs page:
   • Click on your account name in the top-right corner of the UI
   • Open the dropdown menu
   • Select Logs
2. In the External Integrations section, click the Connect button
3. In the modal that opens, select Datadog as your log streaming destination
4. Enter your Datadog configuration:
   • API Key: Your Datadog API key
   • Site (optional): Your Datadog site region (defaults to datadoghq.com if not specified)
   • Service Name (optional): Custom service name for logs in Datadog (defaults to elementary)
   • Source (optional): Custom source tag for logs (defaults to elementary-cloud)

6. Click Connect to enable log streaming
7. After connecting, you will see the Datadog integration listed in the External Integrations section. You can edit or disable the integration at any time.

​

Log Format in Datadog

Logs are sent to Datadog with the following structure:

​

User Activity Logs

Each user activity log entry includes:

• timestamp: ISO 8601 timestamp of the event
• status: Log level (info for successful actions, error for failed actions)
• message: Human-readable description of the action
• service: Service name (configurable, defaults to elementary)
• source: Source tag (configurable, defaults to elementary-cloud)
• dd.tags: Additional tags including:
  • log_type:audit
  • action:<action_name> (e.g., user_login, create_test)
  • success:<true|false>
  • user_email:<user_email>
  • env_id:<environment_id> (if applicable)
  • env_name:<environment_name> (if applicable)
• Custom attributes from the event content (JSON parsed)

​

System Logs

Each system log entry includes:

• timestamp: ISO 8601 timestamp of the event
• status: Log level (info for successful operations, error for failed operations)
• message: Human-readable description of the system event
• service: Service name (configurable, defaults to elementary)
• source: Source tag (configurable, defaults to elementary-cloud)
• dd.tags: Additional tags including:
  • log_type:system
  • action:<action_name> (e.g., dbt_data_sync_completed, alerts_sent)
  • success:<true|false>
  • env_id:<environment_id> (if applicable)
  • env_name:<environment_name> (if applicable)
• Custom attributes from the event content (JSON parsed)

​

Viewing Logs in Datadog

Once configured, logs will appear in your Datadog Log Explorer within a few seconds of being generated. You can filter logs using:

• source:elementary-cloud - All Elementary logs
• log_type:audit - User activity logs only
• log_type:system - System logs only
• action:<action_name> - Specific action types
• env_name:<environment_name> - Logs from a specific environment
• success:false - Failed operations only

​

Troubleshooting

​

Logs not appearing in Datadog

1. Verify API Key: Ensure your Datadog API key is valid and has the necessary permissions
2. Check Site Configuration: Verify you’ve selected the correct Datadog site region
3. Review Log Types: Confirm the log types you want to stream are enabled
4. Check Datadog Status: Verify your Datadog account is active and not rate-limited

​

Rate Limiting

Datadog has rate limits for log ingestion. If you’re experiencing issues:

• Check your Datadog account’s rate limits in the Usage & Billing page
• Consider filtering which log types you stream if you have high log volume
• Contact Datadog support if you need to increase your rate limits

​

Disabling Log Streaming

To disable log streaming to Datadog:

1. Navigate to the Logs page
2. In the External Integrations section, find your Datadog integration
3. Click Disable or remove the Datadog configuration
4. Confirm the action