Tracking Messages

Table of Contents

Introduction

Events

Webhooks

Tagging

Tracking Open Messages

Tracking Clicks

Open and Click Bot Detection

Tracking Unsubscribes

Tracking Spam Complaints

Tracking Failures

Tracking Deliveries

Stats

Introduction to Tracking Messages

Mailgun makes it easy to track your messages through Events, Stats, and Tagging.

In addition to tracking messages, Mailgun permanently stores them when they cannot be delivered due to a hard bounce (permanent failure), or when a recipient unsubscribes or marks the message as spam. In these cases, Mailgun will not attempt to deliver messages to those recipients in the future. This is to protect your sending reputation.

Accessing Data on Your Emails Mailgun provides a variety of methods to access data on your emails:

• To see every event that happened to every message, view and search Events through the Logs tab in the Control Panel. You can search by fields, like recipient, subject line, and even fields that don't use up in the Logs, such as message-id. Data is stored for at least 30 days for paid accounts, and at least 2 days for free accounts.
• Access data on Events programmability through the Events API. Data is stored for at least 30 days for paid accounts and at least 2 days for free accounts.
• View, search, and edit tables for Bounces, Unsubscribes, and Spam Complaints in the Suppressions table of the Control Panel of their retrospective APIs (Bounces API, Unsubscribes API, Complaints API). Data is stored indefinitely.
• Access statistics aggregated by tags in the Analytics tab of the Control Panel or the Stats API. Data is stored for at least six months.
• Receive notifications of events through he Webhook each time an Event happens and store the data on your side.

Enable Tracking

Event tracking is automatically enabled, with the exception for Unsubscribes , Opens , and Clicks.

You can enable Unsubscribes tracking for your domain via the Domains tab of the Control Panel. You can also manage unsubscribes per message by using unsubscribe variables. (See Tracking Unsubscribes )

You can enable Opens and Clicks tracking on two levels – per sending domain and per message.

• You can enable Open and Click tracking on a per domain basis under the Track Settings section of a particular domain's setting page.
• Tracking can also be toggled by setting o:tracking, o:tracking-clicks and o:tracking-opens parameters when sending your message. This will override the domain-level setting.

You can also disable click tracking for a specific link by including the HTML attribute, disable-tracking=true in the HTML tag of the link. With this HTML attribute in the link's HTML tag, Mailgun will not rewrite the URL.

Example : <a href="http://mailgun.com" disable-tracking=true>Mailgun</a>

Events

For every message, Mailgun keeps track of every event, for both inbound and outbound messages. The data is stored for at least 30 days for paid accounts and 2 days for free accounts.

Tracked Events

You can access Events through a few interfaces:

• Webhooks (we POST data to your configured URL(s))
• The Events API (you GET data through the API)
• The Logs tab of the Control Panel (GUI)

Webhooks

Webhooks allows you to programmatically handle events that happen with your messages sent through Mailgun. By configuring URL(s) in the Webhooks tab of the Control Panel, Mailgun can send HTTP/HTTPS POST requests to specified endpoints when certain events occur. Webhooks are configured at the domain level, enabling you to set unique endpoints for each domain via the domain drop-down selector.

Supported Webhook types:

• Accepted, Delivered, Clicks, Spam Complaints, Unsubscribes, Permanent Failures, Temporary Failures, Spam Complaints

| Note: If you want to include an HTTPS endpoint, it must be configured with a trusted CA (Certificate Authority) signed SSL certificate, not a self-signed certificate.

You can read more about the data that is posted in the appropriate section below (Tracking Opens, Tracking Clicks, Tracking Unsubscribes, Tracking Spam Complaints, Tracking Failures, Tracking Deliveries). We recommend using http://bin.mailgun.net/ for creating temporary URLs to test and debug your webhooks. | | --- |

For Webhook POSTs, Mailgun listens for the following codes from your server and reacts accordingly:

• If Mailgun receives a 200 (Success) code, it will determine the webhook POST is successful and not retried.
• If Mailgun receives a 406 (Not Acceptable) code, Mailgun will determine the POST is rejected and not retry.
• For any other code, Mailgun will retry POSTing according to the schedule below for Webhooks other than the delivery notification.

If your application is unable to process the webhook request, but you do not return a 406 error code, Mailgun will retry (other than for delivery notification) during 8 hours at the following intervals before stopping to try: 5 minutes, 10 minutes, 15 minutes, 1 hour, 2 hours, and 4 hours.

The Webhooks API endpoint allows you to programmatically manipulate the webhook URLs defines for a specific domain. See the Webhooks API for more details.

Payload

When something happens to your email, your URL will be called with application/JSON payload with the following data:

{
  “signature”:
  {
    "timestamp": "1529006854",
    "token": "a8ce0edb2dd8301dee6c2405235584e45aa91d1e9f979f3de0",
    "signature": "d2271d12299f6592d9d44cd9d250f0704e4674c30d79d07c47a66f95ce71cf55"
  }
  “event-data”:
  {
    "event": "opened",
    "timestamp": 1529006854.329574,
    "id": "DACSsAdVSeGpLid7TN03WA",
    // ...
  }
}

The 'signature' parameters are described in Securing Webhooks and the 'event-data' parameters are the same as described in Event Structure.

Securing Webhooks

To ensure the authenticity of event requests, Mailgun signs them and posts the signature alongside the webhook's event-data.

A signature takes the following form:

To verify the webhook originated from Mailgun, you will need to:

• Concatenate timestamp and token values together with no separator.
• Encode the resulting string with the HMAC algorithm, using your Webhook Signing Key as a key and SHA256 digest mode.
• Compare the resulting hexdigest to the signature. If they do not match then the webhook is not from Mailgun!
• Optionally, you can cache the token value locally and not honor any subsequent request with the same token. This will prevent replay attacks.
• Optionally, you can check if the timestamp is not too far from the current time.
  • There can be delays in webhook processing that are outside of Mailgun's control so you don't want to be too aggressive with this check.

To visualize, here's a sample NodeJS snippet that checks the Webhook signature:

const crypto = require('crypto')

const verify = ({ signingKey, timestamp, token, signature }) => {
    const encodedToken = crypto
        .createHmac('sha256', signingKey)
        .update(timestamp.concat(token))
        .digest('hex')

    return (encodedToken === signature)
}

Attaching Data to Messages

When sending messages, you can attach data for later retrieval. For instance, you can attach campaign or recipient identifiers to messages to help relate webhook payloads or events retrieved from mailgun back to marketing campaigns or individual recipients in your system.

There are two methods for attaching data to emails depending on if you're sending messages via SMTP or via API.

Attaching Data to emails via SMTP

When sending messages via SMTP, you can attach data by providing a X-Mailgun-Variables header. You can provide multiple X-Mailgun-Variables headers, their map values will be combined. The header data must be in JSON map format, as shown in the example below.

X-Mailgun-Variables: {"first_name": "John", "last_name": "Smith"}
X-Mailgun-Variables: {"my_message_id": 123}

Attaching Data to emails via API

If you are sending email via the HTTP API, you can attach data by providing a form parameter with v:

For example:

v:first_name=John
v:last_name=Smith
v:my_message_id=123

The data provided will be included in the recipient's email via a header called X-Mailgun-Variables. Additionally, the data will also be available via webhook payloads, and events returned from the events API. The data will be attached to these payloads via the user-variables field as a JSON map. For example:

 {
    "event": "delivered",
    "user-variables": {
        "first_name": "John",
        "last_name": "Smith",
        "my_message_id": "123"
    }
}

When sending batches of emails, you can use values from recipient variables to provide a custom variable per recipient using templating. For example, given a variable of v:recipient-id=%recipient.id%and a recipient variable of{"user1@example.com" : { "id": 123 }}events andwebhooks associated with the recipient user1@example.comwill contain auser-variablefield with the content of{ "recipient-id": "123" }

When using variables, the X-Mailgun-Variables header will be included in the MIME of the delivered email. This means that recipients who receive emails when variables are used will be able to see the variables if they view the MIME headers.

Tagging

Mailgun allows you to categorize outgoing email traffic by tagging each outgoing message with a custom value. When you access stats on your messages, they will be aggregated by these tags.

The application of tags is more useful when paired with Mailgun's other tracking features. Tags are unique to each send and allow you to collect data on different message distributions being sent out.

Examples:

• How many recipients opened messages with a given tag
• How many clicks on linked URLs in messages of tags.

Tagging provides the ability to review the overall performance of tags, as well as give the ability to compare one tag against another.

Tagging Code Samples

Supply one or more o:tag parameters to tag the message

curl -s --user 'api:YOUR_API_KEY' \
    https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages \
    -F from='Excited User <postmaster@YOUR_DOMAIN_NAME>' \
    -F to=recipient@example.com \
    -F subject="Hello there!" \
    -F text='Testing some Mailgun awesomeness!' \
    -F o:tag='September newsletter' \
    -F o:tag='newsletters'

Tracking Open Messages

Mailgun can keep track of every time a recipient opens a message by viewing Opens on the Logs tab. You can also see counters of opens aggregated by tags by visiting the Analytics tab on the Control Panel. In addition, you can be notified through a webhook, or get the data programmatically through the Events API.

Open tracking can be enabled in the Tracking Settings which can be found on your domain's settings page, which can be found on the Domains tab of the Control Panel. When sending a message, you can also use the parameters, o:tracking, or o:tracking-opens. Additionally, you will also have to add the appropriate CNAME records to your DNS as specified in the Domain Verification & DNS section , which is also found on your domain's settings page found on the Domains tab of the Control Panel.

Tracking Clicks

When you enable Click tracking, links will be overwritten and point to Mailgun's servers so they can track clicks. You can specify that you only want links rewritten in the HTML part of the message with the parameter o:tracking-clicks and passing HTML only.

Open Webhook

You can specify webhook URLs programmatically using the Webhooks API. When a user opens an email that you have sent, your opened URLs will be called with the following webhooks payload.

Clicks Webhook

You can specify webhook URLs programmatically using the Webhooks API. Every time a user clicks on a link inside of your messages, your clicked URLs will be called with the following webhooks payload.

Open and Click Bot Detection

Mailgun uses tracking pixels and URL redirects to track when a user opens the message and clicks links in the email. However, there are various third-party automated systems that will automatically open and message and follow the links for virus scanning and user activity obfuscation, such as Apple Mail Privacy Protection.

Because automated systems can affect the accuracy of open and click tracking, Mailgun will attempt to detect when one of the systems retrieves the tracking pixel or clicks a link. When a bot is detected opening or clicking a link in the email, Mailgun will indicate this via the client-info.bot field in the open/click events.

{
    "client-info": {
      "client-name": "unknown",
      "client-type": "unknown",
      "user-agent": "Mozilla/5.0",
      "device-type": "unknown",
      "client-os": "unknown",
      "bot": "apple"
    },
    "tags": [],
    "timestamp": 1652883435.279025,
    "recipient": "bot@apple.com",
    "geolocation": {
      "region": "Unknown",
      "country": "US",
      "city": "Unknown"
    },
    "event": "opened",
}

The bot field can have one of the possible values:

Tracking Unsubscribes

Every time a recipient requests to unsubscribe from mailings, Mailgun can keep track of it. When you enable unsubscribe tracking, Mailgun will insert unsubscribe links and remove those recipients from your mailings automatically for you.

To see unsubscribes, go to the Logs tab, or see counters of unsubscribes aggregated by tags found on the Analytics tab of the Control Panel. You can also get notifications through a webhook, or get data programmatically through the Events or Bounces API

Mailgun supports three types of unsubscribe levels: domain, tag, Mailing Lists

Auto-Handling

Mailgun will automatically prevent future emails from being sent to recipients who have unsubscribed when you enable the Unsubscribe functionality by going to the settings in your domain area on the Control Panel. You can also edit the unsubscribed address list from the Control Panel or through the API.

Unsubscribe Variables

Mailgun provides you with several unsubscribe variables:

When these variables are included in your emails, any recipient who clicks on the URL will be automatically unsubscribed, and those email addresses will be blocked from receiving future emails from that domain or message tag.

Mailgun can automatically provide a customizable unsubscribe footer in each email you send. You can edit the unsubscribe footer by editing the settings in your control panel.

To enable/disable unsubscribes programmatically, you can do the following:

• Enable the Unsubscription feature for your domain.
• Remove text in the HTML and text footer so they won't be appended automatically.
• Insert a variable in the HTML and text bodies of your email when you need unsubscribe links.
  • This variable will be replaced by the corresponding unsubscribe link.

When you go to the Suppressions tab of the Control Panel or through the API, you can also:

• View/get a list of unsubscribed addresses
• Remove an unsubscribed address from the list
• Add a new unsubscribed address

Learn how to programmatically manage lists of unsubscribed users by visiting the Unsubscribes section of the API reference.

Unsubscribe Webhook

You can specify webhook URLs programmatically using Webhooks API. When a recipient unsubscribes, Mailgun will invoke the unsubscribed webhook with the following webhooks payload.

Tracking Spam Complaints

Email service providers (ESP) are very sensitive to users clicking on spam complaint buttons. It's important to monitor that activity to maintain a good sending reputation. While not all ESP supports Feedback Loop (FBL) notifications, Mailgun makes sure that you get data on all the ones that you do, will not remove recipients from future messages if the complaint has been filed by the recipient, Mailgun automatically tracks all messages recipients report as spam.

You can view spam complaints on the control panel by clicking on the logs tab or by clicking on the Analytics tab to see counters of complaints aggregated by tags. You can also be notified through a webhook. When a recipient report one of your emails as spam, Mailgun will invoke the complained webhook with the following payload. Another way to track complaints is to get the data programmatically through the Events API or the Complaints API. You can Mailgun provides the Spam Complaint API to programmatically manage lists of users who have complained.

Tracking Failures

Mailgun tracks all delivery failures, which consists of both hard bounces (permanent failures), and soft bounces (temporary failures).

When an email message is said to "bounce", this means that it was rejected by the recipient SMTP server. Bounced addresses are found on the "Bounces" table, which is found on the Suppressions tab on the Control Panel. With respect to failure persistence, Mailgun classifies bounces into two categories:

• Hard bounces (permanent failure): This based on these criteria:
  • The recipient is not found
  • The recipient email server specifies that the recipient does not exist
• Soft bounces (temporary failure):
  • Email is not delivered because the mailbox is full or for other reasons.

Permanent and Temporary Failure Webhooks There may be a few reasons why Mailgun needs to stop trying to deliver messages. The most common reason being that Mailgun has received a hard bounce or repeatedly received soft bounces. Continually attempting to deliver the message may affect the sender's reputation with the receiving ESP. When the address is on one of the 'Do Not Send" lists because the recipient has previously bounced, unsubscribed, or reported spam, Mailgun will drop the message and stop trying to send it. If one of these events occurs, Mailgun will POST the following webhooks payload to your permanent_fail URLs. You can specify webhook URLs programmatically using the Webhook API.

With respect to when the recipient SMTP server has rejected incoming messages, Mailgun will classify bounces into the following categories:

• Immediate bounce :
  • An email message is rejected by the recipient SMTP server during the SMTP session.
• Delayed (asynchronous) bounce:
  • The recipient SMTP server accepts an email message during the SMTP session. After some time, it will send a Non-Delivery Report email message to the message sender.

Tracking Deliveries

Mailgun tracks all successful deliveries that occur when the recipient email server responds that it has accepted the message.

You can see when a message has been successfully sent by clicking on the Logs tab found on the Control Panel. You can also be notified when a message has been delivered through a webhook or get the data programmatically through the Events API.

Do note that, for messages that get Routed to a HTTP endpoint, we do not send a Delivered webhook to your configured URL(s). We do emit the delivered event for the message itself so that the Routed delivery shows up in the Events API.

Stats

Stats allow you to see a summary of the events that occur with your messages. They can be aggregated by a tag (see Tagging). To see your current statistics, go to the Control Panel Dashboard.