Meltwater API

Your Account

How to Guides

API Reference

Help

Exports - API Quickstart

In this tutorial we'll cover the basics of creating an export using the API.

Before you start

To run through this tutorial, you will need:

  • Your Meltwater API key - you can find this on the API Credentials page
  • A Saved Search in your Meltwater App account

What we will cover

  1. Authentication
  2. Obtaining a Meltwater search
  3. Creating a one-time export
  4. Creating a recurring export

Types of export

There are two types of exports:

  • One-time exports - these exports are run once, the data will not be refreshed automatically.
  • Recurring exports - these exports are run on a schedule you specify. Each time the export runs the data for the export is overwritten.

1. Authentication

You need to provide your API key when you call any Meltwater API endpoint.

You provide the API key as a header called apikey, for example:

curl -X GET \
  --url "https://api.meltwater.com/v3/searches" \
  --header "Accept: application/json" \
  --header "apikey: **********"

For full details take a look at the Authentication page.

2. Obtaining a Meltwater search

Exports are created with an existing Meltwater search.

In this tutorial we'll cover using an existing search, but you can create / edit searches using the API. See the Managing Saved Searches guide for more details.

To create an export you need to provide the id of the required search. You can use the GET /v3/searches endpoint to list your current searches:

curl -X GET \
  --url "https://api.meltwater.com/v3/searches" \
  --header "Accept: application/json" \
  --header "apikey: **********"

Example response:

{
  "searches": [
    {
      "updated": "2018-01-10T14:42:10.000Z",
      "type": "news",
      "search_id": 2382415,
      "name": "Elon Musk",
      "id": 2382415
    }
  ]
}

3. Creating a one-time export

One-time exports are created using the POST /v3/exports/one-time endpoint. You need to provide a start_date, end_date and a search_id to create an export.

Note that times are provided in UTC timezone. This is required for one-time exports.

curl -X POST \
  --url "https://api.meltwater.com/v3/exports/one-time" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --header "apikey: **********" 
  --data "{
  \"onetime_export\": {
    \"search_ids\": [<search_id>],
    \"start_date\": \"2018-09-01T01:00:00\",
    \"end_date\": \"2018-10-01T01:00:00\"
  }
}"

Example response:

{
  "onetime_export": {
    "updated_at": "2018-10-04T08:38:02.034866",
    "status_reason": "Export run has not completed yet",
    "status": "PENDING",
    "start_date": "2018-09-01T00:00:00.000000Z",
    "searches": [
        {
            "id": <search_id>,
            "name": <search_name>
        }
    ],
    "tags": [],
    "inserted_at": "2018-10-04T08:38:02.034858",
    "id": <export_id>,
    "end_date": "2018-10-01T00:00:00.000000Z",
    "data_url": <data_url>,
    "company_name": <company_name>
  }
}

Fetching export results

You can check the status of a one-time export using the GET /v3/exports/one-time/<export_id> endpoint.

curl -X GET \
  --url "https://api.meltwater.com/export/v3/exports/one-time/<export_id>" \
  --header "Accept: application/json" \
  --header "apikey: **********"

Example response:

{
  "onetime_export": {
    "updated_at": "2018-10-04T08:38:02.034866",
    "status_reason": "Export run has not completed yet",
    "status": "PENDING",
    "start_date": "2018-09-01T00:00:00.000000Z",
    "searches": [
        {
            "id": <search_id>,
            "name": <search_name>
        }
    ],
    "tags": [],
    "inserted_at": "2018-10-04T08:38:02.034858",
    "id": <export_id>,
    "end_date": "2018-10-01T00:00:00.000000Z",
    "data_url": "https://exports.meltwater.com/v3/one-time/<export_id>?data_key=6d0cee74-593b-4b1c-91c3-4a7227224dfd",
    "company_name": <company_name>
  }
}

Once the status is FINISHED there will be results in JSON format at the data_url. If the status is still PENDING, the data_url will still be available, but just contain an empty list of documents.

Export execution time
The size of an export depends on how many results the search generates, and how large time window it covers. The larger the export, the longer it will take to generate - this can vary from a minute for small exports up to an hour for very large exports.

4. Creating a recurring export

Recurring exports run on a schedule you specify. Each time the schedule runs it overwrites the data provided at the data_url.

Specifying a time window and schedule

When you create a recurring export you have a number of parameters you can use to control the schedule and the period of data each run should include.

The window_time_unit parameter sets the frequency of the schedule, you can choose:

  • DAY - exports are run daily
  • WEEK - exports are run weekly
  • MONTH - exports are run monthly

The window_size allows you to specify the period of data to include in each export. Think of this value as multiple of the window_time_unit you chose. For example, specifying window_time_unit as DAY and window_size as 2 will set a recurring export to run every day which will include the last 2 days of data in each run.

You can use the following parameters to specify precisely the window of data included for each run:

  • For daily exports you can specify the daily start time with window_time
  • For weekly exports you can specify the day of the week with window_weekday and start time with window_time
  • For monthly exports you can specify the day of the month with window_monthday and start time with window_time

You can use the timezone parameter to specify the timezone for your window_time. The timezone must be a valid zone as detailed in the IANA database.

As a full example, the following parameters create a recurring export that will run every day at 09:00 UTC including the last 7 days of data.

{
    "window_time_unit": "DAY",
    "window_size": 7,
    "window_time": "09:00",
    "timezone": "Etc/UTC"
}

Default values for reccuring export attributes are as follows:

  • window_time: "00:00:00"
  • window_weekday: 1 (Monday)
  • window_monthday: 1
  • timezone: Etc/UTC
Scheduling of recurring exports
Note that exports are executed by the platform 30 minutes after the end time of the required export period. This is to allow data to be ingested by the platform from providers.

Creating a recurring export

Recurring exports are created using the POST /v3/exports/recurring endpoint.

curl -X POST \
  --url "https://api.meltwater.com/v3/exports/recurring" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --header "apikey: **********" \
  --data "{
  \"export\": {
    \"search_ids\": [<search_id>],
    \"window_time_unit\": \"DAY\",
    \"window_time\": \"09:00:00\",
    \"window_size\": 7,
    \"timezone\": \"Etc/UTC\"
  }
}"

Example response:

{
  "recurring_export": {
    "window_time_unit": "DAY",
    "window_time": "00:00:00.000000",
    "window_size": 1,
    "updated_at": "2018-10-04T08:19:11.673139",
    "timezone": "Etc/UTC",
    "status_reason": "Export run has not completed yet",
    "status": "PENDING",
    "searches": [
          {
            "id": <search_id>,
            "name": <search_name>
          }
    ],
    "tags": [],
    "next_run_date": "2018-10-05T01:00:00.000000Z",
    "inserted_at": "2018-10-04T08:19:10.002973",
    "id": <export_id>,
    "data_url": <data_url>,
    "company_name": <company_name>
  }
}

Fetching export results

You can check the status of a recurring export using the GET /v3/exports/recurring/<export_id> endpoint.

curl -X GET \
  --url "https://api.meltwater.com/v3/exports/recurring/<export_id>" \
  --header "Accept: application/json" \
  --header "apikey: **********"

Example response:

{
  "recurring_export": {
    "window_time_unit": "DAY",
    "window_time": "00:00:00.000000",
    "window_size": 1,
    "updated_at": "2018-10-04T08:19:11.673139",
    "timezone": "Etc/UTC",
    "status_reason": "Export run has not completed yet",
    "status": "PENDING",
    "searches": [
        {
            "id": <search_id>,
            "name": <search_name>
        }
    ],
    "tags": [],
    "next_run_date": "2018-10-05T01:00:00.000000Z",
    "inserted_at": "2018-10-04T08:19:10.002973",
    "id": <export_id>,
    "data_url": "https://exports.meltwater.com/v3/exports/recurring/<export_id>?data_key=daa25136-a8be-41a7-82aa-548988790f8d",
    "company_name": <company_name>
  }
}

Once the status is ACTIVE there will be results in JSON format at the data_url. If the status is still PENDING, the data_url will still be available, but just contain an empty list of documents.

The first time an export is run for a recurring export the data is available at data_url. For subsequent runs the data will override previous results at the same data_url.