Data Shipping

🚧

Data Shipping requires the Business or Enterprise plan.

Overview

The data shipping feature delivers click, conversion, and impression data direct to an Amazon AWS S3 bucket. This enables you to ingest large amounts of data without making requests to the Kevel reporting API. You can also perform advanced analytics on the raw data beyond what's available in the Kevel platform.

Integration Steps

❗️

You should use the CloudFormation template below to properly set up data shipping. The template will create your S3 bucket for you.

Kevel's CloudFormation template for data shipping will create:

  • An S3 bucket for your Kevel data shipping logs
  • A bucket policy authorizing Kevel to create objects in the bucket and list the objects the bucket contains. ListBucket permission is required to ensure that all data shipping files are uploaded correctly.
  1. Let your account manager know that you're interested in data shipping and which AWS region you want your S3 bucket to be in. Your account should be on the Business or Enterprise plan.
  2. Choose a name for the CloudFormation stack: STACK_NAME, and a name for the new S3 bucket: BUCKET_NAME.
  3. Get our CloudFormation template and use the AWS CLI to create the CloudFormation stack:
$ wget https://static.adzerk.net/docs/cloudformation-templates/create-datashipping-s3-bucket.yaml
$ aws cloudformation create-stack \
  --stack-name $STACK_NAME \
  --parameters ParameterKey=BucketName,ParameterValue=$BUCKET_NAME \
  --template-body file://create-datashipping-s3-bucket.yaml
  #Check output for support team
  aws cloudformation describe-stacks --stack-name $STACK_NAME  | grep Output

👍

Use brew install wget to install wget on Mac OS.

  1. Let Kevel support know when the bucket has been successfully created, or if you have run into any issues. To complete the integration, we need to know:

Info

Example

S3 bucket name (not the URL or another identifier)

mysite-kevel-rawdata

The AWS region the S3 bucket is in

us-east-1

Your Kevel network ID

12345

👍

**The actual bucket name will receive the prefix 'kevel-dataship-'
above example will produce a S3 bucket named: 'kevel-dataship-mysite-rawdata'.
Contact support if another naming convention is needed.

Data shipping will return a JSON object of an event (impression, click, or conversion). An example impression object is below.

For impressions, each ad serving engine will deliver a file approximately once per minute. They are delivered as gzipped .log files.

For clicks and events data, each gzipped file will be delivered once per day at the end of the day.

📘

If you use the AWS UI to download data shipping files from S3, they will not be gzipped, although their file extension suggests they are. This is because your browser is unzipping the file content as it is downloaded. Downloading from your S3 bucket via other methods will return a gzipped file.

JSON Object

🚧

Certain fields, such as Location, may not be included as part of the JSON at all if there is no data to associate with the fields.

{
   "Id":"e1cf155248e541ba853b2ed482efd4ed",
   "CreatedOn":"/Date(1430156219349)/",
   "AdTypeId":5,
   "AuctionBids":1,
   "BrandId":15408,
   "BurnIn":0,
   "CampaignId":22318,
   "Categories":[

   ],
   "ChannelId":7731,
   "CreativeId":57609,
   "CreativePassId":82224,
   "Datacenter":true,
   "DeliveryMode":2,
   "Device":{  
      "brandName":"Google",
      "modelName":"Chrome",
      "osRawVersion":"0",
      "osMajorVersion":0,
      "osMinorVersion":0,
      "browser":"Chrome Desktop",
      "browserRawVersion":"63.0",
      "browserMajorVersion":63,
      "browserMinorVersion":0,
      "formFactor":"desktop",
      "deviceIdentified":true
   },
   "Ecpm":0,
   "FirstChannelId":7731,
   "GdprComputed":false,
   "GdprIsEU":false,
   "GdprSource":"none",
   "ImpressionCount":1,
   "IsNoTrack":false,
   "IsTrackingCookieEvents":false,
   "IsPublisherPayoutExempt":false,
   "IsValidUA":true,
   "Keywords":"",
   "Location": {
      "CountryCode": "US",
      "CountryName": "United States",
      "Region": "VA",
      "City": "Ashburn",
      "PostalCode": "20147",
      "MetroCode": 511,
      "DmaCode": 511,
      "AreaCode": 703
    },
   "MatchingKeywords":[

   ],
   "NetworkId":4161,
   "PassId":41596,
   "PhantomCreativePassId":0,
   "PlacementName":"3423",
   "PhantomPassId":0,
   "PriorityId":19301,
   "Price":"0",
   "RateType":1,
   "Revenue":0,
   "RemoteIP":"0.0.0.0",
   "RtbBids": 4,
   "SiteId":22032,
   "TimeZoneId":"America/New_York",
   "User":{
      "Key":"ue1-9040e0cc7c6a45afbf2ca1f40a67b75c"
   },
   "UserAgent":"",
   "UserKey":"ue1-9040e0cc7c6a45afbf2ca1f40a67b75c",
   "ZoneId":0
}