Introduction
This guide will help you get started with the Crux Dataset Health API by Delivery ID. This API allows you to retrieve granular dataset health information about a specific Delivery ID, including metadata about each task in a delivery attempt (ingest, process, and delivery dispatch).
In this quick start guide, you’ll learn:
- How to call Crux Dataset Health information for a given Delivery ID
Prerequisites
Before you begin, ensure you have the following:
- API Key: You need an API key to authenticate your requests. Log into the Crux App, and go to Settings > API Keys menu to copy your API key. Note: you may regenerate a new API key, if needed, on the same page.
- Delivery ID: You need a deliveryId as part of the API endpoint path. You may pick one from the dataset health API response.
- Base URL: The base URL for the API https://api.cruxinformatics.com/v2/ops/health/deliveries/{deliveryId}/tasks
Key Terms to Know
- Task: one of the primary steps in a data pipeline operation (e.g. ingestion, processing, delivery)
- Delivery ID: the unique identifier for a data pipeline delivery attempt
Step 1: Authenticate Your Requests
To authenticate your requests, you need to include your API key in the request headers.
Request: GET /v2/ops/health/deliveries/{deliveryId}/tasks
Host: api.cruxinformatics.com
Authorization: Bearer YOUR_API_KEY
Step 2: Make a Request
Use the following endpoint to retrieve dataset health information for a specific delivery.
Endpoint
GET https://api.cruxinformatics.com/v2/ops/health/deliveries/{deliveryId}/tasks
Path Parameters
You need to input a deliveryId
of interest as part of the API endpoint path. The API response attributes will be based on the deliveryId
that you provide. You may pick a deliveryId
from the dataset health list API response.
Example Request
Below is an example of a request to retrieve the dataset health information for a given delivery ID.
GET
/v2/ops/health/deliveries/{deliveryId}/tasks
Host: api.cruxinformatics.com
Authorization: Bearer YOUR_API_KEY
Step 3: Handle the Response
The response contains the dataset health information for a specific delivery, including the task status, destination information, and failure reasons if applicable.
Example Response
{
"ingest": {
"status": "string",
"timestamp": "datetime",
"tasks": [
{
"taskId": "string",
"taskName": "string",
"deliveryId": "string",
"cduId": "string",
"datasetId": "string",
"frameId": "string",
"scheduledAt": "datetime",
"status": "string",
"reportedStatus": "string",
"taskStartedAt": "datetime",
"taskEndedAt": "datetime",
"shortCode": "string",
"codeDetails": "string",
"message": "string",
"details": {
"runId": "string",
"deliveryId": "string"
},
"createdAt": "datetime"
}
]
},
"normalize": {
"status": "string",
"timestamp": "string",
"tasks": []
},
"process": {
"status": "string",
"timestamp": "datetime",
"tasks": [
{
"taskId": "string",
"taskName": "string",
"deliveryId": "string",
"cduId": "string",
"datasetId": "string",
"frameId": "string",
"scheduledAt": "datetime",
"status": "string",
"reportedStatus": "string",
"taskStartedAt": "datetime",
"taskEndedAt": "datetime",
"shortCode": "string",
"codeDetails": "string",
"message": "string",
"details": {
"runId": "string",
"deliveryId": "string"
},
"createdAt": "datetime"
}
]
},
"dispatch": {
"status": "string",
"timestamp": "datetime",
"tasks": [
{
"taskId": "string",
"taskName": "string",
"deliveryId": "string",
"cduId": "string",
"datasetId": "string",
"frameId": "string",
"scheduledAt": "datetime",
"status": "string",
"reportedStatus": "string",
"taskStartedAt": "datetime",
"taskEndedAt": "datetime",
"shortCode": "string",
"codeDetails": "string",
"message": "string",
"details": {
"deliveryId": "string",
"destinationId": "string",
"destinationName": "string",
"destinationType": "string",
"subscriberOrgId": "string"
},
"createdAt": "datetime"
}
]
}
}
Conclusion
You are now ready to use the Crux Dataset Health API to retrieve and manage dataset health information by Delivery ID. If you have any questions or need further assistance, please refer to the official API documentation or contact Crux support.
Happy coding!