API Reference

Searches the dataset aggregated health. This includes that of the latest delivery as well as historical records.

Introduction

This guide will help you get started with the Crux Dataset Health API. This API allows you to retrieve the list of datasets in your organization and their respective health information based on latest delivery details.
In this quick start guide, you’ll learn:

  • How to retrieve the dataset health list for your organization

Prerequisites

Before you begin, ensure you have the following:

API Key: You need an API key to authenticate your requests.

  1. Log into the Crux App
  2. Go to Settings > API Keys to copy your API key. Note: you may regenerate a new API key if needed on the same page.

Base URL: The base URL for the API https://api.cruxinformatics.com/v2/ops/health/datasets/deliveries

Step 1: Authenticate Your Requests

To authenticate your requests, you need to include your API key in the request headers.

Request:GET /v2/ops/health/datasets/deliveries
Host: api.cruxinformatics.com
Authorization: Bearer YOUR_API_KEY

Step 2: Make a Request

Use the following endpoint to retrieve the dataset health list.

Endpoint

GET https://api.cruxinformatics.com/v2/ops/health/datasets/deliveries

Query Parameters

You can use the following query parameters to filter and paginate the results.

  • pageSize: Number of items per page (default: 50)
  • pageToken: Token for pagination
  • filter: String value to filter results
  • sort: Field to sort results by
  • latestDeliveriesCount: Number of latest deliveries to retrieve
?pageSize=<NUM>&pageToken=<TOKEN HASH>

Filter definitions

  • All filters are joined logically via an AND operator.
  • Filter Strings are formed by joining with Dots(.)
    • For example - datasetName.like.*SampleDataset*
Field IDAllowed Filter TypesDescription
legacyDatasetIdeq, neq, inThe unique identifier for the v1 dataset
datasetIdeq, neq, in, like, ilikeThe unique identifier for the current version's dataset
datasetNameeq, neq, in, like, ilikeThe dataset name
deliveryStatuseq, neq, inThe latest delivery status
dataProducteq, neq, in, like, ilikeThe data product name
supplierIdeq, neq, in, like, ilikeThe unique identifier for the supplier
deliveryIdeq, neq, inThe unique identifier for the delivery attempt

Allowed Sort Fields

  • latestDeliveryTimestamp
  • latestDeliveryStatus (order: FAILED, MISSING, DELIVERED_LATE, HEALTHY)

Example Request

Below is an example of a request to retrieve the dataset health list with filters and pagination.

Request: GET /v2/ops/health/datasets/deliveries?pageSize=50&pageToken=YOUR_PAGE_TOKEN&filter=datasetName.like.*SampleDataset*&sort=latestDeliveryTimestamp HTTP/1.1
Host: api.cruxinformatics.com
Authorization: Bearer YOUR_API_KEY

Step 3: Handle the Response

The response contains the dataset health information, including the legacy dataset ID, dataset name, delivery events, and the latest delivery status.

Example Response

{
  "items": [
    {
      "legacyDatasetId": "string",
      "datasetId": "string",
      "datasetName": "string",
      "cduId": "string",
      "cduName": "string",
      "dataProductId": "string",
      "dataProductName": "string",
      "ownerOrgId": "string",
      "supplier": {
        "id": "string",
        "name": "string"
      },
      "currentProcessingDeadlineConfig": {
        "minute": "string",
        "hour": "string",
        "dayOfTheMonth": "string",
        "month": "string",
        "dayOfWeek": "string",
        "year": "string",
        "timezone": "string",
        "cronExpression": "string"
      },
      "deliveryId": "string",
      "deliveryEventList": [
        {
          "status": "string",
          "deliveryDeadline": "datetime",
          "deliveredAt": "datetime",
          "processingDeadlineConfig": {
            "minute": "string",
            "hour": "string",
            "dayOfTheMonth": "string",
            "month": "string",
            "dayOfWeek": "string",
            "year": "string",
            "timezone": "string",
            "cronExpression": "string"
          },
          "deliveryId": "string"
        }
      ],
      "latestDeliveryStatus": "string",
      "latestDeliveryTimestamp": "datetime",
      "nextExpectedDeliveryTimestamp": "datetime"
    }
  ],
  "nextPageToken": "string",
  "previousPageToken": "string",
  "kind": "string"
}

Conclusion

You are now ready to use the Crux Dataset Health API to retrieve and manage dataset health information for your organization. If you have any questions or need further assistance, please reach out to Crux support!

Happy coding!


Language
Credentials
Bearer
Click Try It! to start a request and see the response here!