Skip to content

Pagination

The Connect API exposes many endpoints that provide a collection of resources. Endpoints like /shipments or /locations/sites, even /customers tend to grow over time.

In most cases, it does not make sense to return all the entries for these endpoints. A client probably is not interested in Shipments that were completed years ago.

The best approach is to provide a means to paginate over the data and let the client decide how and how many items to retrieve.

In this document, you will learn:

  • The approach the Connect API uses for pagination.
  • How the data should be provided to clients.
  • Any special cases in how the data is provided.

⚠️ Pagination in the Connect API is a work in progress and might not be supported by all implementations. This document serves as both a guide for implementations to support pagination, and a best practices guide for clients. The specifics of each implementation should be consulted.

Pagination Basics

Every collection endpoint in the Connect API must support direct request for the collection resource (GET /entities). If pagination is supported in the collection, the implementation should respond with:

  • A first page with a default limit of items.
  • Information about the request for the next, and potentially previous, pages.

The following example shows the response payload for an example request GET /entities:

{
    "self": "/entities?limit=100",
    "next": "/entities?offset=100&limit=100",
    "items": [
        {
            "id": "1",
            "self": "/entities/1"
        },
        {
            "id": "2",
            "self": "/entities/2"
        },
        {
            "id": "3",
            "self": "/entities/3"
        }
        ...
    ]
}

The response contains a self property that indicates the exact request used to retrieve the returned items (/entities?limit=100). It also provides the request that can be used to return the next page using offset and limit query parameters, e.g. /entities?offset=100&limit=100.

This example shows that the default page limit for the implementation is 100 items. It also shows that a limit query parameter is used to limit the number of items in the response, and that an offset parameter can be used to access a specific page.

⚠️ The offset is not based on the ID of an entity. IDs can be numeric but they are not treated as numbers.

Let's assume that a client retrieves the second page based on the information from the first response (/entities?offset=100&limit=100):

This returns the second page based on our example, and the response will look as follows:

{
    "self": "/entities?offset=100&limit=100",
    "next": "/entities?offset=200&limit=100",
    "previous": "/entities?offset=0&limit=100",
    "items": [
        {
            "id": "101",
            "self": "/entities/101"
        },
        {
            "id": "102",
            "self": "/entities/102"
        },
        {
            "id": "103",
            "self": "/entities/103"
        }
        ...
    ]
}

The response introduces the previous property that provides the information to retrieve the previous page. That information contains the offset and limit for the previous page.

ℹ️ An implementation might decide to omit the offset property on the first page.

A client can decide to use its own page size by providing different values for the limit and offset parameters. This gives control to a client to retrieve pages more efficiently:

GET /entities?limit=300 # Initial call sets the limit to 300, offset is assumed to be 0
GET /entities?limit=300&offset=300 # Next page is at offset 300 (0 + 300)
GET /entities?limit=300&offset=600 # Next page is at offset 600 (300 + 300)
GET /entities?limit=300&offset=900 # and so on...

The client in the example requests 300 items per page and handles the offset by incrementing it by that number.

⚠️ Requesting an offset, limit, or offset + limit that goes beyond the number of records is not an error, and the response should include the partial data that can be retrieved. If there's no data in the requested range, an empty array should be returned.

Stopping Pagination

The implementation must provide information in the response when the last page is reached. This should be done by setting the next property to an empty string:

{
    "self": "/entities?offset=3400&limit=200",
    "next": "",
    "items": [
        {
            "id": "3401",
            "self": "/entities/3401"
        },
        {
            "id": "3402",
            "self": "/entities/3402"
        },
        {
            "id": "3403",
            "self": "/entities/3403"
        }
        ...
    ]
}

The response still provides a next property, but it is set to an empty string to indicate that there is no additional data.

⚠️ The value of the next property should be set to an empty string, not null and should not be omitted (see the API General Guidelines for explanation).

Collection Items Order

The order of the collection items is implementation-dependent. An implementation that uses numeric IDs for their resources might decide to return them in numerical order where another implementation might store a "created" date and use that for sorting.

Clients should not rely on any of these properties to make decisions about how items should be processed. Different implementations might decide to use a different approach.

Implementations, for the most part, should return items in the order in which they are fetched from the database. The order in which items were created is a good candidate, as well as an intuitive sort order depending on the type of resource.

The Connect API specification does not dictate how items in a collection are returned except for a few cases in which the sort order on the items will help client implementations.

Shipments API

The /shipments endpoint returns information about the shipments/orders in the system. This information grows on a daily basis and loses relevance as those orders are completed and days go by.

Paginating the shipments in ascending order by the date they were created will force most clients to navigate through all the pages to retrieve the items they need. Because of this, the /shipments collection endpoint should return the items in reverse order, so that the most recent items are returned first.

Conclusion

The Connect API exposes multiple resource collection endpoints. Some of these endpoints might have a significant amount of data that can affect performance. To alleviate this problem, the Connect API specifies the approach to pagination.

Pagination is driven by two query parameters to the collection endpoints named limit and offset. Clients can specify a limit and an offset to indicate the number of items and the location of those items within the collection.

Collections that support pagination should handle a default limit and only return a default number of items if the endpoint is called without a specified size. The response must provide information about this limit in the self property, as well as next and previous properties to help clients with navigation through pages.

Most collection endpoints should provide the items sorted by a property that makes sense, like the ID or the date the resource was created. Some collections will work better if the sort order is modified, so clients can avoid fetching all the records in the collection.