Limiting Query Results and Pagination

Provides information about Limiting Query Results and Pagination.

For large data sets,e.g. devices, groups and functional items, limiting the amount of data returned is important from a band-width standpoint. But it is also important from a UI processing standpoint as a UI often can only display a small portion of a huge data set. In cases where the data set grows indefinitely, it is helpful to limit the amount of data returned by default. Aside from limiting the amount of data returned, we also need to consider how to "page" or scroll through that large data set if more than that first subset needs retrieval. This is referred to as pagination creating "pages" of data, returning known sections of a larger list and being able to page "forward" and "backward" through that large data set. It is recommended that paging for collection resources is enabled by default, with offset=0 and limit=20.

Paging

Paging may be supported via Range headers or via query parameters. Regardless of the approached used by the client to limit the collection of resources retrieved, the resources that represent collections shall conform to some common schema enabling pagination by including offset and limit fields and navigational links to ease the traversing the returned subset of resources. The following common collection resource structure is recommended for enabling collection resource pagination:

Field	Type	Resolution	Description
href	Link	mandatory	Link to this collection resource. URIs for querying and filtering of the collection of resources represented by this resource can easily be constructed by appending the query parameters to this link.
offset	Integer	optional	Offset in the collection of resources where the currently retrieved subset of resource representations starts from. The value shall be the same as the offset defined in the pagination constraint. This field is omitted in case the underlying collection resource contains no entries.
limit	Integer	optional	Maximum number of entries in the currently retrieved subset of resource representations. The value shall be the same as the offset defined in the pagination constraint. This field is omitted in case the underlying collection resource contains no entries.
first	Link	optional	Link to the first set of resource representations that satisfy the filtering and limitation constraints for the collection. The size of the set and offset in the resulting collection are provided via pagination rules. In case there are no resource representations to satisfy the constraints this field is omitted.
previous	Link	optional	Link to the previous set of resource representations that satisfy the filtering and limitation constraints for the collection given as query parameters. The size of the set and offset in the resulting collection are provided via pagination rules. In case there are no resource representations to satisfy the constraints then this field is omitted.
next	Link	optional	Link to the next set of resource representations that satisfy the filtering and limitation constraints for the collection. The size of the set and offset in the resulting collection are provided via pagination rules. In case there are no resource representations to satisfy the constraints then this field is omitted.
last	Link	optional	Link to the last set of resource representations that satisfy the filtering and limitation constraints for the collection. The size of the set and offset in the resulting collection are provided via pagination rules. In case there are no resource representations to satisfy the constraints then this field is omitted.
entries	Array	optional	Collection of resource representations that satisfy the filtering and limitation constraints given as query parameters. In case there are no resource representations to satisfy the constraints then this field is omitted.

Limiting Responses via Range Header

Clients can request paginated responses via the HTTP Range header which shall specify a range of the collection resource's entries. The usage of the HTTP Range header for the purposes of collection resource pagination must be consistent with the HTTP specification and how it uses the Range header to request byte ranges. The range must be zero based and inclusive:

Range: entries=0-24 would return the first 25 entries, assuming there were at least 25 entries in the data set
Range: entries=0- would return all entries starting from the first one, in case default paging is enabled then the maximum entries per size are considered to determine the end index
Range: entries=0-0 would return the first entry of the collection resource

For range-based requests, whether via Range HTTP header or query-string parameters, the server should respond with status code 200 OK and include both an Accept-Ranges header - to indicate that the supported unit is entries, and a Content-Range header – to indicate how many entries are being returned and how many total entries exist in the collection resource:

Content-Range: entries 0-24/66 indicates that 25 entries are returned out of total 66;

Content-Range: entries 0-24/* indicates that 25 entries are returned and the total number of entries is either unknown at response time, or too expensive to calculate.

Header	Description
Range	Clients that wish to use paginated retrieval of collection resources shall include this header in HTTP requests. The header shall define the offset in the collection of resources where the currently retrieved subset of resource representations starts from and the maximum number of entries in the subset. The value of the parameter shall match the regular expression entries=(\d+)-(\d+).
Accept-Ranges	Clients that have requested paginated retrieval of collection resources shall receive this header included in the HTTP response. The header allows the server to indicate its acceptance of range requests toward the collection resource's entries, thus it is recommended that the supported range unit returned by the server is entries.
Content-Range	Clients that have requested paginated retrieval of collection resources shall receive this header included in HTTP response. The header shall define the offset in the collection of resources where the currently retrieved subset of resource representations starts from and the maximum number of entries in the subset. The value of the parameter shall match the regular expression entries=(\d+)-(\d)/((\d+)\|(\ )).

Pagination via Query Parameters

Clients can request paginated responses via query parameters which shall define a zero-based begin index in the collection of resources where the currently retrieved subset of resource representations starts from (offset) and the maximum number of entries in the subset (limit).

Parameter	Description
offset	Clients that wish to use paginated retrieval of collection resources shall include this query parameter in conjunction with limit. The query parameter shall define the offset (via a zero-based index) in the collection of resources where the currently retrieved subset of resource representations starts from. The value of the parameter shall represent an integer number.
limit	Clients that wish to use paginated retrieval of collection resources shall include this query parameter in conjunction with offset. The query parameter shall define the maximum number of entries in the currently retrieved subset of resource representations. The value of the parameter shall represent an integer number.

Pagination and Caching

When a specific page of a collection resource is requested, the server should send a 200 OK response, with the Content-Range header specifying which page of the resource has been sent, and an ETag header to identify the current version of the collection. Even though a single page of the collection resource is returned, the ETag included in the response should be the ETag calculated for the collection resource as a whole. For example, the ETag of a collection may be considered to be that of the most recently modified resource the collection contains following the pattern <last-modified-resource-identifier>-<last-modified-resource-etag>, thus the ETag will uniquely identify the current version of the whole collection and differ when elements are added, removed or modified.

In order to request a specific page of a collection, clients should use the Range header or offset and limit query parameters and include the If-Match header with the ETag of the collection obtained from previously performed requests (starting from offset=0) to consistently acquire next parts of the same collection. The server can therefore verify that the collection has not changed before sending the requested page. If a more recent version of the collection exists, a 412 Precondition Failed response should be returned to invite the clients to retrieve the collection from scratch. This is necessary because it could mean that some elements might have been added or removed from the collection before or after the currently requested page and pagination is no longer consistent.

In order to optimize the network traffic, clients may check if the collection has changed by requesting the first page of the collection using an If-None-Match header with the ETag of the collection obtained from previously performed requests. If the collection has not changed since then 403 Not Modified is returned to the client identifying that there is no need to further request next pages of the collection. If the collection has changed then the first page of the latest version of the collection is transferred to the client along with the new Etag of the collection. Further requests for retrieving next pages should employ the If-Match header as defined above.