Why API Pagination Next Links Should Be Treated as Opaque URLs

When an API response includes a URL for the next page, follow that link rather than rebuilding it from guessed query parameters. The provider may encode a cursor, change pagination strategy, or require other query values to remain intact. Treating the link as opaque helps your client keep pace with those details—and makes it less likely to skip or repeat results.

This article covers response-body links and HTTP Link headers, distinguishes cursor-based from offset-based pagination, and shows how to validate a supplied URL before requesting it. Provider-specific response formats differ, so check the API's current documentation for its exact pagination and authentication behavior. I cannot retrieve provider documentation live for this article; the examples below are illustrative, not a claim about any particular API.

Key Takeaways

• Follow the server-provided next URL; do not reconstruct its cursor or offset parameters.
• A next link can appear in the response body or in an HTTP Link header with rel="next".
• Cursor pagination uses a continuation token; offset pagination identifies a position or page. Either may be represented by a URL.
• “Opaque” does not mean “trusted.” Validate the scheme and destination against an allowlist before making a request.
• Send credentials only to the intended trusted origin, and avoid silently forwarding them across redirects.

Where API Pagination Links Appear

An API can return the next page in a JSON field:

{
  "data": [
    { "id": "item-101" },
    { "id": "item-102" }
  ],
  "next": "https://api.example.com/v1/items?cursor=eyJvZmZzZXQiOjEwMn0"
}

The field might instead be named next_page, nextUrl, or something specific to the provider. Some APIs return null or omit the field when there are no more results. Follow the provider's documented response format rather than assuming a universal field name.

Other APIs put pagination links in the HTTP Link response header. For example:

Link: <https://api.example.com/v1/items?cursor=abc>; rel="next", <https://api.example.com/v1/items?cursor=xyz>; rel="prev"

The rel="next" parameter identifies the link for the next page. A Link header can contain multiple links and parameters, so do not parse it by splitting blindly on commas. Use a parser that understands the header format when your client needs to extract a relation from it. The link value may also be relative, in which case resolve it against the URL of the response that supplied it.

Why the URL Should Be Opaque

A client that constructs its own next URL has to know how the provider encodes continuation state. That can be fragile: a cursor may be signed, encoded, or tied to a particular query; a provider may require filters or sorting parameters to persist; and an API may change its pagination scheme.

Treating the supplied link as opaque means preserving it as a URL and following it without editing its pagination parameters. It does not mean accepting it without checks. Validate that it is safe to request, and do not infer that every URL in a response is trustworthy.

Cursor-Based vs. Offset-Based Pagination

These terms describe pagination strategies, not necessarily URL formats. A cursor-based API can put its cursor in a URL, and an offset-based API can provide a complete next URL. In either case, follow the provider's link so the provider controls how continuation works.

A Safe Client Pattern

The following TypeScript example assumes a response body shaped like { data: T[], next: string | null } and a single API origin. It uses Node.js 20 or later, where fetch is available. Adjust the response type and origin allowlist to match the provider's documentation and your deployment.

const API_ORIGIN = "https://api.example.com";
const MAX_PAGES = 100;

interface Page<T> {
  data: T[];
  next: string | null;
}

function validateNextUrl(link: string, currentUrl: string): URL {
  const url = new URL(link, currentUrl);

  if (url.protocol !== "https:") {
    throw new Error("Pagination link must use HTTPS");
  }

  if (url.origin !== API_ORIGIN) {
    throw new Error("Pagination link points to an untrusted origin");
  }

  if (url.username || url.password || url.hash) {
    throw new Error("Pagination link contains disallowed URL components");
  }

  return url;
}

async function collectItems<T>(token: string): Promise<T[]> {
  let currentUrl = new URL("/v1/items", API_ORIGIN).toString();
  const visited = new Set<string>();
  const results: T[] = [];

  for (let pageNumber = 0; pageNumber < MAX_PAGES; pageNumber++) {
    const url = validateNextUrl(currentUrl, currentUrl);

    if (visited.has(url.href)) {
      throw new Error("Pagination link loop detected");
    }
    visited.add(url.href);

    const response = await fetch(url, {
      headers: { Authorization: `Bearer ${token}` },
      redirect: "manual",
      signal: AbortSignal.timeout(10_000),
    });

    // Do not automatically follow a redirect with credentials attached.
    if (response.status >= 300 && response.status < 400) {
      throw new Error("Unexpected redirect while fetching a page");
    }

    if (!response.ok) {
      throw new Error(`API request failed with status ${response.status}`);
    }

    const page = (await response.json()) as Page<T>;
    results.push(...page.data);

    if (!page.next) {
      return results;
    }

    currentUrl = validateNextUrl(page.next, url.href).href;
  }

  throw new Error(`Pagination exceeded the ${MAX_PAGES}-page limit`);
}

The example checks each returned link before using it, requires HTTPS, restricts requests to one configured origin, rejects redirects, detects repeated URLs, and caps the number of pages. The bearer token is sent only after the URL passes the same-origin check. If a provider legitimately uses more than one API host, add each permitted origin deliberately and decide which credentials, if any, belong on each one; do not forward a token to every allowed host by default.

For APIs that provide pagination in a Link header, extract the link whose relation is next with a standards-aware parser, then pass its URL through the same validation step. The sample does not implement Link header parsing.

Security Checks Before Following a Link

A server-provided link is data from the response, not automatic permission to make a request to any destination it names. This matters especially when URLs can be influenced by users or when the API itself is not fully trusted.

• Restrict destinations. Compare the parsed origin against a configured allowlist. A hostname suffix check such as endsWith("example.com") is not an adequate substitute for exact origin validation.
• Require HTTPS. Reject unexpected schemes rather than allowing a link to switch the client to plain HTTP or another protocol.
• Control credentials. Attach authentication only to the intended host. Avoid copying headers blindly to a different origin, and do not follow redirects while carrying credentials unless you have an explicit, safe redirect policy.
• Set resource limits. Use request timeouts and a maximum page count. Consider response-size limits when the API can return large pages.
• Avoid leaking URLs. Pagination URLs can contain tokens or other sensitive state. Avoid logging full URLs where that information might be exposed.

If your service makes requests to URLs supplied by users or an untrusted upstream, the risk is broader than credential leakage: this can become server-side request forgery (SSRF). An origin check alone may not address DNS rebinding, redirects, or resolution to private and link-local addresses. Use controls appropriate to that threat model, including network egress restrictions and revalidation of destinations across redirects and DNS resolution. A pagination-link check is not a general SSRF defense.

Common Mistakes

• Rebuilding the next URL. Incrementing page or offset can discard provider-specific state or produce incorrect results.
• Assuming every API uses the same response shape. Read the provider's documentation to determine whether the next link is in the body, a header, or both.
• Treating a cursor as a number. Cursor values are opaque tokens, not necessarily values a client can interpret or calculate.
• Trusting every supplied URL. Validate the scheme and origin before requesting it; do not use a general URL parser as if it were a security policy.
• Forwarding authentication indiscriminately. A link or redirect to a different host should not automatically receive the original request's credentials.
• Parsing Link headers with string splitting. The header has syntax and parameters; use an appropriate parser to find the next relation.

Best Practices

1. Follow the API's documented pagination mechanism and use its supplied next link.
2. Treat pagination values as opaque; preserve them without decoding or rewriting.
3. Validate every link immediately before requesting it, including links returned on later pages.
4. Keep an explicit host allowlist and a deliberate policy for credentials and redirects.
5. Add timeouts, page limits, and loop detection so malformed pagination cannot run indefinitely.
6. Test normal completion, missing next links, malformed URLs, untrusted hosts, redirects, repeated links, and API errors.

Conclusion

An API pagination next link is the provider's instruction for continuing through a result set. Follow it rather than reconstructing its parameters, but validate its destination and apply a clear credential, redirect, and resource-limit policy before making each request.

References

No provider-specific API documentation could be retrieved for this article. Check the current documentation for the API you integrate, especially its response format, link semantics, and authentication rules. The link-header example reflects the general Web Linking model described by RFC 8288, Web Linking.