Subgraphs: Paginate results

Read every pool, holder, or vote in a collection for a report or a sync job. Order by a stable field and ask for the rows after the last id you received, so each request is bounded and a worker can resume where it stopped.

Prerequisites

A Subgraphs API key.
A Subgraph endpoint that exposes the entity collection you want to scan.
A stable ordering field such as id, createdAtTimestamp, or another indexed field in the Subgraph schema.

Choose a pagination method

For a few pages, first and skip are usually enough. For long scans, filter past the last value of a stable field, such as where: { id_gt: $lastId } with orderBy: id. Each page then starts where the previous one ended, however far into the collection you are:

Each page asks for the rows after the last id of the page before it. Save that id after you process a page, and stop when a page returns fewer rows than you requested.

Request

This request reads one page of up to 10 pools in the current indexed view after $lastId. For the first page, set $lastId to a value that sorts before every id, such as the zero address in these examples.

Prerequisites

Set GRAPHOPS_API_KEY to your GraphOps API key.
Replace <ALIAS> with your Endpoint Alias.

curl

Example project

# Start below every id. For the next page, set lastId to the last id in this response.
curl --fail-with-body --silent --show-error "https://subgraphs.graphops.xyz/v1/alias/<ALIAS>" \
  -H "Authorization: Bearer ${GRAPHOPS_API_KEY}" \
  -H "Content-Type: application/json" \
  --data '{
    "query": "query PoolsPage($first: Int!, $lastId: ID!) { pools(first: $first, where: { id_gt: $lastId }, orderBy: id, orderDirection: asc) { id token0 { symbol } token1 { symbol } } _meta { block { number hash } } }",
    "variables": { "first": 10, "lastId": "0x0000000000000000000000000000000000000000" },
    "operationName": "PoolsPage"
  }'

Output

{  "data": {    "pools": [Up to $first rows, ordered by id      {        "id": "0x000024feb293b6c6c3a80a95f1f830a8746400b9",        "token0": {          "symbol": "TRUMP"        },        "token1": {          "symbol": "WETH"        }      },      {        "id": "0x0000d36ab86d213c14d93cd5ae78615a20596505",Last id: your next $lastId        "token0": {          "symbol": "RALPH"        },        "token1": {          "symbol": "WETH"        }      },      {        "id": "0x00013bffe4c38c1372a450fac78b01cb4f554781",        "token0": {          "symbol": "Pectra"        },        "token1": {          "symbol": "WETH"        }      },      {        "id": "0x0001fcbba8eb491c3ccfeddc5a5caba1a98c4c28",        "token0": {          "symbol": "BCZ"        },        "token1": {          "symbol": "WETH"        }      },      {        "id": "0x00024b66a38a676c0eae59126c1367ad141afa79",        "token0": {          "symbol": "WETH"        },        "token1": {          "symbol": "TI"        }      },      {        "id": "0x0002e63328169d7feea121f1e32e4f620abf0352",        "token0": {          "symbol": "wNXM"        },        "token1": {          "symbol": "ICHI"        }      },      {        "id": "0x0003c9d75e102b8a9fe77280ca410e9135beb150",        "token0": {          "symbol": "TRANCHE-bWBTC-A"        },        "token1": {          "symbol": "USDT"        }      },      {        "id": "0x0003e7d8d22d0e8e53d56885069fc8a254f0431f",        "token0": {          "symbol": "RLP"        },        "token1": {          "symbol": "USDC"        }      },      {        "id": "0x0004cd8474e882278e32e584699090be496f410e",        "token0": {          "symbol": "ZERO"        },        "token1": {          "symbol": "USDC"        }      },      {        "id": "0x0005079292756443f2e711570f8ab4e1951d4f97",        "token0": {          "symbol": "EDG"        },        "token1": {          "symbol": "WETH"        }      }    ],    "_meta": {      "block": {        "number": 26250897,Log with each page        "hash": "0x7c2450505fd5a13e82ef63e3177dfb4521f50793653c7c60b85efca1acc25c2f"      }    }  }}

Run the full scan

Send the first request with $lastId set before every id.
Process the page, then save the last id you processed.
Send the next request with that id as $lastId.
Stop when a page returns fewer rows than $first.

Keep these in mind for long scans:

Keep page sizes bounded, and back off after 429 responses.
Prefer id_gt filters over high skip values.
Persist the last processed id so a worker can resume after a retry or restart.
Log _meta.block.number with each page. Historical block queries are not supported, so pages cannot be pinned to one block and later pages can reflect newer data.

Prerequisites

Choose a pagination method

Request

Run the full scan

Next steps