Skip to content

Run Substreams with the GraphOps CLI

Stream a Substreams package through GraphOps with your organization's key, without copying endpoints or keys into commands.

Run the Substreams CLI against GraphOps without putting an endpoint or a key in each command. graphops substreams run reads the package's network, adds the GraphOps endpoint for it, and gives your Substreams key to the Substreams CLI, which runs with the rest of your arguments unchanged.

Prerequisites

  • The GraphOps CLI and the Substreams CLI installed.
  • An organization with Substreams access.

Store a Substreams key

Create a key for your organization and store it in your operating system's credential store. Creating a key needs a sign-in from the last 12 hours; run graphops auth login first if needed.

CLI · GraphOps CLI

# Create a Substreams key for the selected organization and store it in your OS credential store.
graphops substreams key create --name laptop

To use a key you already created in API keys, store it instead. This needs no sign-in.

CLI · GraphOps CLI

# Store a Substreams key you already have, read from standard input (here the macOS clipboard).
pbpaste | graphops substreams key set

The CLI keeps one stored key. Replace it with graphops substreams key create --replace or another key set; the previous key stays active until you revoke it. In CI, set GRAPHOPS_SUBSTREAMS_API_KEY instead, which is never stored. graphops substreams key show prints the ID and last four characters of the key your commands use. When GRAPHOPS_SUBSTREAMS_API_KEY is set, that is the variable's key.

Stream a package

This streams USDC Transfer events from six Ethereum blocks with the pinned ethereum-common@v0.4.0 package, the same request as Backfill a range.

CLI · GraphOps CLI

# The package's network selects the GraphOps endpoint; graphops adds --endpoint and the key.
graphops substreams run \
  "https://spkg.io/v1/packages/ethereum-common/v0.4.0" \
  "filtered_events" \
  --params "filtered_events=evt_addr:0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48 && evt_sig:0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" \
  --start-block 18000002 \
  --stop-block 18000008 \
  -o jsonl

The package declares the Ethereum mainnet network, so the CLI streams from the Ethereum endpoint. Pass --network to choose the network yourself, for a package that does not declare the one you want. A network GraphOps does not serve stops the command with code 2 and lists the supported names; see Supported networks.

The output and exit code are the Substreams CLI's own. The key reaches only the Substreams CLI process, as SUBSTREAMS_API_KEY. graphops substreams gui opens the Substreams GUI the same way.

graphops refuses --endpoint (-e), --plaintext, --insecure, --api-key-envvar and --api-token-envvar, because they would send your key elsewhere or without TLS. To use another endpoint, run substreams directly.

Run a sink

graphops substreams sink noop, protojson and webhook run those Substreams CLI sinks the same way, with the endpoint and key added. Run any other sink, such as the Postgres sink in Run a sink or a substreams-sink-* binary, through graphops exec. It gives the sink your Substreams key as SUBSTREAMS_API_KEY, and --network sets the GraphOps endpoint for that network. Pass the endpoint with -e as well, so the sink uses it whatever network the package declares. Run the sink's setup the same way first.

CLI · GraphOps CLI

# Run the Postgres sink with your stored Substreams key and the GraphOps Ethereum endpoint.
graphops exec --product substreams --network mainnet -- \
  substreams sink postgres "substreams-template@v0.3.1" \
  -e "$(graphops substreams endpoint --network mainnet)" \
  --start-block 12292922 \
  --stop-block 12293022 \
  --dsn "$DSN"

graphops exec removes Substreams endpoint and credential variables your shell already has, such as SUBSTREAMS_ENDPOINT or SUBSTREAMS_API_TOKEN, so an endpoint set for another provider cannot receive your key. The key never enters your shell's environment.

To use the endpoint and key in your shell instead, export them:

CLI · GraphOps CLI

# Export the endpoint and key for a standalone sink or another Substreams tool.
eval "$(graphops substreams env --network mainnet)"
echo "$SUBSTREAMS_ENDPOINT"

graphops substreams env --network <network> prints SUBSTREAMS_API_KEY and SUBSTREAMS_ENDPOINT as export lines. Run it only in a shell you trust, because the key stays in that shell's environment. graphops substreams endpoint --network <network> prints the endpoint alone.

Next steps

Last updated