Run the Substreams CLI against GraphOps without putting an endpoint or a key in each command.
graphops substreams run reads the package's network, adds the GraphOps endpoint for it, and gives
your Substreams key to the Substreams CLI, which runs with the rest of your arguments unchanged.
Prerequisites
- The GraphOps CLI and the Substreams CLI installed.
- An organization with Substreams access.
Store a Substreams key
Create a key for your organization and store it in your operating system's credential store.
Creating a key needs a sign-in from the last 12 hours; run graphops auth login first if needed.
CLI · GraphOps CLI
# Create a Substreams key for the selected organization and store it in your OS credential store.
graphops substreams key create --name laptop
To use a key you already created in API keys, store it instead. This needs no sign-in.
CLI · GraphOps CLI
# Store a Substreams key you already have, read from standard input (here the macOS clipboard).
pbpaste | graphops substreams key set
The CLI keeps one stored key. Replace it with graphops substreams key create --replace
or another key set; the previous key stays active until you revoke it. In CI, set
GRAPHOPS_SUBSTREAMS_API_KEY instead, which is never stored. graphops substreams key show prints
the ID and last four characters of the key your commands use. When GRAPHOPS_SUBSTREAMS_API_KEY is
set, that is the variable's key.
Stream a package
This streams USDC Transfer events from six Ethereum blocks with the pinned
ethereum-common@v0.4.0 package, the same request as
Backfill a range.
CLI · GraphOps CLI
# The package's network selects the GraphOps endpoint; graphops adds --endpoint and the key.
graphops substreams run \
"https://spkg.io/v1/packages/ethereum-common/v0.4.0" \
"filtered_events" \
--params "filtered_events=evt_addr:0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48 && evt_sig:0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" \
--start-block 18000002 \
--stop-block 18000008 \
-o jsonl
The package declares the Ethereum mainnet network, so the CLI streams from the Ethereum endpoint.
Pass --network to choose the network yourself, for a package that does not declare the one you
want. A network GraphOps does not serve stops the command with code 2 and lists the supported
names; see Supported networks.
The output and exit code are the Substreams CLI's own. The key reaches only the Substreams CLI
process, as SUBSTREAMS_API_KEY. graphops substreams gui opens the Substreams GUI the same way.
graphops refuses --endpoint (-e), --plaintext, --insecure, --api-key-envvar and
--api-token-envvar, because they would send your key elsewhere or without TLS. To use another
endpoint, run substreams directly.
Run a sink
graphops substreams sink noop, protojson and webhook run those Substreams CLI sinks the same
way, with the endpoint and key added. Run any other sink, such as the Postgres sink in
Run a sink or a substreams-sink-* binary, through graphops exec. It gives the
sink your Substreams key as SUBSTREAMS_API_KEY, and --network sets the GraphOps endpoint for
that network. Pass the endpoint with -e as well, so the sink uses it whatever network the package
declares. Run the sink's setup the same way first.
CLI · GraphOps CLI
# Run the Postgres sink with your stored Substreams key and the GraphOps Ethereum endpoint.
graphops exec --product substreams --network mainnet -- \
substreams sink postgres "substreams-template@v0.3.1" \
-e "$(graphops substreams endpoint --network mainnet)" \
--start-block 12292922 \
--stop-block 12293022 \
--dsn "$DSN"
graphops exec removes Substreams endpoint and credential variables your shell already has, such
as SUBSTREAMS_ENDPOINT or SUBSTREAMS_API_TOKEN, so an endpoint set for another provider cannot
receive your key. The key never enters your shell's environment.
To use the endpoint and key in your shell instead, export them:
CLI · GraphOps CLI
# Export the endpoint and key for a standalone sink or another Substreams tool.
eval "$(graphops substreams env --network mainnet)"
echo "$SUBSTREAMS_ENDPOINT"
graphops substreams env --network <network> prints SUBSTREAMS_API_KEY and
SUBSTREAMS_ENDPOINT as export lines. Run it only in a shell you trust, because the key stays in
that shell's environment. graphops substreams endpoint --network <network> prints the endpoint
alone.
Next steps
Last updated