Testing Ethereum Substreams-powered Subgraphs(MIPs TheGraph program)

Olga L
2 min readJun 8, 2023

What is Firehose?

  • Blockchain-agnostic way of storing and retrieving data
  • Makes it easy for applications to handle “forks” using “cursors”
  • Extract everything you would ever need into flat files
  • Flat files allow massively parallel processing using things like substreams*
  • Unit of storage are “Merged Blocks” aka “Firehose Blocks”
  1. install execution client:

Firehose-instrumented execution client (fork of geth) running as a full nod Recommended build: v1.11.5-fh2.2–1 as of April 19, 2023 https://github.com/streamingfast/go-ethereum/releases/download/geth-v1.11.5-fh2.2-1/geth_linux

2. install consensus client Prysm:

mkdir prysm && cd prysm
curl https://raw.githubusercontent.com/prysmaticlabs/prysm/master/prysm.sh --output prysm.sh && chmod +x prysm.sh

# ### Generate JWT Secret
openssl rand -hex 32 | tr -d "\n" > "jwt.hex"

3. Run an execution client:

# Firehose config.yml for Ethereum mainnet

start:
args:
- merger
- firehose
- reader-node
- relayer
- combined-index-builder
flags:
data-dir: /var/lib/firehose
common-chain-id: "1"
common-network-id: "1"
reader-node-path: /root/geth_linux
substreams-rpc-endpoints: $ETHEREUM_ARCHIVE_RPC
substreams-enabled: true
substreams-partial-mode-enabled: true
firehose-grpc-listen-addr: 0.0.0.0:13042
substreams-request-stats-enabled: true
reader-node-arguments: "--mainnet
--datadir=/var/lib/firehose/geth
--ipcpath=/var/lib/firehose/reader/ipc
--http --http.api=eth,net,web3
--authrpc.jwtsecret /root/geth/jwtsecret
--http.port=8555 --http.addr=0.0.0.0 --http.vhosts=*
--firehose-enabled --port=30303 --cache=2048"

Start fireeth with the following command: fireeth -c /etc/firehose/config.yml start

4. Run a beacon node using Prysm:

./prysm.sh beacon-chain --execution-endpoint=http://localhost:8545 --jwt-secret=/root/geth/jwtsecret --checkpoint-sync-url=https://mainnet.checkpoint.sigp.io/ --genesis-beacon-api-url=https://mainnet.checkpoint.sigp.io/

Configure Graph Node for substreams-powered subgraphs

[chains.${FIREHOSE_CHAIN}]
shard = "primary"
provider = [
{ label = "firehose", details = { type = "firehose", url = "${FIREHOSE_RPC}" } }
]

Sync and serve a substreams-powered subgraph: QmeWyu8pzV3zViL5nz813NxKSab7REh9qJBivjzkCzE9Vw

--

--