Sync Reth in 6 hours with Snapshots

At merkle we run reth in production. Syncing a new node takes a week on fast hardware:

Syncing this node took 7 days

This server has 128GB of RAM, fast NVMe disks and 1 Gbp/s bandwidth in a datacenter on the USA east coast. We have been looking for ways to speed up the process. Snapshots have been the most effective so far.

Generating a snapshot

We currently run 4 reth nodes in production. We use one of them to periodically generate a snapshot.

The process is simple. First we stop the node:

sudo systemctl stop reth

Then we compress the mdbx.dat file (the reth database) located at ~/.local/share/reth/mainnet/db/ by default using tar and lz4 (best ratio for speed and compression), which takes about 1 hour on our hardware:

sudo tar --use-compress-program=lz4 -cvf /root/.local/share/reth/mainnet/db/reth.tar.lz4 /root/.local/share/reth/mainnet/db/mdbx.dat

Then, we upload the snapshot to a remote server, within our infrastructure with transfer speeds of 5 GB/s.

This setup allows us to sync a new node in less than four hours instead of 7+ days, speeding up the process 40x fold.

Public snapshots

We've been looking for ways to share our snapshots with the community. We have looked at multiple options and finally settled on CloudFlare R2 for their unlimited egress.

As of today, in addition to our private snapshots, we are also sharing public snapshots. You can find them at snapshots.merkle.io. New snapshots are uploaded every Monday and Thursday.

Download speed

In order to offer snapshots for free to everyone, we had to compromise on download speed (only ~100-300MB/s with R2).

Note: If you don't want to run reth in production, use our public unlimited RPC eth.merkle.io instead.