Published on

Sync Reth in 6 hours with Snapshots

Authors
Monetize your wallet transactions today.
👋 Welcome to our Engineering blog. merkle specializes in MEV protection and monetization on Ethereum. We guarantee minimum $0.08 per transaction. Ideal for Wallets and RPC Providers looking to protect their customers against Sandwiches while generating revenue.

At merkle we run reth in production. Syncing a new node takes a week on fast hardware:

Reth sync

Syncing this node took 7 days

This server has 128GB of RAM, fast NVMe disks and 1 Gbp/s bandwidth in a datacenter on the USA east coast. We have been looking for ways to speed up the process. Snapshots have been the most effective so far.

Generating a snapshot

We currently run 4 reth nodes in production. We use one of them to periodically generate a snapshot.

The process is simple. First we stop the node:

sudo systemctl stop reth

Then we compress the mdbx.dat file (the reth database) located at ~/.local/share/reth/mainnet/db/ by default using tar and lz4 (best ratio for speed and compression), which takes about 1 hour on our hardware:

sudo tar --use-compress-program=lz4 -cvf /root/.local/share/reth/mainnet/db/reth.tar.lz4 /root/.local/share/reth/mainnet/db/mdbx.dat

Then, we upload the snapshot to a remote server, within our infrastructure with transfer speeds of 5 GB/s.

This setup allows us to sync a new node in less than four hours instead of 7+ days, speeding up the process 40x fold.

Public snapshots

We've been looking for ways to share our snapshots with the community. We have looked at multiple options and finally settled on CloudFlare R2 for their unlimited egress.

As of today, in addition to our private snapshots, we are also sharing public snapshots. You can find them at snapshots.merkle.io. New snapshots are uploaded every Monday and Thursday.

Download speed

In order to offer snapshots for free to everyone, we had to compromise on download speed (only ~100-300MB/s with R2).

Note: If you don't want to run reth in production, use our public unlimited RPC eth.merkle.io instead.

RETH RPC
The first RPC service powered by RETH, 20ms faster on average and 10x cheaper.
Sign up
Transaction Stream
Stream all Ethereum, Polygon and BSC transactions to your application in real-time. Faster than any transaction stream on the market. Built with RETH.
Sign up
Simulation API
Simulate bundles of up to 10 transactions on Ethereum, Polygon and BSC for $0.2 / 1,000,000,000 of gas.
Sign up