Tuning ZFS for FDB

Snowflake is evaluating ZFS as a means of providing snapshots for FoundationDB. Because using ZFS will come at a performance cost, we’re interested in how to tune ZFS to minimize that cost.

We’re interested in hearing from anyone has run FDB on ZFS.

  • What sort of tunables did you set?
    • logbias? primarycache? recordsize?
    • How do you manage the tradeoff between ARC size and FDB’s page cache?
  • What does your pool layout look like?
    • Since we are only interested in snapshots, we’re looking at using a separate zpool per disk.

hey Sam,

did you get any further with this? In general I’d not use a separate zpool per disk as this obviously limits the overall iops of the system, and would make it difficult to get benefits out of a zil or slog.

BTW there’s a FreeBSD foundationdb-devel port now that might be of interest to you, and as soon as FDB 6.3.0 lands (which has cmake for proper C binding support) the main databases/foundationdb one will be updated too.