Tuning ZFS for FDB

Snowflake is evaluating ZFS as a means of providing snapshots for FoundationDB. Because using ZFS will come at a performance cost, we’re interested in how to tune ZFS to minimize that cost.

We’re interested in hearing from anyone has run FDB on ZFS.

  • What sort of tunables did you set?
    • logbias? primarycache? recordsize?
    • How do you manage the tradeoff between ARC size and FDB’s page cache?
  • What does your pool layout look like?
    • Since we are only interested in snapshots, we’re looking at using a separate zpool per disk.
4 Likes

hey Sam,

did you get any further with this? In general I’d not use a separate zpool per disk as this obviously limits the overall iops of the system, and would make it difficult to get benefits out of a zil or slog.

BTW there’s a FreeBSD foundationdb-devel port now that might be of interest to you, and as soon as FDB 6.3.0 lands (which has cmake for proper C binding support) the main databases/foundationdb one will be updated too.

Hey Dave – I plan to have some test results by the end of the quarter.

Re: FreeBSD – I’ve seen that! Though I’m a big fan of FreeBSD we’re a Linux shop so I’m not testing that platform. The results I get should carry over.

well feel free to reach out, I have a reasonable BSD-based test env over here, and I’m just getting my head around all the FDB related stuff.