# Production optimizations

**URL:** <https://forums.foundationdb.org/t/production-optimizations/601>\
**Category:** Using FoundationDB\
**Created:** [July 30, 2018, 9:18pm UTC](https://forums.foundationdb.org/t/production-optimizations/601 "2018-07-30T21:18:56Z")\
**Posts on this page:** 1\
**Showing post:** 11

<div class="post-metadata">

**Author:** ![alexmiller](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/alexmiller/32/326_2.png) [@alexmiller](https://forums.foundationdb.org/u/alexmiller)\
**Post date:** [August 2, 2018, 7:33pm UTC](https://forums.foundationdb.org/t/production-optimizations/601/11 "2018-08-02T19:33:54Z")

</div>

> [@ThomasJ](#):
>
> - What about the rest? What are those going to do?

Probably 80-90 of the 100 severs are just going to be running storage servers. It’s probably slightly better to not assign them any process class, rather than assign them storage process class, so that in the case of failures, you have processes to fall back on that you can recruit other roles from.

> [@ThomasJ](#):
>
> - Additionally how would you structure those servers, specially hard-drives? Are they equal (e.g. 1TB space each?)

It doesn’t really matter, depending on your workload. Data distribution aims to keep the used space percent roughly equal among storage servers. So nothing bad will happen if your run a cluster, half with 1TB drives and half with 10TB drives.

You will, however, have created a cluster where you have 10x more disk time for some data than other data. If you’re randomly reading keys from your database, 1TB storage servers are 10x more likely to have that data in memory, and if they don’t, have 10x less requests going to their drives. If you have largely cold data, then this won’t matter. If you’re latency sensitive, then you want to size your drives equally, and preferably to match your IO/s,

> [@ThomasJ](#):
>
> - What about coordinators? Is it sensible to make one process from each “physical” server a coordinator? What is coordinator anyway. Is it a process that is dedicated to something, or just a discovery mechanism and it truly doesn’t matter how many are there?

Think of coordinators as a zookeeper cluster. You’ll want to run 2n+1 of then, where n is the number of failures you can take and keep running. If you’re running in triple replication, then n=2. If you’re running double replication, then n=1.

If you’re running across multiple DC’s, it’s best to adjust this math a little bit, so that you have 2n+1 coordinators per DC, as always having a local coordinator will save you WAN traffic.

> [@ThomasJ](#):
>
> - What’s the best spread of classes? Based on this [guide](https://apple.github.io/foundationdb/configuration.html#guidelines-process-class-config) it looks like 10 for logs, 9 for proxies. From your example you had 2 for logs, 6 for storage. So extrapolating that on 10 servers: 7 storage, 3 logs, 30 stateless with 10 of them used as proxies (on 4 core server)

The exact best number of logs and proxies is going to vary strongly based on your workload. You essentially want the minimum number of them that can still support your load with some overhead left for safety.

My very rough guess would be to say when running with with the `ssd` storage engine, you’ll probably want about 1 log for every 8 storage servers, and probably around 1 proxy per log. But I would strongly suggest running some representative performance test, and varying the number of proxies and logs until you find an optimal spot.

> [@ThomasJ](#):
>
> - Would it be more sensible to have just 2 core server then?

Quite possibly, yes. The extra cores could be useful to run stateless roles, but they have their own demands. Proxies and logs are both network bandwidth hungry, as all commits in the system flow through them, so it’s good to plan out their network bandwidth allocations as well.

I’m used to thinking about this in a bare metal world, and not one in which your disk (and network?) scales with the number of cores you have. FDB, overall, should be more disk and network bottlenecked than CPU, as its primary job is to do IO.

If you’re writing any layers to run on top of FDB, then extra cores alongside your storage servers are great places to run you layers processes – both from a latency and binpacking perspective.

---

_[View the full topic](https://forums.foundationdb.org/t/production-optimizations/601)._