# The relationship between uniform distribution of data and the number of storage units

**URL:** <https://forums.foundationdb.org/t/the-relationship-between-uniform-distribution-of-data-and-the-number-of-storage-units/4435>\
**Category:** Running FoundationDB\
**Tags:** performance\
**Created:** [April 16, 2024, 2:17am UTC](https://forums.foundationdb.org/t/the-relationship-between-uniform-distribution-of-data-and-the-number-of-storage-units/4435 "2024-04-16T02:17:15Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vivan](https://avatars.discourse-cdn.com/v4/letter/v/6bbea6/32.png) [@vivan](https://forums.foundationdb.org/u/vivan)\
**Post date:** [April 16, 2024, 2:17am UTC](https://forums.foundationdb.org/t/the-relationship-between-uniform-distribution-of-data-and-the-number-of-storage-units/4435/1 "2024-04-16T02:17:15Z")

</div>

In our testing, we found that the data cannot be evenly distributed onto the disks, leading to some disks being excessively utilized, resulting in abnormal cluster states. I want to know if adding more storage can make the data writes distribute more evenly across the disks, or if there are other ways to avoid this issue.

---

<div class="post-metadata">

**Author:** ![vivan](https://avatars.discourse-cdn.com/v4/letter/v/6bbea6/32.png) [@vivan](https://forums.foundationdb.org/u/vivan)\
**Post date:** [April 16, 2024, 2:35am UTC](https://forums.foundationdb.org/t/the-relationship-between-uniform-distribution-of-data-and-the-number-of-storage-units/4435/2 "2024-04-16T02:35:36Z")

</div>

Our deployment consists of a single Availability Zone (AZ) with 10 nodes. Each node has 10 processes, including 6 storage, 3 stateless, and 1 log process. Below is a screenshot of the monitoring.

 ![image](https://global.discourse-cdn.com/foundationdb/original/2X/b/bb4a015d570c968a998acbef1ad8f41c86373989.jpeg)

---

<div class="post-metadata">

**Author:** ![SteavedHams](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/steavedhams/32/18_2.png) [@SteavedHams](https://forums.foundationdb.org/u/SteavedHams)\
**Post date:** [April 18, 2024, 4:38am UTC](https://forums.foundationdb.org/t/the-relationship-between-uniform-distribution-of-data-and-the-number-of-storage-units/4435/3 "2024-04-18T04:38:03Z")

</div>

What storage engine are you using and how are you calculating disk utilization? Does each storage process have its own disk?

---

<div class="post-metadata">

**Author:** ![vivan](https://avatars.discourse-cdn.com/v4/letter/v/6bbea6/32.png) [@vivan](https://forums.foundationdb.org/u/vivan)\
**Post date:** [April 19, 2024, 1:59am UTC](https://forums.foundationdb.org/t/the-relationship-between-uniform-distribution-of-data-and-the-number-of-storage-units/4435/4 "2024-04-19T01:59:40Z")

</div>

We are using the SSD-2 storage engine and calculating disk utilization using fdb-exporter. Each group of three storage processes shares one disk.

---

<div class="post-metadata">

**Author:** ![SteavedHams](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/steavedhams/32/18_2.png) [@SteavedHams](https://forums.foundationdb.org/u/SteavedHams)\
**Post date:** [April 19, 2024, 2:51am UTC](https://forums.foundationdb.org/t/the-relationship-between-uniform-distribution-of-data-and-the-number-of-storage-units/4435/5 "2024-04-19T02:51:28Z")

</div>

I’m not familiar with that tool so I don’t know what fields it is actually using for this.

The metric which FDB should balance across StorageServers total logical KV bytes it holds replicas for. This is reported in two places.

- Status JSON as `stored_bytes` for a `role=storage` role in a process
- Trace log in the `Type=StorageMetrics` trace events as `BytesStored` for each StorageServer

Check if this metric is balanced across your Storage Servers.

If you look at disk utilization or file sizes, there are several reasons they will not match. A storage server holds X amount of logical data but in \> X amount of disk usage because of overhead, internal fragmentation, and internal reusable free space.