I’m am trying to understand how the internals work. Right now I’m wondering how/where getPenalty in the StorageServerActor is used:
Where is the penalty handled? Related but I don’t see where the RateKeeper informations are used can anyone point me to the source code that use this info?
The storage server’s current penalty is returned as part of get() and getRange() responses for clients to use when load balancing requests across members of a storage team.
Regarding the penalty, for some types of requests the load balancer keeps track of the number of requests outstanding to each server and tries to keep that number balanced. The penalty is used as a multiplier on how much this number is incremented for each request. For example, a penalty of 5 means each request counts as 5 requests for balancing purposes, which reduces the traffic to that server relative to the others by roughly a factor of 5.
You can see some of this at work in https://github.com/apple/foundationdb/blob/master/fdbrpc/QueueModel.cpp, and the QueueModel itself being used in https://github.com/apple/foundationdb/blob/16a0a4cbe2d463ea2ca41a54c65e362265a1eab8/fdbrpc/LoadBalance.actor.h#L42.
Ratekeeper is implemented here: https://github.com/apple/foundationdb/blob/master/fdbserver/Ratekeeper.actor.cpp. Is that what you are looking for?