Throttle list not listing auto tags

Using FoundationDB Running FoundationDB
1

I am trying to observe Ratekeeper behavior to gain more insight. In attempting to saturate the db I can see the system is clearly reducing the transaction rate and health metrics says batch_limited is true. However when I run throttle list it says no tags are being throttled and status outputs a count of 0 for auto throttled tags. How do I get the system to tell me what throttles it is putting in place?

2

Priority-based throttling is different from tag-based throttling. batch_limited means that all transaction with “Batch Priority” is throttled, while tag-based throttling only throttle transactions with a specific tag.
I think to make the Ratekeeper report auto throttled tags, you need to enable this feature with throttle enable auto and tag transactions use AUTO_THROTTLE_TAG option.

Detailed document: Transaction Tagging — FoundationDB 7.1

3

All my transactions are tagged with auto throttle tag option and auto throttling has been enabled from fdbcli but it still lists auto tags being throttled as count: 0. The system is clearly slowing down transactions as well.

4

Did you see any trace event like BusiestReadTag or BusiestWriteTag?

The system is clearly slowing down transactions as well.

I see. But I doubt the slow down is due to global-wise throttling not tag throttling. Could you confirm what you see in status json? I’m interested in qos part.

 "batch_transactions_per_second_limit":0,
 "transactions_per_second_limit":0,
 "batch_released_transactions_per_second":0,
 "released_transactions_per_second":0,
"throttled_tags":{
            "auto" : {
                "busy_read" : 0,
                "busy_write" : 0,
                "count" : 0,
                "recommended_only": 0
            },
            "manual" : {
                "count" : 0
            }
         },
5

I do see a busiest read tag, using version 6.3.23 without hot write support so didn’t think there would be busiest write. That output is the correct tag (one I am trying to saturate the system with) and I also have some status reports.

"batch_performance_limited_by" : {
                "description" : "Storage server running out of space (approaching 100MB limit).",
                "name" : "storage_server_min_free_space",
                "reason_id" : 6,
                "reason_server_id" : "71c6cf932f5fe65c"
            },
"released_transactions_per_second" : 1.25329,
            "throttled_tags" : {
                "auto" : {
                    "count" : 0
                },
                "manual" : {
                    "count" : 0
                }
            },
            "transactions_per_second_limit" : 0.421682

"busiest_read_tag" : {
                            "estimated_cost" : {
                                "hz" : 123.313
                            },
                            "fractional_cost" : 0.84090899999999991,
                            "tag" : "read"
6

The “busiest_read_tag” looks right to me. Maybe you need to search for trace events RkSetAutoThrottle to see what happened…