Question about Watches limit

I would like to get a clarification about watches limits.

The documentation here states:
https://apple.github.io/foundationdb/api-python.html#watches
“By default, each database connection can have no more than 10,000 watches that have not yet reported a change.”

Does that number apply to the number of watched keys or the number of outstanding watch requests (represented as Futures returned by the call Transaction.watch()).
To make it even more concrete, let’s say we have 10 different keys I use to watch. On each key we have 5 different client processes calling Transaction.watch() at the same time. Does that count as 50 watches for the sake of this limit or as 10 watches?

Technically in this case, it counts only as 10 watches, but I believe for a different reason than you’re thinking. If you create watches on the same key, each one counts separately, so if your 10 watches on each client were for the same key, then it would still count as 10.

The reason that this is only counting as 10 against the limit rather than 50 is that the limit applies to each database connection separately. Each of your 5 different client processes is going to count the number of watches created locally, and in your case each has 10 watches.

I see. The limit is per client (per DB connection). And the limit counts the number of outstanding watch requests (active watch Futures), even if they are all for the same key.
Thank you.

I’m curious why the limit is set at client instead of at server side?

If the client’s machine is similar powerful as the SS machine, won’t SS reach limit first?

Storage servers have their own limits, and when they are exceeded they result is that the client falls back to automatic polling. This all happens transparently.