I found two statements are seemingly contradictory in fdb document layer’s documentation.
“Like the FoundationDB Key-Value Store, the Document Layer achieves its maximum performance with a highly concurrent workload. Multiple client processes are one way of achieving this, but in many situations, it is advantageous for each individual client to open multiple connections to the Document Layer and round-robin requests across them. Clients using MongoDB® drivers that automatically instantiate connection pools may need to manually increase pool size to saturate the Document Layer and Key-Value Store server processes.”
“Transactions are tracked on a per-connection basis. Therefore, care must be taken when using a MongoDB® client or driver that automatically round-robins operations across a connection pool. To ensure correct behavior, the safest approach is to deactivate such features and manage connections manually.”
So my question is how to nicely handle Mongo drivers that automatically round-robin?
Documentation could have been clearer here. The first paragraph was talking about normal requests which are not part of explicit client transactions. As FoundationDB is better with parallel requests, having more parallel requests on the Document Layer would turn into parallel FDB transactions.
The second statement is about explicit transactions started from the application using beginTransaction command. All the requests between beginTransaction and commitTransaction will be committed under a single transaction. But there is no transaction ID or object that groups all these requests into a transaction. So, the protocol couples transaction with the connection. All requests on the connection would be considered as part of the transaction. Hence, it is important to not use load balancing features in drivers together with the Document Layer transactions. This is why we should use explicit transactions only with the sidecar architecture. Even then we should be careful with load balancing in the driver as explained above.
Our plan is to support transactions through MongoDB sessions. Then, session ID can be used to group requests in a transaction just like MongoDB v4.0.0 transactions. That will allow us to not have many of these limitations.
Just want to add a little bit more on the Document Layer request processing. Every request on the Document Layer is part of a FDB transaction, whether it is part of an explicit client transaction or not. Even a request not part of an explicit transaction would start a FDB transaction just for itself. For example, an insert/update operation would start one FDB transaction for both index update and document update. This gives better consistency guarentees even for normal requests.