Hello,
I’m looking at the implementation of two streaming modes in fdb_c.cpp: WANT_ALL and ITERATOR. While WANT_ALL is generally recommended by the documentation for complete extraction of the desired range at maximum throughput, practically it is less performant than ITERATOR on payloads ~ >= 1.6MiB.
Unless I understand it wrong, in the following code the last “state” of iteration_progress is 1.5 times bigger than what WANT_ALL is configured with (120000 > 80000). Which means that after a series of iterations an accumulated throughput of the ITERATOR will exceed the one provided by WANT_ALL. In theory this should happen when the total extracted payload exceeds 1.6 MiB mark. This behavior was introduced by 4416.
My question is: if I understand it right, shouldn’t target_bytes for WANT_ALL be assigned 120k as well?
/* _ITERATOR mode maps to one of the known streaming modes
depending on iteration */
const int mode_bytes_array[] = { GetRangeLimits::BYTE_LIMIT_UNLIMITED, 256, 1000, 4096, 80000 };
/* The progression used for FDB_STREAMING_MODE_ITERATOR.
Goes 1.5 * previous. */
static const int iteration_progression[] = { 4096, 6144, 9216, 13824, 20736, 31104, 46656, 69984, 80000, 120000 };