Setting value size limit knob during fdbrestore

I’ve been trying to restore from a cluster where my Java client that used to write to it, had the value_size_limit knob set to 3MB. How can I set this knob for the new cluster I’m trying to restore to? It’s a client knob so I believe putting it in foundationdb.conf wouldn’t be useful.
I keep getting the error:

‘Value length exceeds limit’ on ‘restore_range_data’

I’m not exactly sure how the restore works now, but I believe you are going to need to set the value_size_limit knob on the agents responsible for actually doing the restore, which I think are the backup_agent processes. I believe this can be done the same way as for server processes, by using --knob_value_size_limit=3000000 or using knob_value_size_limit=3000000 in foundationdb.conf.

Also be aware that there are a couple places that we make use of this value on the server side, so I would recommend also having this value set on all of your servers to avoid any weird behaviors.

1 Like

That makes sense, I will try out setting the value size knob in all processes of the server and backup_agent as well.

I think I might have seen at least one weird behaviour as a result of setting them only on the client - we do some parsing of backup files for some internal analysis, and noticed that log ranges had incomplete mutation blocks for certain versions. The size encoding at the beginning of each block mentioned (10000x + y) bytes for a particular version, but we found only 10000z bytes in that block, where x > z. Here, I am accounting for the fact that the mutation logs for each version can sometimes be spread across multiple parts and we have to concatenate those parts to get the final set of logs per version. This happened for a significant number of mutation blocks.