Rust has recently enabled a debug assertion which checks pointer alignment on dereference and it started failing with misaligned pointer dereference when dereferencing objects provided by the core client (for example in the fdb_transaction_get_range result).
I tried to find out why this is the case, and stumbled upon this note in the Arena.cpp which has this note.
Does this mean that any object returned by the shared lib might be unaligned?
Dereferencing unaligned pointers is treated as undefined behaviour in most languages, and is usually tied with a performance hit too. While there are workarounds (like silencing the warning, or handling the unaligned addresses to re-align them) it would be nice to not have to add additional manual memory management when interacting with the core client.
I agree with all points. Unfortunately, misaligned pointers is currently required by the existing ABI, since the the FDBKeyValue type is declared within a pragma pack as follows. There’s no way to respect the pragma pack and align both pointers to 8 bytes.
That said, we do technically have a mechanism for changing the ABI (in new header versions) without impacting existing users in theory.
This is actually referring to a much deeper internal problem, but one that’s mostly an implementation detail and shouldn’t affect the shared lib ABI. There are misaligned accesses happening in the shared lib internally, but rust’s instrumentation presumably won’t be able to see that.
The resulting pointer has address 0x7f034cefb0a8 and the address it points to is 0x7f034cc916c6. When converting this to a rust pointer and dereferencing, the complaint is that the address that it points to is not a multiple of 4. Indeed the referenced address is simply a multiple of 2. The address of the pointer itself is aligned.
The complaint from the alignment checker is
misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7f034cc916c6'
Which is also referring to the pointed address.
Digging in the FoundationDB side of things, it looks like the referenced address is directly pointing to a Standalone<>, which if I understand correctly is allocated through the Arena and could be at any point in memory.
The “fix” involves reallocating the memory needed for the resulting array, and copying it over, so that makes me think that the packing itself is not a problem because it’s not the pointers’ addresses that are checked here, but the memory that they point to instead.
What do you think? Could this point to the arena allocations leaking to the ABI?
Quick update here, in case anyone bumps into this in the future. For Rust to understand that the API structs are misaligned, when using rust you need to make sure that you are exposing the packed structs, and not wrapping them in a transparent.