Unaligned pointers in the core client library

Development FoundationDB Core
1

Hi,

Rust has recently enabled a debug assertion which checks pointer alignment on dereference and it started failing with misaligned pointer dereference when dereferencing objects provided by the core client (for example in the fdb_transaction_get_range result).

I tried to find out why this is the case, and stumbled upon this note in the Arena.cpp which has this note.

Does this mean that any object returned by the shared lib might be unaligned?

Dereferencing unaligned pointers is treated as undefined behaviour in most languages, and is usually tied with a performance hit too. While there are workarounds (like silencing the warning, or handling the unaligned addresses to re-align them) it would be nice to not have to add additional manual memory management when interacting with the core client.

Is this a known issue?

2

Hi! Welcome.

I agree with all points. Unfortunately, misaligned pointers is currently required by the existing ABI, since the the FDBKeyValue type is declared within a pragma pack as follows. There’s no way to respect the pragma pack and align both pointers to 8 bytes.

#pragma pack(push, 4)
...
typedef struct keyvalue {
	const uint8_t* key;
	int key_length;
	const uint8_t* value;
	int value_length;
} FDBKeyValue;
...
#pragma pack(pop)

That said, we do technically have a mechanism for changing the ABI (in new header versions) without impacting existing users in theory.

This is actually referring to a much deeper internal problem, but one that’s mostly an implementation detail and shouldn’t affect the shared lib ABI. There are misaligned accesses happening in the shared lib internally, but rust’s instrumentation presumably won’t be able to see that.

There’s an issue tracking aligning memory properly, but it doesn’t have any momentum right now it seems: Undefined behaviour analyzer : Member access within misaligned address · Issue #1434 · apple/foundationdb · GitHub

1 Like
3

Thanks for the quick response :slight_smile:

I think the issue I’m seeing is not related to the packed struct pointer addresses.

One of the calls that’s affected by this is fdb_transaction_get_range which results in a Future<[keyvalue]>.

This should eventually resolve to a pointed pointing at an array of keyvalue which is pack(4) foundationdb/fdb_c.h at 65ed7775fd5777a95547239b22c91e1800f9cdf3 ¡ apple/foundationdb ¡ GitHub. The expected alignment should be 4.

The resulting pointer has address 0x7f034cefb0a8 and the address it points to is 0x7f034cc916c6. When converting this to a rust pointer and dereferencing, the complaint is that the address that it points to is not a multiple of 4. Indeed the referenced address is simply a multiple of 2. The address of the pointer itself is aligned.

The complaint from the alignment checker is

misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7f034cc916c6'

Which is also referring to the pointed address.

Digging in the FoundationDB side of things, it looks like the referenced address is directly pointing to a Standalone<>, which if I understand correctly is allocated through the Arena and could be at any point in memory.

The “fix” involves reallocating the memory needed for the resulting array, and copying it over, so that makes me think that the packing itself is not a problem because it’s not the pointers’ addresses that are checked here, but the memory that they point to instead.

What do you think? Could this point to the arena allocations leaking to the ABI?

4

Thanks for digging. I think you’re right.

I think this is the case after all

1 Like
5

Quick update here, in case anyone bumps into this in the future. For Rust to understand that the API structs are misaligned, when using rust you need to make sure that you are exposing the packed structs, and not wrapping them in a transparent.

6

Thanks @behos for digging into this issue and @atn34 for confirming. I just ran into this issue with my binding, and in my case decided to use ptr::read_unaligned instead of regular Rust dereference.