FoundationDB on docker/mac m2

Has anybody had any success running foundationdb/foundationdb image on docker? I run with --platform linux/amd64 while there are no apple silicon images available, but I hit the following:

foundation-1  | <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
foundation-1  | <jemalloc>: (This is the expected behaviour if you are running under QEMU)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | Configuring database
foundation-1  | Starting FDB server on 127.0.0.1:4500
foundation-1  | <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
foundation-1  | <jemalloc>: (This is the expected behaviour if you are running under QEMU)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)
foundation-1  | ERROR: Disk i/o operation failed (1510)

I also tried supplying --knob_disable_posix_kernel_aio=1 but it didn’t make a difference.

Could you provide some more information like what Docker image you used? You should make sure to use the images with an even tag e.g. 7.1.28 as those have AVX disabled (I believe that’s not available in qemu).

Based on your comment it seems like you already tried this command: Support Apple Silicon · Issue #4111 · apple/foundationdb · GitHub?

Are you able to share the whole command that you used? I don’t have an M2 available, but I would expect that the M1 instructions work too.

Thanks for your reply, unfortunately that doesn’t quite work either.
I created a small repo for reproducing the issue:

How hard is it to build an apple specific docker image? How long should a build take?

Not a perfect solution but we have an apple silicon version of foundationdb built here Docker that you can try out.

1 Like

Thanks for the reply! Did you use GitHub - FoundationDB/fdb-build-support: FoundationDB build and development resources for building that? I tried on an m2, and also tried cross-building via dockerx but didn’t have any luck with that.

I was able to dig a little further, here’s what I found:

  • to use fdb/docker on an apple sillicon, you need a .pkg with client libraries built for arm64
  • the java fdb lib needs to match the fdb docker
  • you can override lib locations via java properties (eg: -DFDB_LIBRARY_PATH_FDB_C=xxx and -DFDB_LIBRARY_PATH_FDB_JAVA=yyy
  • I extracted the libfdb.java.jnilib from the maven jarfile and put it in /usr/local/lib, then chmod 755 <file>
    • You can test if the libraries are loadable via (System/load "/usr/local/lib...), it shouldn’t barf
      • if the lib is for a different arch, using Azul SDK gives a more meaningful message:
        /usr/local/lib/libfdb_c.dylib: dlopen(/usr/local/lib/libfdb_c.dylib, 0x0001): tried: '/usr/local/lib/libfdb_c.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/lib/libfdb_c.dylib' (no such file), '/usr/local/lib/libfdb_c.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
      • if the versions don’t match, you will see something like: Execution error (UnsatisfiedLinkError) at jdk.internal.loader.NativeLibraries/load (NativeLibraries.java:-2). /usr/local/lib/libfdb_java.jnilib: dlopen(/usr/local/lib/libfdb_java.jnilib, 0x0001): Symbol not found: _fdb_database_blobbify_range Referenced from: <EE0D48C0-F8C3-3599-8EAA-C36B69D28883> /usr/local/lib/libfdb_java.jnilib Expected in: <24069827-7F2F-30D3-B9B5-B78D6EF7B8DA> /usr/local/lib/libfdb_c.dylib
1 Like

So, the last version that works for me (on an m2) is foundationdb/foundationb:7.1.24. 7.1.26 throws the illegal instruction thingy:

docker run -ti --platform=linux/amd64 --entrypoint=bash -v $(pwd)/foundationdb:/fdb  foundationdb/foundationdb:7.1.24
[root@8d6f4438923e /]# sed -i '$ s/$/ --knob_disable_posix_kernel_aio=1/' /var/fdb/scripts/fdb.bash
[root@8d6f4438923e /]# /var/fdb/scripts/fdb.bash
Starting FDB server on 172.17.0.2:4500
FDBD joined cluster.
^C
[root@8d6f4438923e /]# exit


docker run -ti --platform=linux/amd64 --entrypoint=bash -v $(pwd)/:/fdb  foundationdb/foundationdb:7.1.26
[root@699567e0b323 /]# sed -i '$ s/$/ --knob_disable_posix_kernel_aio=1/' /var/fdb/scripts/fdb.bash; /var/fdb/scripts/fdb.bash
Starting FDB server on 172.17.0.2:4500
/var/fdb/scripts/fdb.bash: line 68:    26 Illegal instruction     fdbserver --listen-address 0.0.0.0:"$FDB_PORT" --public-address "$PUBLIC_IP:$FDB_PORT" --datadir /var/fdb/data --logdir /var/fdb/logs --locality-zoneid="$(hostname)" --locality-machineid="$(hostname)" --class "$FDB_PROCESS_CLASS" --knob_disable_posix_kernel_aio=1
[root@699567e0b323 /]# exit

Unfortunately, 7.1.24 does not have the .pkg client library download for arm64.
Is anybody able to run 7.1.26 or higher on an m2?

FWIW things seem to work ok outside docker. On my m1 I downloaded and installed https://github.com/apple/foundationdb/releases/download/7.1.31/FoundationDB-7.1.31_arm64.pkg, downloaded https://repo1.maven.org/maven2/org/foundationdb/fdb-java/7.1.31/fdb-java-7.1.31.jar, and was able to run the example code from Overview (FoundationDB Java Client API).

$ cat Example.java
import com.apple.foundationdb.Database;
import com.apple.foundationdb.FDB;
import com.apple.foundationdb.tuple.Tuple;

public class Example {
  public static void main(String[] args) {
    FDB fdb = FDB.selectAPIVersion(710);

    try(Database db = fdb.open()) {
      // Run an operation on the database
      db.run(tr -> {
        tr.set(Tuple.from("hello").pack(), Tuple.from("world").pack());
        return null;
      });

      // Get the value of 'hello' from the database
      String hello = db.run(tr -> {
        byte[] result = tr.get(Tuple.from("hello").pack()).join();
        return Tuple.fromBytes(result).getString(0);
      });

      System.out.println("Hello " + hello);
    }
  }
}

$ javac -cp fdb-java-7.1.31.jar:. Example.java

$ java -cp fdb-java-7.1.31.jar:. Example
Hello world

$ fdbcli --exec 'getrange "" \xff'

Range limited to 25 keys
`\x02hello\x00' is `\x02world\x00'

I had exactly the same problems with the official docker images. I also found that after version 7.1.24, all images cause the Illegal instruction error. For both Qemu and Mac OS Virtualization Framework.

I made a prototype of a native arm64 container build based on Fedora 40. @mping-exo you can take my code here and use it any way you want.

1 Like

CC @ammolitor are you aware of any changes in the build process that could cause those Illegal instruction errors?

Thanks alot, I’ll definitely take a look!