Document layer unexpectedly slow with naive test

Using Layers Document Layer
1

I’m evaluating FoundationDB and thought an interesting first step would be to try out the Document Layer. I wrote a simple app talking to MongoDB which loads about 3MB (in text) of data, and does a few queries to make sure the data loaded as expected. It takes less than a second to run in MongoDB. I then pointed the same app at the document layer, it timed out after 660 seconds (which is 11 minutes for those of you counting). Note that both of these are docker containers running on my Mac Pro (the trash can, not the cheese grater). It does make progress through the test, just very slowly.

I’m not even sure where to begin with this. I certainly wasn’t expecting it to be 1000x slower. Is Document Layer intended to be used? Or is it just a demo?

Thanks,
Greg

2

Okay, I decided to start by profiling. And, of course, it wasn’t where I thought it was. The better MongoDB comparison was < 0.5 seconds and 6.3 seconds for FoundationDB. So it’s only 12x slower. That I can start to figure out what to do with. The timeout was caused by the docker container hanging on shutdown from a deferred task, so it showed up as failing in the test. The container does not respond to signals correctly.

Is anyone working on the Document Layer? It looks like there haven’t been any changes for the docker container in 9 months or so. (And 5 months since any changes).

Thanks,
Greg

3

Hi Greg:

I am going down this path as well and I was wondering how you got the document layer talking to foundationDB inside docker containers in your mac. I have them as two separate containers and have a basic docker-compose to wire them up. The containers seem to start but I have no luck connecting to fdb-document-layer and there seems to connection errors in the fdb-document-layer log files. Any ideas ?

<Event Severity="40" Time="1601782864.377070" Type="StartupFailure" ID="0000000000000000" phase="ConnectToCluster" timeout="1945" Backtrace="addr2line -e fdbdoc.debug -p -C -f -i 0x686074 0x6840df 0x454b2d 0x459585 0x457821 0x41a731 0x6b535f 0x448793 0x7f806b88db97" Machine="127.0.0.1:27017" LogGroup="default" />

# cat fdb.cluster
docker:docker@172.20.0.2:4500

$ docker ps
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS              PORTS                      NAMES
764b110b3dfe        foundationdb/fdb-document-layer:1.7.2   "/bin/sh -c /var/fdb…"   44 minutes ago      Up 44 minutes       0.0.0.0:27017->27017/tcp   foundation-db_fdb-doc-layer_1
c7eb5283aaaf        foundationdb/foundationdb:6.3.8         "/bin/sh -c /var/fdb…"   44 minutes ago      Up 44 minutes       0.0.0.0:4500->4500/tcp     foundation-db_fdb_1

$ docker exec foundation-db_fdb_1 fdbcli --exec status
Using cluster file `/var/fdb/fdb.cluster'.

Configuration:
  Redundancy mode        - single
  Storage engine         - ssd-2
  Coordinators           - 1
  Usable Regions         - 1

Cluster:
  FoundationDB processes - 1
  Zones                  - 1
  Machines               - 1
  Memory availability    - 5.6 GB per process on machine with least available
  Fault Tolerance        - 0 machines
  Server time            - 10/04/20 03:54:03

Data:
  Replication health     - Healthy
  Moving data            - 0.000 GB
  Sum of key-value sizes - 0 MB
  Disk space used        - 210 MB

Operating space:
  Storage server         - 31.6 GB free on most full server
  Log server             - 31.6 GB free on most full server

Workload:
  Read rate              - 7 Hz
  Write rate             - 0 Hz
  Transactions started   - 3 Hz
  Transactions committed - 0 Hz
  Conflict rate          - 0 Hz

Backup and DR:
  Running backups        - 0
  Running DRs            - 0

Client time: 10/04/20 03:54:03