How to actually use FDB OpenTelemetry tracing?

Using FoundationDB
1

Is there anyone using OpenTelemetry tracing in production? If so, how is it configured? Would love a detailed example, since the docs are not helpful.

I tried a simple example:

First, create a fake trace collector listening to the default port:

import socket

# Listen on UDP 0.0.0.0:8889
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_address = ('0.0.0.0', 8889)
sock.bind(server_address)

print("Listening for UDP packets on 0.0.0.0:8889...")

try:
    while True:
        # Receive packet (buffer size 4096 bytes)
        data, address = sock.recvfrom(4096)
        print(f"Received packet from {address}")

except KeyboardInterrupt:
    print("\nStopping UDP listener...")
finally:
    sock.close()

Now, create a Python script to read/write against FDB:

import fdb
import time

fdb.api_version(730)
fdb.options.set_distributed_client_tracer("network_lossy")

@fdb.transactional
def set_and_get(tr):
    tr[b'abc'] = str(time.time()).encode('utf-8')
    print(tr[b'abc'])

db = fdb.open()
set_and_get(db)

Even after setting the “distributed client tracer” mode to “network_lossy”, no received packets are received.

Using fdbcli and setting network option environment variables as suggested in the docs doesn’t do anything either:

% FDB_NETWORK_OPTION_DISTRIBUTED_CLIENT_TRACER=network_lossy FDB_NETWORK_OPTION_TRACING_SAMPLE_RATE=1.0 fdbcli 
Using cluster file `/usr/local/etc/foundationdb/fdb.cluster'.

The database is available.

Welcome to the fdbcli. For help, type `help'.
fdb> writemode on
fdb> set efg abc
Committed (2714370047951)
fdb> get efg
`efg' is `abc'
fdb> get abc
`abc' is `1745899487.4607291'
fdb>

I also tried the environment variables against the Python script above.

I have even set these knobs

% fdbcli                   
Using cluster file `/usr/local/etc/foundationdb/fdb.cluster'.

The database is available.

Welcome to the fdbcli. For help, type `help'.
fdb> getknob tracing_udp_listener_port
`tracing_udp_listener_port' is `8889'
fdb> getknob tracing_udp_listener_addr
`tracing_udp_listener_addr' is `'127.0.0.1''
fdb> getknob tracing_sample_rate
`tracing_sample_rate' is `1.000000'
fdb>

I realize I also need to set up the server side. I have set this custom parameter in the Kubernetes operator config, but I haven’t tested if it works or not:

    general:
      customParameters:
        - "tracer=network_lossy"
2

I think this feature is never completed. It’d be nice someone can make it work.