Transaction Log files Info and three_data_center data replication model

jarvis · June 23, 2020, 6:13pm

I have started working with FDB recently. Could anyone please expand on the following:

What are the different types that are maintained in foundation db trace files like “GetValueDebug” etc ?
What is the location of transaction log file? Is it stored in the data dir as configured in the configuation file ? and is there any prefix for the log files ? Also, if you could expound on the format of data stored in transaction ogs ?

I also have few doubt related to three_datacenter_mode:

Doc says that TLog processes are maintained in 2 data centers and the third datacenter only has storage server processes, how is the data replicated in the third datacenter and the second datacenter ( assuming second and first data center both have tlogs and client has started transaction request from first datacenter) ?
Since three_datacenter_mode is based upon synchronous data replication, and commit latencies observe round trip latencies also, wanted to know what is the process of data replication here, couldn’t find any detailed article

jarvis · June 23, 2020, 6:26pm

@ajbeamon @alloc if you guys have any insights

ajbeamon · July 2, 2020, 3:42pm

What are the different types that are maintained in foundation db trace files like “GetValueDebug” etc ?

We don’t really have much documentation for the contents of trace files. Probably the best we have are a couple partially completed wiki pages:

The transaction logs’ data files (rather than trace logs) are stored in the data directory. There are two sets of files here:

Disk queue - These files have the form logqueue-*.fdq. All mutations sent to a transaction log get appended to one of these, and storage servers gradually pop data off the front.

Persistent data - These files are usually b-tree files using the ssd-2 storage engine, and in that case will have the form log*.sqlite and log*.sqlite-wal. If the disk queue gets “full”, then data gets spilled to this data structure from the queue until it can be popped or is no longer needed.

The filenames of both of these also contain some version information about the log system used to generate them. There are some good write-ups of the design of this in these documents:

github.com

apple/foundationdb/blob/main/design/tlog-spilling.md.html

<meta charset="utf-8">

# TLog Spill-By-Reference Design

## Background

(This assumes a basic familiarity with [FoundationDB's architecture](https://www.youtu.be/EMwhsGsxfPU).)

Transaction logs are a distributed Write-Ahead-Log for FoundationDB.  They
receive commits from commit proxies, and are responsible for durably storing 
those commits, and making them available to storage servers for reading.

Clients send *mutations*, the list of their set, clears, atomic operations,
etc., to commit proxies. Commit proxies collect mutations into a *batch*, which
is the list of all changes that need to be applied to the database to bring it
from version `N-1` to `N`. Commit proxies then walk through their in-memory 
mapping of shard boundaries to associate one or more *tags*, a small integer 
uniquely identifying a destination storage server, with each mutation. They 
then send a *commit*, the full list of `(tags, mutation)` for each mutation in
a batch, to the transaction logs.

This file has been truncated. show original

github.com

apple/foundationdb/blob/main/design/tlog-forward-compatibility.md.html

<meta charset="utf-8">

# Forward Compatibility for Transaction Logs

## Background

A repeated concern with adopting FoundationDB has been that upgrades are one
way, with no supported rollback.  If one were to upgrade a cluster running 6.0
to a 6.1, then there's no way to roll back to 6.0 if the new version results in
worse client application performance or unavailability.  In the interest of
increasing adoption, work has begun on supporting on-disk forward
compatibility, which allows for upgrades to be rolled back.

The traditional way of allowing roll backs is to have one version, `N`, that
introduces a feature, but is left as disabled.  `N+1` enables the feature, and
then `N+2` removes whatever was deprecated in `N`.  However, FDB currently has
a 6 month release cadence, and waiting 6 months to be able to use a new feature
in production is unacceptably long.  Thus, the goal is to have a way to be able
to have a sane and user-friendly, rollback-supporting upgrade path, but still
allow features to be used immediately if desired.

This file has been truncated. show original

I don’t have a lot of experience with three_datacenter mode, but I think this works fairly similarly to normal configurations. Commits must synchronously write data to all the logs in the two datacenters, and storage servers asynchronously grab data from the logs.

A commit from the primary datacenter would have to interact with that datacenter to initiate the commit, the data would get written to the first and second datacenter transaction logs, and then the commit would succeed. Asynchronously, the storage servers in all datacenters would grab the updated data from their transaction logs to ensure that it ends up everywhere.

Topic		Replies	Views
Create audit logs from transaction log or redo log Running FoundationDB	2	424	July 2, 2022
Transaction Log Using FoundationDB	3	2783	October 22, 2018
Logging of transactions for debugging purposes Using FoundationDB	5	2265	October 22, 2018
Is there more detailed design documents? FoundationDB Core	4	1506	April 30, 2018
What are the functions of the files in the fdb data directory? Using FoundationDB	0	379	October 20, 2022

Transaction Log files Info and three_data_center data replication model

Related Topics