# Transaction Log files Info and three\_data\_center data replication model

**URL:** <https://forums.foundationdb.org/t/transaction-log-files-info-and-three-data-center-data-replication-model/2202>\
**Category:** Using FoundationDB\
**Created:** [June 23, 2020, 6:13pm UTC](https://forums.foundationdb.org/t/transaction-log-files-info-and-three-data-center-data-replication-model/2202 "2020-06-23T18:13:23Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jarvis](https://avatars.discourse-cdn.com/v4/letter/j/e36b37/32.png) [@jarvis](https://forums.foundationdb.org/u/jarvis)\
**Post date:** [June 23, 2020, 6:13pm UTC](https://forums.foundationdb.org/t/transaction-log-files-info-and-three-data-center-data-replication-model/2202/1 "2020-06-23T18:13:23Z")

</div>

I have started working with FDB recently. Could anyone please expand on the following:

1. What are the different types that are maintained in foundation db trace files like “GetValueDebug” etc ?
2. What is the location of transaction log file? Is it stored in the data dir as configured in the configuation file ? and is there any prefix for the log files ? Also, if you could expound on the format of data stored in transaction ogs ?

I also have few doubt related to three\_datacenter\_mode:

1. Doc says that TLog processes are maintained in 2 data centers and the third datacenter only has storage server processes, how is the data replicated in the third datacenter and the second datacenter ( assuming second and first data center both have tlogs and client has started transaction request from first datacenter) ?

2. Since three\_datacenter\_mode is based upon synchronous data replication, and commit latencies observe round trip latencies also, wanted to know what is the process of data replication here, couldn’t find any detailed article

---

<div class="post-metadata">

**Author:** ![jarvis](https://avatars.discourse-cdn.com/v4/letter/j/e36b37/32.png) [@jarvis](https://forums.foundationdb.org/u/jarvis)\
**Post date:** [June 23, 2020, 6:26pm UTC](https://forums.foundationdb.org/t/transaction-log-files-info-and-three-data-center-data-replication-model/2202/2 "2020-06-23T18:26:22Z")

</div>

@ajbeamon @alloc if you guys have any insights

---

<div class="post-metadata">

**Author:** ![ajbeamon](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/ajbeamon/32/13_2.png) [@ajbeamon](https://forums.foundationdb.org/u/ajbeamon)\
**Post date:** [July 2, 2020, 3:42pm UTC](https://forums.foundationdb.org/t/transaction-log-files-info-and-three-data-center-data-replication-model/2202/3 "2020-07-02T15:42:02Z")

</div>

> 1. What are the different types that are maintained in foundation db trace files like “GetValueDebug” etc ?

We don’t really have much documentation for the contents of trace files. Probably the best we have are a couple partially completed wiki pages:

> **[Understanding FoundationDB Log Files](https://github.com/apple/foundationdb/wiki/Understanding-FoundationDB-Log-Files)**
>
> FoundationDB - the open source, distributed, transactional key-value store - apple/foundationdb

> **[FoundationDB Trace Metrics Definitions](https://github.com/apple/foundationdb/wiki/FoundationDB-Trace-Metrics-Definitions)**
>
> FoundationDB - the open source, distributed, transactional key-value store - apple/foundationdb

> [@jarvis](#):
>
> What is the location of transaction log file? Is it stored in the data dir as configured in the configuation file ? and is there any prefix for the log files ? Also, if you could expound on the format of data stored in transaction ogs ?

The transaction logs’ data files (rather than trace logs) are stored in the data directory. There are two sets of files here:

Disk queue - These files have the form `logqueue-*.fdq`. All mutations sent to a transaction log get appended to one of these, and storage servers gradually pop data off the front.

Persistent data - These files are usually b-tree files using the `ssd-2` storage engine, and in that case will have the form `log*.sqlite` and `log*.sqlite-wal`. If the disk queue gets “full”, then data gets spilled to this data structure from the queue until it can be popped or is no longer needed.

The filenames of both of these also contain some version information about the log system used to generate them. There are some good write-ups of the design of this in these documents:

> <https://github.com/apple/foundationdb/blob/main/design/tlog-spilling.md.html>

> <https://github.com/apple/foundationdb/blob/main/design/tlog-forward-compatibility.md.html>

> [@jarvis](#):
>
> - Doc says that TLog processes are maintained in 2 data centers and the third datacenter only has storage server processes, how is the data replicated in the third datacenter and the second datacenter ( assuming second and first data center both have tlogs and client has started transaction request from first datacenter) ?
> - Since three\_datacenter\_mode is based upon synchronous data replication, and commit latencies observe round trip latencies also, wanted to know what is the process of data replication here, couldn’t find any detailed article

I don’t have a lot of experience with `three_datacenter` mode, but I think this works fairly similarly to normal configurations. Commits must synchronously write data to all the logs in the two datacenters, and storage servers asynchronously grab data from the logs.

A commit from the primary datacenter would have to interact with that datacenter to initiate the commit, the data would get written to the first and second datacenter transaction logs, and then the commit would succeed. Asynchronously, the storage servers in all datacenters would grab the updated data from their transaction logs to ensure that it ends up everywhere.
