# CometDB: A FoundationDB queue manager for Erlang

**URL:** <https://forums.foundationdb.org/t/cometdb-a-foundationdb-queue-manager-for-erlang/2991>\
**Category:** Using FoundationDB\
**Tags:** bindings\
**Created:** [November 4, 2021, 6:57am UTC](https://forums.foundationdb.org/t/cometdb-a-foundationdb-queue-manager-for-erlang/2991 "2021-11-04T06:57:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![wfnuser](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/wfnuser/32/1150_2.png) [@wfnuser](https://forums.foundationdb.org/u/wfnuser)\
**Post date:** [November 4, 2021, 6:57am UTC](https://forums.foundationdb.org/t/cometdb-a-foundationdb-queue-manager-for-erlang/2991/1 "2021-11-04T06:57:34Z")

</div>

The project link is [here](https://github.com/wfnuser/CometDB).

# CometDB

CometDB is an Erlang implementation of queue model over FoundationDB.

## About

This project aims to provide you a set of simple interfaces so that you can use FoundationDB as a Distributed, High Concurrency and Low Latency Queue.

## Features

- You can insert data in batch.
- You can fetch data in batch.
- You can delete data in range.
- You can insert data of any size; we overcome the size limitation of FDB.

## How to use

We need to setup the FDB environment first. You can find details in [FoundationDB official website](https://apple.github.io/foundationdb/getting-started-linux.html).

The usage is really simple. Here is an example:

```auto
P = comet:start(). 
comet:insert(P, <<"testq">>, <<"testvalue">>). 
comet:batch_insert(P, <<"testq">>, [<<"testvalue_2">>, <<"testvalue_3">>, <<"testvalue_4">>]). comet:fetch(P, <<"testq">>, 100). 
comet:delete(P, <<"testq">>, 1).

```

## TODO

- type checking
- stream
- cursors
- index
- priority queue
- connection management
- performance
- tests …

---

<div class="post-metadata">

**Author:** ![cjbottaro](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/cjbottaro/32/1256_2.png) [@cjbottaro](https://forums.foundationdb.org/u/cjbottaro)\
**Post date:** [December 8, 2021, 5:41pm UTC](https://forums.foundationdb.org/t/cometdb-a-foundationdb-queue-manager-for-erlang/2991/2 "2021-12-08T17:41:01Z")

</div>

Hello fellow BEAM user… 🙂 Very cool!

I’m looking to use FoundationDB to implement queues also and I came across this in the documentation:

> _High-Contention Dequeue Operations_
> 
> To minimize conflicts during dequeue operations, we can use a staging technique to service the requests. If a dequeue operation doesn’t initially succeed, it registers a dequeue request in a semi-ordered set of such requests. It then enters a retry loop in which it attempts to fulfill outstanding requests.

Did you implement that algorithm? Or how did you deal with high contention consumers?

Do you know how that “staging” algorithm works? Can you help explain it?

Thanks for the help!

---

<div class="post-metadata">

**Author:** ![wfnuser](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/wfnuser/32/1150_2.png) [@wfnuser](https://forums.foundationdb.org/u/wfnuser)\
**Post date:** [April 15, 2022, 8:45am UTC](https://forums.foundationdb.org/t/cometdb-a-foundationdb-queue-manager-for-erlang/2991/3 "2022-04-15T08:45:19Z")

</div>

Hi. Sorry for the late reply. We didn’t use foundationdb in our product environment at last. So I didn’t notice the comments.

I know little about “staging” algorithm… But if you need any help just let me know. Maybe we can build a solid queue model for FoundationDB in erlang together.

---

<div class="post-metadata">

**Author:** ![arun-pathiban-ddog](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/arun-pathiban-ddog/32/1618_2.png) [@arun-pathiban-ddog](https://forums.foundationdb.org/u/arun-pathiban-ddog)\
**Post date:** [June 26, 2023, 2:52pm UTC](https://forums.foundationdb.org/t/cometdb-a-foundationdb-queue-manager-for-erlang/2991/4 "2023-06-26T14:52:17Z")

</div>

@wfnuser Hi! I’m interested in knowing more about your staging algorithm. Could you please describe it?

---

<div class="post-metadata">

**Author:** ![Rjerk](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/rjerk/32/1056_2.png) [@Rjerk](https://forums.foundationdb.org/u/Rjerk)\
**Post date:** [June 29, 2023, 7:00am UTC](https://forums.foundationdb.org/t/cometdb-a-foundationdb-queue-manager-for-erlang/2991/5 "2023-06-29T07:00:42Z")

</div>

I’m not familiar with Erlang, it would be nice if you could describe your implementation of the `Distributed, High Concurrency and Low Latency Queue `.