FoundationDB Meetup in San Jose

Hello community!

My name is Aravind. I would like to announce a monthly FoundationDB meetup that I’m helping organize in Adobe’s HQ in San Jose, CA.

Our main objective is to build a supportive environment where people who work with FoundationDB can connect, share their ideas, and learn from one another’s experiences. Additionally, we provide opportunities for networking and informal knowledge exchange, allowing attendees to forge meaningful connections and expand their professional networks.

If you are interested in attending, please RVSP on the meetup.com page. If you would like to present in the meetup, please send me a message via the meetup page.

The first event is scheduled for 2024-05-16T01:00:00Z.

How Tigris Leverages FoundationDB for Global Metadata Storage
Himank Chaudhary [Co-founder & CTO, Tigris Data]

Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk we will share the technical details around why we chose FoundationDB and how we are using it at scale to power our object storage service.

Planned upcoming talks

  • FDB Cluster Management - Senthil Kumar Ramamoorthy [Snowflake]
  • Benchmarking FDB in Azure and AWS clouds - Aravind Sriram [Adobe]
  • Building a GraphDB using FDB
  • Consistent replication between FDB and DataLake
7 Likes

Do you plan to have recordings available for talks? Cannot attend but the topics seem very interesting.

1 Like

Yes, the talks will be recorded and made available on youtube.

You can also attend the meetup remotely. I’ll add the details to the meetup page next week.

2 Likes

Very cool. I would definitely join is I was local. At the very least I’ll be attending remotely.

1 Like

We will have two interesting talks at this inaugural hybrid meetup event - FoundationDB Meetup [In-Person and Online], Wed, May 15, 2024, 6:00 PM | Meetup

Talk 1 :
Title: How Tigris Leverages FoundationDB for Global Metadata Storage
Speaker: Himank Chaudhary (Co-founder/CTO, Tigris)
Speaker Bio:
Himank is the Co-Founder and CTO of Tigris Data where he is building a globally distributed object storage service. Prior to co-founding Tigris, Himank was the tech lead of Docstore, Uber’s distributed database that powered petabytes of data and facilitated millions of queries per second in a geo-distributed hybrid cloud deployment. Himank Chaudhary - Tigris Data Inc | LinkedIn
Talk Summary:
Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk, Himank will share the technical details around why they chose FoundationDB and how we are using it at scale to power our object storage service.

Talk 2:
Title: Encryption in FoundationDB
Speaker: Ata E Husain Bohra (Snowflake)
Speaker Bio:
Ata is a core member of Snowflake’s FoundationDB team. Snowflake has been working with FoundationDB for more than a decade. Ata is an industry veteran with over 10 years of experience designing/implementing large-scale distributed systems. Prior to Snowflake, he played an instrumental role in building and productionizing Uber’s Distributed Ledger platform. He is a startup enthusiast and has worked at multiple successful startups as a core member of the distributed filesystem team. Ata E Husain Bohra - Snowflake | LinkedIn
Talk Summary:
Snowflake enhanced FoundationDB by designing/implementing an enterprise-grade native encryption data-at-rest solution. The talk deep-dives into encryption at-rest architecture, supported configuration modes, and ability to integrate with existing enterprise’s encryption infrastructure.

1 Like

Hi Folks,

I’m delighted to share that our monthly FoundationDB meetup initiative is off to a fantastic start! Our inaugural meetup saw over 60 attendees, including key members of the FoundationDB team from Apple and Snowflake such as Jingyu Zhou, Joshua McManus, Ata E Husain Bohra and Stephen Pimentel. Thanks to everyone who was able to make it to the event. It was a pleasure connecting with you all! If anyone is interested in presenting at the meetup (remote or in-person), please reach out to me.

The next meetup event is scheduled for Jun 20th at Adobe HQ in San Jose. Please RSVP here.

Recording of the talks

Snowflake enhanced FoundationDB by designing/implementing an enterprise-grade native encryption data-at-rest solution. In this talk, Ata E Husain Bohra (Technical lead manager, Snowflake) will deep dive on the encryption at-rest architecture, supported configuration modes, and ability to integrate with existing enterprise’s encryption infrastructure.


Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk, Himank Chaudhary (Co-founder & CTO, Tigris data) will share the technical details around why they chose FoundationDB and how we are using it at scale to power our object storage service.


3 Likes

This month’s meetup will take place in Adobe’s San Jose office on Thursday, June 20th at 6PM PST.

I will be presenting on Adobe’s FDB usage and journey. RSVP here to reserve your spot for either in-person or remote attendance -
FoundationDB Meetup [In-Person and Online], Thu, Jun 20, 2024, 6:00 PM | Meetup

Talk Summary:
Adobe Experience Platform’s core capability is delivering personalized customer experiences at scale. Identity Graphs are an integral concept to making this a reality. Adobe’s Identity Graph system manages over 50 billion identities, enables real-time ingestion of over 400,000 messages per second, and supports millions of API lookups per second.

In this talk, Aravind will provide insights into this large-scale distributed system and deep dive into how FoundationDB is leveraged as the backing store for the Identity Graph system. He will also dive into Adobe’s FoundationDB journey, including benchmarking FoundationDB in Azure and AWS clouds, Kubernetes vs VM deployment, fault-tolerance testing, and HA strategies.

Recording of the talk and Q& A

2 Likes

Join us for this month’s FDB meetup on the 24th @ Adobe HQ in downtown San Jose !!

This meetup is a hybrid event (in-person + remote). RSVP here to secure your spot. Online meeting link will be made available closer to the event. I hope to see you all there!

Agenda:

  • Social Hour - 30 mins
  • Talk: Designing a control plane for FDB fleet management - 45 mins
  • Talk: Spark connector for FDB - 15 mins

Talk Title: Designing a control plane for FDB fleet management
Speaker: Senthil Kumar Ramamoorthy (Senior Engineer, Snowflake)
Speaker Bio:
With over two decades of experience, Senthil has made significant contributions to the development and optimization of advanced storage technologies. His expertise spans snapshots, replication, name-space management, space accounting, garbage collection, cache, and performance optimization across NFS and CIFS protocols. He is also an inventor with multiple patents related to storage efficiency and optimization.
Currently, he is an engineer at Snowflake, where he led the development of the snapshot-based backup and restore feature. He is also the lead engineer for designing and implementing the control plane for FDB fleet management and is a committer for FoundationDB. Senthil brings deep expertise in both the architectural and practical aspects of storage technologies and is passionate about optimizing storage performance for modern distributed systems.
Talk Summary:
In this talk, Senthil will delve into how FoundationDB is managed at scale at Snowflake, with a focus on the control plane for fleet management. He will discuss key aspects such as provisioning, scaling, autoscaling, and load balancing across large fleets of FDB clusters. The session will also cover the challenges of managing distributed systems at scale, upcoming enhancements in fleet management, and lessons learned from real-world deployments.

Talk Title: Spark connector for FDB
Speaker: Akanksha Nagpal (Senior Engineer, Adobe)
Speaker Bio: Akanksha is a Software Engineer with extensive experience in designing and building large-scale distributed systems within the Adobe Experience Platform (AEP). She has led the development of systems for efficient data ingestion into Adobe’s Identity Graph, including building highly scalable data pipelines with Apache Spark and Scala. Akanksha has also played key role in exploring FoundationDB’s architecture and encryption capabilities, focusing on its potential for Customer Managed Keys (CMK) within Adobe’s infrastructure. Passionate about innovation in data processing, she is dedicated to contributing to the broader engineering community.
Talk Summary:
This talk will delve into how Adobe developed a custom Spark connector for FoundationDB to enhance data processing capabilities, addressing limitations of existing solutions. Akanksha will discuss the motivation behind this development, how the connector integrates Spark with FoundationDB for large-scale data operations, and its role in powering customer insights and reporting in AEP. She will also share key lessons learned, operational insights for managing the connector in production, and Adobe’s future roadmap, including planned enhancements and potential open-source contributions.

Recording of the talk and Q& A

Join us for the first fully Virtual FoundationDB Meetup with guest speakers from Germany and India! The event will take place at 9 AM PST tomorrow (Nov 20th)*

RSVP here to secure your spot. Online meeting link will be visible once you RSVP on meetup.com.
Folks from Europe and India regions we hope to see you there!

Agenda:

  • Ice Breakers - 10 mins
  • Talk: FoundationDB on Kubernetes - 45 mins
  • Talk: FoundationDB Infrastructure Excellence - 20 mins

Talk Title: FoundationDB on Kubernetes
Speaker: Johannes Scheuermann (Site Reliability Engineer, Apple)
Speaker Bio:
With over a decade of experience, Johannes started his journey in the Kubernetes eco-system in early 2015 by on-boarding projects and applications onto Kubernetes. Since 2020, Johannes works as an SRE for FoundationDB at Apple and is leading the development of the open source FoundationDB operator.
Talk Summary:
In this talk, Johannes covers the design of the FoundationDB operator on Kubernetes, emphasizing the approaches to overcome limitations in Kubernetes and highlights implementation choices to address unique aspects of the FoundationDB Operator. He will outline the different options to run an highly available FoundationDB Cluster on Kubernetes. Additionally, Johannes will cover the challenges with running stateful services at scale on Kubernetes and different ways it can be managed. The talk concludes with an outlook on future design changes and planned features for the FoundationDB operator.

Talk Title: FoundationDB Infrastructure Excellence
Speaker: Aakash Setia (Site Reliability Engineer, Adobe) and Deekshit Kumar GVD (Site Reliability Engineer, Adobe)
Speaker Bio:
Aakash has over 10 years of experience in building reliable, scalable, and secure systems. Currently at Adobe for the past 3+ years, he has focused on infrastructure setup for multi-cloud environment and streamlining FoundationDB provisioning to enhance efficiency and usability.

Deekshit is experienced in multi-cloud providers like AWS, Azure and GCP, specializing in provisioning distributed databases such as Elasticsearch, Couchbase, Redis, Cassandra, Druid. For the past 2+ years at Adobe, he has focused on ensuring the operational excellence of FoundationDB in Azure and AWS environments
Talk Summary:
This talk will delve into some high-level concepts for provisioning and configuration of FoundationDB database. They will explore how to achieve operational readiness for FDB, focusing on essential aspects like key metrics for monitoring, logging and debugging in production environments. Additionally, they will focus on key learnings such as perpetual storage wiggle and usage of open source utilities, showcasing how they optimize and streamline production FDB deployments.

1 Like

Hi @arsriram - was the virtual meetup on Nov 20 recorded?

Hi @swr, It was recorded. Due to the holiday break, I will be posting links next week. Thanks!

1 Like

Talk recordings from the Nov 20th online meetup -

@swr ^^

3 Likes