My name is Aravind. I would like to announce a monthly FoundationDB meetup that I’m helping organize in Adobe’s HQ in San Jose, CA.
Our main objective is to build a supportive environment where people who work with FoundationDB can connect, share their ideas, and learn from one another’s experiences. Additionally, we provide opportunities for networking and informal knowledge exchange, allowing attendees to forge meaningful connections and expand their professional networks.
If you are interested in attending, please RVSP on the meetup.com page. If you would like to present in the meetup, please send me a message via the meetup page.
The first event is scheduled for 2024-05-16T01:00:00Z.
How Tigris Leverages FoundationDB for Global Metadata Storage Himank Chaudhary [Co-founder & CTO, Tigris Data]
Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk we will share the technical details around why we chose FoundationDB and how we are using it at scale to power our object storage service.
Talk 1 : Title: How Tigris Leverages FoundationDB for Global Metadata Storage Speaker: Himank Chaudhary (Co-founder/CTO, Tigris) Speaker Bio:
Himank is the Co-Founder and CTO of Tigris Data where he is building a globally distributed object storage service. Prior to co-founding Tigris, Himank was the tech lead of Docstore, Uber’s distributed database that powered petabytes of data and facilitated millions of queries per second in a geo-distributed hybrid cloud deployment. Himank Chaudhary - Tigris Data Inc | LinkedIn Talk Summary:
Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk, Himank will share the technical details around why they chose FoundationDB and how we are using it at scale to power our object storage service.
Talk 2: Title:Encryption in FoundationDB Speaker: Ata E Husain Bohra (Snowflake) Speaker Bio:
Ata is a core member of Snowflake’s FoundationDB team. Snowflake has been working with FoundationDB for more than a decade. Ata is an industry veteran with over 10 years of experience designing/implementing large-scale distributed systems. Prior to Snowflake, he played an instrumental role in building and productionizing Uber’s Distributed Ledger platform. He is a startup enthusiast and has worked at multiple successful startups as a core member of the distributed filesystem team. Ata E Husain Bohra - Snowflake | LinkedIn Talk Summary:
Snowflake enhanced FoundationDB by designing/implementing an enterprise-grade native encryption data-at-rest solution. The talk deep-dives into encryption at-rest architecture, supported configuration modes, and ability to integrate with existing enterprise’s encryption infrastructure.
I’m delighted to share that our monthly FoundationDB meetup initiative is off to a fantastic start! Our inaugural meetup saw over 60 attendees, including key members of the FoundationDB team from Apple and Snowflake such as Jingyu Zhou, Joshua McManus, Ata E Husain Bohra and Stephen Pimentel. Thanks to everyone who was able to make it to the event. It was a pleasure connecting with you all! If anyone is interested in presenting at the meetup (remote or in-person), please reach out to me.
The next meetup event is scheduled for Jun 20th at Adobe HQ in San Jose. Please RSVP here.
Snowflake enhanced FoundationDB by designing/implementing an enterprise-grade native encryption data-at-rest solution. In this talk, Ata E Husain Bohra (Technical lead manager, Snowflake) will deep dive on the encryption at-rest architecture, supported configuration modes, and ability to integrate with existing enterprise’s encryption infrastructure.
Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk, Himank Chaudhary (Co-founder & CTO, Tigris data) will share the technical details around why they chose FoundationDB and how we are using it at scale to power our object storage service.
Talk Summary:
Adobe Experience Platform’s core capability is delivering personalized customer experiences at scale. Identity Graphs are an integral concept to making this a reality. Adobe’s Identity Graph system manages over 50 billion identities, enables real-time ingestion of over 400,000 messages per second, and supports millions of API lookups per second.
In this talk, Aravind will provide insights into this large-scale distributed system and deep dive into how FoundationDB is leveraged as the backing store for the Identity Graph system. He will also dive into Adobe’s FoundationDB journey, including benchmarking FoundationDB in Azure and AWS clouds, Kubernetes vs VM deployment, fault-tolerance testing, and HA strategies.
Join us for this month’s FDB meetup on the 24th @ Adobe HQ in downtown San Jose !!
This meetup is a hybrid event (in-person + remote). RSVP here to secure your spot. Online meeting link will be made available closer to the event. I hope to see you all there!
Agenda:
Social Hour - 30 mins
Talk: Designing a control plane for FDB fleet management - 45 mins
Talk: Spark connector for FDB - 15 mins
Talk Title: Designing a control plane for FDB fleet management Speaker: Senthil Kumar Ramamoorthy (Senior Engineer, Snowflake) Speaker Bio:
With over two decades of experience, Senthil has made significant contributions to the development and optimization of advanced storage technologies. His expertise spans snapshots, replication, name-space management, space accounting, garbage collection, cache, and performance optimization across NFS and CIFS protocols. He is also an inventor with multiple patents related to storage efficiency and optimization.
Currently, he is an engineer at Snowflake, where he led the development of the snapshot-based backup and restore feature. He is also the lead engineer for designing and implementing the control plane for FDB fleet management and is a committer for FoundationDB. Senthil brings deep expertise in both the architectural and practical aspects of storage technologies and is passionate about optimizing storage performance for modern distributed systems. Talk Summary:
In this talk, Senthil will delve into how FoundationDB is managed at scale at Snowflake, with a focus on the control plane for fleet management. He will discuss key aspects such as provisioning, scaling, autoscaling, and load balancing across large fleets of FDB clusters. The session will also cover the challenges of managing distributed systems at scale, upcoming enhancements in fleet management, and lessons learned from real-world deployments.
Talk Title: Spark connector for FDB Speaker: Akanksha Nagpal (Senior Engineer, Adobe) Speaker Bio: Akanksha is a Software Engineer with extensive experience in designing and building large-scale distributed systems within the Adobe Experience Platform (AEP). She has led the development of systems for efficient data ingestion into Adobe’s Identity Graph, including building highly scalable data pipelines with Apache Spark and Scala. Akanksha has also played key role in exploring FoundationDB’s architecture and encryption capabilities, focusing on its potential for Customer Managed Keys (CMK) within Adobe’s infrastructure. Passionate about innovation in data processing, she is dedicated to contributing to the broader engineering community. Talk Summary:
This talk will delve into how Adobe developed a custom Spark connector for FoundationDB to enhance data processing capabilities, addressing limitations of existing solutions. Akanksha will discuss the motivation behind this development, how the connector integrates Spark with FoundationDB for large-scale data operations, and its role in powering customer insights and reporting in AEP. She will also share key lessons learned, operational insights for managing the connector in production, and Adobe’s future roadmap, including planned enhancements and potential open-source contributions.
Join us for the first fully Virtual FoundationDB Meetup with guest speakers from Germany and India! The event will take place at 9 AM PST tomorrow (Nov 20th)*
RSVP here to secure your spot. Online meeting link will be visible once you RSVP on meetup.com. Folks from Europe and India regions we hope to see you there!
Agenda:
Ice Breakers - 10 mins
Talk: FoundationDB on Kubernetes - 45 mins
Talk: FoundationDB Infrastructure Excellence - 20 mins
Talk Title: FoundationDB on Kubernetes Speaker: Johannes Scheuermann (Site Reliability Engineer, Apple) Speaker Bio:
With over a decade of experience, Johannes started his journey in the Kubernetes eco-system in early 2015 by on-boarding projects and applications onto Kubernetes. Since 2020, Johannes works as an SRE for FoundationDB at Apple and is leading the development of the open source FoundationDB operator. Talk Summary:
In this talk, Johannes covers the design of the FoundationDB operator on Kubernetes, emphasizing the approaches to overcome limitations in Kubernetes and highlights implementation choices to address unique aspects of the FoundationDB Operator. He will outline the different options to run an highly available FoundationDB Cluster on Kubernetes. Additionally, Johannes will cover the challenges with running stateful services at scale on Kubernetes and different ways it can be managed. The talk concludes with an outlook on future design changes and planned features for the FoundationDB operator.
Talk Title: FoundationDB Infrastructure Excellence Speaker: Aakash Setia (Site Reliability Engineer, Adobe) and Deekshit Kumar GVD (Site Reliability Engineer, Adobe) Speaker Bio:
Aakash has over 10 years of experience in building reliable, scalable, and secure systems. Currently at Adobe for the past 3+ years, he has focused on infrastructure setup for multi-cloud environment and streamlining FoundationDB provisioning to enhance efficiency and usability.
Deekshit is experienced in multi-cloud providers like AWS, Azure and GCP, specializing in provisioning distributed databases such as Elasticsearch, Couchbase, Redis, Cassandra, Druid. For the past 2+ years at Adobe, he has focused on ensuring the operational excellence of FoundationDB in Azure and AWS environments Talk Summary:
This talk will delve into some high-level concepts for provisioning and configuration of FoundationDB database. They will explore how to achieve operational readiness for FDB, focusing on essential aspects like key metrics for monitoring, logging and debugging in production environments. Additionally, they will focus on key learnings such as perpetual storage wiggle and usage of open source utilities, showcasing how they optimize and streamline production FDB deployments.
