My name is Aravind. I would like to announce a monthly FoundationDB meetup that I’m helping organize in Adobe’s HQ in San Jose, CA.
Our main objective is to build a supportive environment where people who work with FoundationDB can connect, share their ideas, and learn from one another’s experiences. Additionally, we provide opportunities for networking and informal knowledge exchange, allowing attendees to forge meaningful connections and expand their professional networks.
If you are interested in attending, please RVSP on the meetup.com page. If you would like to present in the meetup, please send me a message via the meetup page.
The first event is scheduled for 2024-05-16T01:00:00Z.
How Tigris Leverages FoundationDB for Global Metadata Storage Himank Chaudhary [Co-founder & CTO, Tigris Data]
Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk we will share the technical details around why we chose FoundationDB and how we are using it at scale to power our object storage service.
Talk 1 : Title: How Tigris Leverages FoundationDB for Global Metadata Storage Speaker: Himank Chaudhary (Co-founder/CTO, Tigris) Speaker Bio:
Himank is the Co-Founder and CTO of Tigris Data where he is building a globally distributed object storage service. Prior to co-founding Tigris, Himank was the tech lead of Docstore, Uber’s distributed database that powered petabytes of data and facilitated millions of queries per second in a geo-distributed hybrid cloud deployment. Himank Chaudhary - Tigris Data Inc | LinkedIn Talk Summary:
Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk, Himank will share the technical details around why they chose FoundationDB and how we are using it at scale to power our object storage service.
Talk 2: Title:Encryption in FoundationDB Speaker: Ata E Husain Bohra (Snowflake) Speaker Bio:
Ata is a core member of Snowflake’s FoundationDB team. Snowflake has been working with FoundationDB for more than a decade. Ata is an industry veteran with over 10 years of experience designing/implementing large-scale distributed systems. Prior to Snowflake, he played an instrumental role in building and productionizing Uber’s Distributed Ledger platform. He is a startup enthusiast and has worked at multiple successful startups as a core member of the distributed filesystem team. Ata E Husain Bohra - Snowflake | LinkedIn Talk Summary:
Snowflake enhanced FoundationDB by designing/implementing an enterprise-grade native encryption data-at-rest solution. The talk deep-dives into encryption at-rest architecture, supported configuration modes, and ability to integrate with existing enterprise’s encryption infrastructure.
I’m delighted to share that our monthly FoundationDB meetup initiative is off to a fantastic start! Our inaugural meetup saw over 60 attendees, including key members of the FoundationDB team from Apple and Snowflake such as Jingyu Zhou, Joshua McManus, Ata E Husain Bohra and Stephen Pimentel. Thanks to everyone who was able to make it to the event. It was a pleasure connecting with you all! If anyone is interested in presenting at the meetup (remote or in-person), please reach out to me.
The next meetup event is scheduled for Jun 20th at Adobe HQ in San Jose. Please RSVP here.
Snowflake enhanced FoundationDB by designing/implementing an enterprise-grade native encryption data-at-rest solution. In this talk, Ata E Husain Bohra (Technical lead manager, Snowflake) will deep dive on the encryption at-rest architecture, supported configuration modes, and ability to integrate with existing enterprise’s encryption infrastructure.
Tigris is a globally distributed S3-compatible object storage service. Tigris uses FoundationDB as the underlying storage engine to store the metadata. In this talk, Himank Chaudhary (Co-founder & CTO, Tigris data) will share the technical details around why they chose FoundationDB and how we are using it at scale to power our object storage service.
Talk Summary:
Adobe Experience Platform’s core capability is delivering personalized customer experiences at scale. Identity Graphs are an integral concept to making this a reality. Adobe’s Identity Graph system manages over 50 billion identities, enables real-time ingestion of over 400,000 messages per second, and supports millions of API lookups per second.
In this talk, Aravind will provide insights into this large-scale distributed system and deep dive into how FoundationDB is leveraged as the backing store for the Identity Graph system. He will also dive into Adobe’s FoundationDB journey, including benchmarking FoundationDB in Azure and AWS clouds, Kubernetes vs VM deployment, fault-tolerance testing, and HA strategies.
Join us for this month’s FDB meetup on the 24th @ Adobe HQ in downtown San Jose !!
This meetup is a hybrid event (in-person + remote). RSVP here to secure your spot. Online meeting link will be made available closer to the event. I hope to see you all there!
Agenda:
Social Hour - 30 mins
Talk: Designing a control plane for FDB fleet management - 45 mins
Talk: Spark connector for FDB - 15 mins
Talk Title: Designing a control plane for FDB fleet management Speaker: Senthil Kumar Ramamoorthy (Senior Engineer, Snowflake) Speaker Bio:
With over two decades of experience, Senthil has made significant contributions to the development and optimization of advanced storage technologies. His expertise spans snapshots, replication, name-space management, space accounting, garbage collection, cache, and performance optimization across NFS and CIFS protocols. He is also an inventor with multiple patents related to storage efficiency and optimization.
Currently, he is an engineer at Snowflake, where he led the development of the snapshot-based backup and restore feature. He is also the lead engineer for designing and implementing the control plane for FDB fleet management and is a committer for FoundationDB. Senthil brings deep expertise in both the architectural and practical aspects of storage technologies and is passionate about optimizing storage performance for modern distributed systems. Talk Summary:
In this talk, Senthil will delve into how FoundationDB is managed at scale at Snowflake, with a focus on the control plane for fleet management. He will discuss key aspects such as provisioning, scaling, autoscaling, and load balancing across large fleets of FDB clusters. The session will also cover the challenges of managing distributed systems at scale, upcoming enhancements in fleet management, and lessons learned from real-world deployments.
Talk Title: Spark connector for FDB Speaker: Akanksha Nagpal (Senior Engineer, Adobe) Speaker Bio: Akanksha is a Software Engineer with extensive experience in designing and building large-scale distributed systems within the Adobe Experience Platform (AEP). She has led the development of systems for efficient data ingestion into Adobe’s Identity Graph, including building highly scalable data pipelines with Apache Spark and Scala. Akanksha has also played key role in exploring FoundationDB’s architecture and encryption capabilities, focusing on its potential for Customer Managed Keys (CMK) within Adobe’s infrastructure. Passionate about innovation in data processing, she is dedicated to contributing to the broader engineering community. Talk Summary:
This talk will delve into how Adobe developed a custom Spark connector for FoundationDB to enhance data processing capabilities, addressing limitations of existing solutions. Akanksha will discuss the motivation behind this development, how the connector integrates Spark with FoundationDB for large-scale data operations, and its role in powering customer insights and reporting in AEP. She will also share key lessons learned, operational insights for managing the connector in production, and Adobe’s future roadmap, including planned enhancements and potential open-source contributions.
