David Max

LinkedIn is the world’s largest professional network with over 530 million members. Over 70% of LinkedIn’s members reside outside the U.S. This talk will describe some of the challenges relating to handling our members' personal information.

This talk will explore the technological issues involved with removing all personally identifiable information (PII) when a member closes their account in the context of LinkedIn's content ingestion system. This project's scope includes production… Show more

LinkedIn is the world’s largest professional network with over 530 million members. Over 70% of LinkedIn’s members reside outside the U.S. This talk will describe some of the challenges relating to handling our members' personal information.

This talk will explore the technological issues involved with removing all personally identifiable information (PII) when a member closes their account in the context of LinkedIn's content ingestion system. This project's scope includes production databases, backups, streaming messages, offline ETL data, and derived datasets.

Link to slides: https://resources.sei.cmu.edu/library/asset-view.cfm?assetID=519129 Show less

Espresso is LinkedIn's strategic distributed, fault-tolerant NoSQL database that powers many LinkedIn services. Espresso has a large production footprint at LinkedIn, with close to a hundred clusters in use, storing about 420 terabytes of source-of-truth (SoT) data and handling more than two million queries per second at peak load.

This talk discusses our strategy for migrating one of our internal services (Babylonia) from using Oracle to using Espresso. We will present an overview of… Show more

Espresso is LinkedIn's strategic distributed, fault-tolerant NoSQL database that powers many LinkedIn services. Espresso has a large production footprint at LinkedIn, with close to a hundred clusters in use, storing about 420 terabytes of source-of-truth (SoT) data and handling more than two million queries per second at peak load.

This talk discusses our strategy for migrating one of our internal services (Babylonia) from using Oracle to using Espresso. We will present an overview of the Espresso platform and its quality attributes that motivated the migration, as well as the particulars of how we accomplished the migration. Our core requirement was to keep Babylonia running uninterrupted throughout the migration process. These same concerns are common to many database migrations, not only at LinkedIn. The talk covers the steps we took to keep the service running through the transition without affecting our clients.

Link to slides: https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=519245 Show less

In this tale, a software team attempts to build a new system to replace their old system that was failing because of its inability to scale. The system they end up building meets all their criteria for scaling, but they discover that it has broken other criteria in ways that they did not anticipate. A lesson in the importance of identifying Architecturally Significant Requirements (ASRs).

Link to slides: https://lnkd.in/gZpzRwp

David Max is a Senior Software Engineer working on the Content Ingestion team. His team is responsible for scraping and storing metadata about all the external content shared by members on LinkedIn. Their main services are Babylonia and Jhubbub, which provide information like titles and thumbnail images to accompany shared URLs.

This post discusses our strategy for migrating one of our internal services (Babylonia) from using Oracle to using Espresso. Our core requirement was to keep Babylonia running uninterrupted throughout the migration process. This post focuses on the steps we took to keep the service running through the transition without affecting our clients. These same concerns are common to many database migrations, not only at LinkedIn.