Beyond the database: Architecting scalable data platforms for the cloud.

Our meetup on Novembe 7, 2024, co-hosted with Snowflake at the Snowflake ❄️ Amsterdam office was an insightful one.

𝐤𝐞𝐲 𝐭𝐚𝐤𝐞𝐚𝐰𝐚𝐲𝐬:

Dimitar Nedev and İpek Özlem's talk highlighted Booking.com's journey to create a modern, cloud-based data platform using Snowflake’s capabilities, with a strong focus on user experience and developer productivity. Their approach, which prioritizes open table formats, unified metadata management, and a consistent user interface across tools, reflects a forward-thinking strategy for managing data at scale.

𝐖𝐡𝐲 𝐒𝐧𝐨𝐰𝐟𝐥𝐚𝐤𝐞?

✦ 𝐒𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐚𝐧𝐝 𝐄𝐥𝐚𝐬𝐭𝐢𝐜𝐢𝐭𝐲: On-demand compute that keeps costs down and management light.
✦ 𝐄𝐟𝐟𝐨𝐫𝐭𝐥𝐞𝐬𝐬 𝐌𝐚𝐧𝐚𝐠𝐞𝐦𝐞𝐧𝐭: Snowflake’s SaaS setup lets Booking.com run things with just a six-person team.
✦ 𝐒𝐭𝐨𝐫𝐚𝐠𝐞 𝐅𝐥𝐞𝐱𝐢𝐛𝐢𝐥𝐢𝐭𝐲: Options to store data where they want, like Iceberg tables in their own S3 buckets.
✦ 𝐏𝐨𝐰𝐞𝐫𝐟𝐮𝐥 𝐃𝐚𝐭𝐚 𝐌𝐚𝐧𝐚𝐠𝐞𝐦𝐞𝐧𝐭: Includes fine-grained access control, data lineage tracking, and stewardship tools to keep data in check.

In Peter Lem’s talk, he shared the successful migration of a transaction reconciliation platform from a MySQL-based data warehouse to Snowflake. Driven by the need for better performance, scalability, and cost-efficiency, Actuals adopted a gradual, tenant-by-tenant migration, maintaining existing functionality while leveraging Snowflake's unique features. The result was faster processing, reduced maintenance costs, and higher user satisfaction. This serves as a valuable case study for organizations considering a transition to a modern data warehouse.

𝐊𝐞𝐲 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬:

✦ Pick a data warehouse that truly fits your business needs, focusing on performance, scalability, and cost-efficiency.
✦ A well-thought-out migration strategy is key - keep it gradual, test thoroughly, and prioritize data integrity.
✦ Leveraging new tech and its unique features can be a game-changer, boosting user experience, optimizing processes, and unlocking fresh opportunities for innovation.

A huge shout-out to the organizing team from Snowflake and PyData Amsterdam:

🧡 ❄️ Dora Kondeva, Maria Nicoletti, Stephanie Andrei, Jason C. Teetz, Elena Drenska, Nikolett Rontó, Astrid Höij, Bas Vlaming, Shunyao Yu ❄️ 💙