#23. The Sandman stories

Topics: Architecture, data thoughts, Spanner, streaming

Is DataOps the Future Of the Modern Data Stack? — The Data Downtime Blog

Am I data engineer? Am I DataOps? What are key differences? Let’s get this straight.

level:medium topic:data-thoughts

A Survey on Transactional Stream Processing — Shuhao Zhang, Juan Soto, Volker Markl

The authors made a big research on different stream processing systems and their transactional guarantees. You will find a classification of transactional stream processing systems, and tradeoffs in the design of real implementations.

level:advanced topic:streaming type:whitepaper

Spanner on a modern columnar storage engine — Yuanyuan Zhao @ Google Cloud blog

Spanner processes over 2 billion QPS in peak and stores over 6 exabytes of data. Do you know how to implement migration from one storage engine to another without outages and performance regressions on such a scale? This article is about the strategy Google applied to implement such significant migration for Spanner.

level:medium topic:spanner

Data meshes and microservices - the similarities are uncanny — Ben Morris

Have you ever noticed that data teams sometimes have the same challenges as software engineering teams? So let’s learn from other’s mistakes.

level:medium topic:data-thoughts

Uber Freight Carrier Metrics with Near-Real-Time Analytics — Uber Engineering Blog

It’s time to design systems with Uber. New use case: near real time data on the Carrier App.

level:medium topic:architecture

Written on September 13, 2022