This is GroupBy, the place where I share with you guys the resources I learn from people smarter than me in data engineer field.
Not subscribed yet? Here you go:
Get more from Vu Trinh in the Substack app
Available for iOS and Android
Robot trying to be a human, human trying to be a robot.
👋 Hi, my name is Vu Trinh, a data engineer.
I enjoy reading good stuff (related to data and engineering), and this newsletter is my effort on the journey to seek the "good stuff" across the entire Internet.
Hope this issue find you well.
🥷 It will steal 37 seconds from you
NEWSLETTER UPDATE.
FOR READER THAT ALREADY SUBSCRIBED:
THIS UPDATE WILL NOT AFFECT YOU READING EXPERIENCE AND NUMBER OF EMAIL YOU WILL RECEIVE WEEKLY.
You still receive only ONE EMAIL EVERY WEEK:
The GROUPBY WEEKLY issue.
(like the one you’re reading)
From beginning of 2024, I will launch a sub-newsletter with co-exist with this newsletter . This mean my newsletter will contain two sub-newsletter:
GroupBy.
Weekly compiled resource of data engineer (like the one you’re reading).
Every Tuesday
Dimensions.
My blog-style writing about what I've learned in data engineering field.
Every Saturday
Subscriber who subscribed:
Before 2024 January 06, will receive emails only from GroupBy.
After 2024 January 06, will receive emails both from GroupBy. and Dimensions.
Subscriber have the control over which newsletter they want to receive:
This project's central goal is creating a structured database design that includes a central table of facts and the required dimension tables to establish connections between different elements. This will enable meaningful comparisons and analysis.
I've heard a lot about Avro, Parquet, ORC, Arrow and Feather, but I also keep hearing about Iceberg and Delta Lake. As a "database person", I’ve been struggling to understand all of these different things, and how they relate to Data Lakes and Data Lakehouses (and what exactly are these?). So, I’ve decided to study them, and consolidate my knowledge in writing.
In this post, we'll explain how we built our RU (rolling update) framework to power a frictionless deployment experience on a large-scale Hadoop cluster, achieving a >99% success rate free from interruptions or downtime and reducing significant toil for our SRE and Dev teams.
But what about the long tail of issues that lurk in the shadows, sometimes remaining undetected until they cause chaos? For these, traditional strategies may not suffice.
This is where uVitals steps onto the stage, ready to seize the opportunity to detect sooner and detect more.
Super Tables (ST) are pre-computed, denormalized, and consistently consolidated attributes and insights of entities or events that are optimized for common and efficient analytic use cases.
...I wanted to provide some tips to help those either in leadership positions or who want to break into these positions plan out their data roadmap for 2024.
In this post we’ll discuss how we can learn from the field of cartography and Google Maps to extract the untapped potential of data lineage, and build this ideal interface to improve data literacy and observability.
In this blog, we will discuss the higher-level design and usage of of Data Access Level, how it fits in within the overall data platform ecosystem, and share some observations and lessons learned.
And so now, we return to the original question that took us down this long and winding path - should we even care about connecting enterprise data to natural language queries by LLMs?
If I was to summarize the goal of this article, it's that we're going to learn to light a campfire with a lighter (GPT2) and not a flamethrower (GPT3.5).
This blog post delves into the learnings and challenges on our journey towards implementing and scaling state-of-the-art deep learning approaches. We’ll shed light on how to use the newest machine-learning approaches in a controlled and reliable manner.
Airbnb had a significant presence at KDD 2023 with two papers accepted into the main conference proceedings and 11 talks and presentations. In this blog post, we’ll summarize our team’s contributions and share highlights from an exciting week of research talks, workshops, panel discussions, and more.
