data engineering

0. Introduction (SFC) are fascinating mathematical constructs with many practical applications in data science and data engineering. While they might sound abstract, they’re often hiding in plain sight—behind terms like Z-ordering or Liquid Clustering...

(or 2010s to be more precise) big-data boom brought the emergence of specialization in data roles. What was solely described as “Business Intelligence Engineer” was further broken down into Business Intelligence Engineers/Analysts, Data...

The aim of this text to offer the reply to the query: “Which one is ‘higher’ — Import or Direct Lake?” since it’s unattainable to reply, as there is no such thing...

tools like dbt make constructing SQL data pipelines easy and systematic. But even with the added structure and clearly defined data models, pipelines can still develop into complex, which makes debugging issues and...

If me for some time, you almost certainly know I began my profession as a QA engineer before transitioning into the world of data analytics. I didn’t go to high school for it,...

I talk over with organisations which have not yet properly began with Data Science (DS) and Machine Learning (ML), they often tell me that they should run an information integration project first,...

Lately, Parquet has grow to be a normal format for data storage in Big Data ecosystems. Its column-oriented format offers several benefits: Faster query execution when only a subset of columns is being processed Quick calculation...

DBeaver is probably the most powerful open-source SQL IDE, but there are several features people don’t learn about. On this post, I'll share with you many features to hurry up your workflow, with zero...