DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Beyond SQL: Solving Data Warehouse Performance Bottlenecks with Smart Algorithms, Not Just Bigger Clusters
esproc_spl profile

Beyond SQL: Solving Data Warehouse Performance Bottlenecks with Smart Algorithms, Not Just Bigger Clusters

#algorithms #database #dataengineering #performance
5
Comments
13 min read
From Pandas to Upstream Control: The Evolution PyData Needs Next
aronchick profile

From Pandas to Upstream Control: The Evolution PyData Needs Next

#dataengineering #python #distributedsystems #machinelearning
Comments
6 min read
Statistics Day 2: Correlation Isn’t Causation — Here’s Why It Matters!
brains_behind_bots profile

Statistics Day 2: Correlation Isn’t Causation — Here’s Why It Matters!

#statistics #datascience #machinelearning #dataengineering
5
Comments
4 min read
Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Dec 9th - Dec15th, 2025)
alexmercedcoder profile

Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Dec 9th - Dec15th, 2025)

#news #dataengineering #opensource
1
Comments
7 min read
Kafka consumer lag—Measure and reduce
aime-bangirahe profile

Kafka consumer lag—Measure and reduce

#devops #performance #dataengineering #monitoring
Comments
5 min read
Understanding Kafka Consumer Lag: Causes, Risks, and How to Fix It
eric_kahindi_cfbfda3bd0f7 profile

Understanding Kafka Consumer Lag: Causes, Risks, and How to Fix It

#systemdesign #performance #dataengineering #monitoring
Comments
3 min read
Building a dbt-UI I Wish Existed
remis_haroon_230334c87790 profile

Building a dbt-UI I Wish Existed

#dbt #data #dataengineering #dataanalytics
1
Comments
3 min read
Building a Real-Time Crypto Data Pipeline with Debezium CDC
aineahsimiyu profile

Building a Real-Time Crypto Data Pipeline with Debezium CDC

#python #kafka #cdc #dataengineering
Comments
5 min read
Undestanding Kafka Lag, Why It Happens and How To Fix It.
mutindakioko profile

Undestanding Kafka Lag, Why It Happens and How To Fix It.

#kafka #programming #datascience #dataengineering
2
Comments
4 min read
The Myth of Distributed Computing as a Silver Bullet for Big Data
esproc_spl profile

The Myth of Distributed Computing as a Silver Bullet for Big Data

#discuss #dataengineering #performance #architecture
5
Comments
10 min read
The State of Apache Iceberg, Polaris, and Arrow: November 5-11
alexmercedcoder profile

The State of Apache Iceberg, Polaris, and Arrow: November 5-11

#opensource #database #dataengineering #data
Comments
5 min read
Understanding Kafka Lag: Why It Happens and How to Fix It
oliver_samuel_028c6f65ad6 profile

Understanding Kafka Lag: Why It Happens and How to Fix It

#dataengineering #monitoring #performance
Comments
4 min read
Right Approach to JSON Log Analysis: A Hands-on Guide to Efficient Practices with Alibaba Cloud SLS
observabilityguy profile

Right Approach to JSON Log Analysis: A Hands-on Guide to Efficient Practices with Alibaba Cloud SLS

#analytics #cloud #tutorial #dataengineering
Comments
7 min read
Understanding reasons behind Kafka lag and how to minimize it.
robbin_murithi_f75005db58 profile

Understanding reasons behind Kafka lag and how to minimize it.

#dataengineering #monitoring #performance
Comments
3 min read
REST API Calls for Data Engineers: A Practical Guide with Examples
chaets profile

REST API Calls for Data Engineers: A Practical Guide with Examples

#discuss #rest #dataengineering #json
1
Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.