Adaptive Query Execution(AQE) in Spark 3.0
Spark 3.0 released while ago. A bunch of new features are added in this new version of spark. One of the interesting features added is Read More
Author: Rameshbabu Gangineni
Parallelize Redshift Load
3 mins Read This article is continuation to Redshift loading strategies. You can refer previous article here Problem Statement: Consider we want to load 10 Read More
Dynamically Generating Airflow DAGS
Problem Statement: Consider we have 3 pipelines and each pipeline have 4 stages in our Data Engineering project and we want create Airflow DAGS for Read More
Apache Presto
Ever thought of joining a table in SQL Server with a table in MySql or a table in RDBMS with a table in NoSQL. That’s Read More