Category: Ecosystem

Query Delta Lake Tables from Presto and Athena, Improved Operations Concurrency, and Merge performance

Posted on January 29, 2020 by Denny Lee and Tathagata Das

We are excited to announce the release of Delta Lake 0.5.0, which introduces Presto/Athena support and improved concurrency. The key

On-Demand Webinar: Geospatial Analytics and AI in the Public Sector

Posted on January 28, 2020 by Michael Johns and Derek Yeager

We recently hosted a live webinar — Geospatial Analytics and AI in Public Sector — during which we covered top

Solving the World’s Toughest Problems with the Growing Open Source Ecosystem and Databricks

Posted on January 23, 2020 by Reynold Xin

We started Databricks in 2013 in a tiny little office in Berkeley with the belief that data has the potential

Processing Geospatial Data at Scale With Databricks

Posted on December 5, 2019 by Nima Razavi and Michael Johns

The evolution and convergence of technology has fueled a vibrant marketplace for timely and accurate geospatial data. Every day billions

Streamlining Variant Normalization on Large Genomic Datasets with Glow

Posted on December 5, 2019 by Kiavash Kianfar

Cross posted from the Glow blog. Many research and drug development projects in the genomics world involve large genomic variant

New Databricks Integration for Jupyter Bridges Local and Remote Workflows

Posted on December 3, 2019 by Bernhard Walter

Introduction For many years now, data scientists have developed specific workflows on premises using local filesystem hierarchies, source code revision

How India is coming of age – A Zomato Story

Posted on October 22, 2019 by Deepinder Goyal

In more than 400 cities across India where Zomato operates, when you ask someone who has ordered food to name

Delta Lake Now Hosted by the Linux Foundation to Become the Open Standard for Data Lakes

Posted on October 16, 2019 by Michael Armbrust and Reynold Xin

At today’s Spark + AI Summit Europe in Amsterdam, we announced that Delta Lake is becoming a Linux Foundation project.

Simple, Reliable Upserts and Deletes on Delta Lake Tables using Python APIs

Posted on October 3, 2019 by Denny Lee and Tathagata Das

Try this Jupyter notebook We are excited to announce the release of Delta Lake 0.4.0 which introduces Python APIs for

Parallelizing SAIGE Across Hundreds of Cores

Posted on October 2, 2019 by Karen Feng

As population genetics datasets grow exponentially, it is becoming impractical to work with genetic data without leveraging Apache Spark™. There