Better Machine Learning through Active Learning
Try this notebook to reproduce the steps outlined below Machine learning models can seem like magical savants. They can distinguish hot dogs from not-hot-dogs, but Read More
Category: Apache Spark
Introducing Apache Spark on Docker on top of Apache YARN with CDP DataCenter release
Introduction Motivation Bringing your own libraries to run a Spark job on a shared YARN cluster can be a huge pain. In the past, you Read More
SQL Analytics at Scale: Selecting the Right SQL Engine for the Right Job
We are all hungry for data. Not just more data… also new types of data so that we can best understand our products, customers, and Read More
Productionizing Distributed XGBoost to Train Deep Tree Models with Large Data Sets at Uber
Michelangelo, Uber’s machine learning (ML) platform, powers machine learning model training across various use cases at Uber, such as forecasting rider demand, fraud detection, food Read More
Processing Geospatial Data at Scale With Databricks
The evolution and convergence of technology has fueled a vibrant marketplace for timely and accurate geospatial data. Every day billions of handheld and IoT devices Read More
Streamlining Variant Normalization on Large Genomic Datasets with Glow
Cross posted from the Glow blog. Many research and drug development projects in the genomics world involve large genomic variant data sets, the volume of Read More
Migration from Hadoop to modern cloud platforms: The case for Hadoop alternatives
Companies rely on their big data and analytics platforms to support innovation and digital transformation strategies. However, many Hadoop users struggle with complexity, unscalable infrastructure, Read More
Celebrating Growth at Databricks and 1,000 Employees!
This November, Databricks hired our 1,000th full-time employee! Founded in Berkeley in 2013, our six co-founders created Databricks to help data teams solve the world’s Read More
Evolving Michelangelo Model Representation for Flexibility at Scale
Michelangelo, Uber’s machine learning (ML) platform, supports the training and serving of thousands of models in production across the company. Designed to cover the end-to-end Read More
How Informatica Data Engineering Goes Hadoop-less with Databricks
Back in May, we announced our partnership with Informatica to build out a rich set of integrations between our two platforms. It’s been exciting work Read More