Optimizing Databricks Workloads

What you will learn Get to grips with Spark fundamentals and the Databricks platform Process big data using the Spark DataFrame API with Delta Lake Analyze data using graph processing in Databricks Use MLflow to manage machine learning life cycles in Databricks Find out how to choose the right clust...

Full description

Bibliographic Details
Main Authors: Kala, Anirudh, Bhatnagar, Anshul (Author), Sarbahi, Sarthak (Author)
Format: eBook
Language:English
Published: Packt Publishing 2021
Edition:1st edition
Subjects:
Online Access:
Collection: O'Reilly - Collection details see MPG.ReNa
LEADER 03743nmm a2200397 u 4500
001 EB002008986
003 EBX01000000000000001171886
005 00000000000000.0
007 cr|||||||||||||||||||||
008 220121 ||| eng
050 4 |a QA76.9.B45 
100 1 |a Kala, Anirudh 
245 0 0 |a Optimizing Databricks Workloads  |h [electronic resource]  |c Kala, Anirudh 
250 |a 1st edition 
260 |b Packt Publishing  |c 2021 
300 |a 230 pages 
653 |a Big data / fast 
653 |a Spark (Electronic resource : Apache Software Foundation) / fast 
653 |a Big data / http://id.loc.gov/authorities/subjects/sh2012003227 
653 |a Spark (Electronic resource : Apache Software Foundation) / http://id.loc.gov/authorities/names/no2015027445 
653 |a Données volumineuses 
653 |a Microsoft Azure (Plateforme informatique) 
653 |a Microsoft Azure (Computing platform) / fast 
653 |a Microsoft Azure (Computing platform) / http://id.loc.gov/authorities/subjects/sh2016001752 
700 1 |a Bhatnagar, Anshul  |e author 
700 1 |a Sarbahi, Sarthak  |e author 
041 0 7 |a eng  |2 ISO 639-2 
989 |b OREILLY  |a O'Reilly 
500 |a Made available through: Safari, an O'Reilly Media Company 
776 |z 9781801819077 
776 |z 9781801811927 
856 4 0 |u https://learning.oreilly.com/library/view/~/9781801819077/?ar  |x Verlag  |3 Volltext 
082 0 |a 005.7 
520 |a What you will learn Get to grips with Spark fundamentals and the Databricks platform Process big data using the Spark DataFrame API with Delta Lake Analyze data using graph processing in Databricks Use MLflow to manage machine learning life cycles in Databricks Find out how to choose the right cluster configuration for your workloads Explore file compaction and clustering methods to tune Delta tables Discover advanced optimization techniques to speed up Spark jobs Who this book is for This book is for data engineers, data scientists, and cloud architects who have working knowledge of Spark/Databricks and some basic understanding of data engineering principles. Readers will need to have a working knowledge of Python, and some experience of SQL in PySpark and Spark SQL is beneficial 
520 |a The book covers how to select the optimal Spark cluster configuration for running big data processing and workloads in Databricks, some very useful optimization techniques for Spark DataFrames, best practices for optimizing Delta Lake, and techniques to optimize Spark jobs through Spark core. It contains an opportunity to learn about some of the real-world scenarios where optimizing workloads in Databricks has helped organizations increase performance and save costs across various domains. By the end of this book, you will be prepared with the necessary toolkit to speed up your Spark jobs and process your data more efficiently.  
520 |a Accelerate computations and make the most of your data effectively and efficiently on Databricks Key Features Understand Spark optimizations for big data workloads and maximizing performance Build efficient big data engineering pipelines with Databricks and Delta Lake Efficiently manage Spark clusters for big data processing Book Description Databricks is an industry-leading, cloud-based platform for data analytics, data science, and data engineering supporting thousands of organizations across the world in their data journey. It is a fast, easy, and collaborative Apache Spark-based big data analytics platform for data science and data engineering in the cloud. In Optimizing Databricks Workloads, you will get started with a brief introduction to Azure Databricks and quickly begin to understand the important optimization techniques.