Cover Image
Read Now

The Definitive Guide to Azure Data Engineering Modern ELT, DevOps, and Analytics on the Azure Cloud Platform

Build efficient and scalable batch and real-time data ingestion pipelines, DevOps continuous integration and deployment pipelines, and advanced analytics solutions on the Azure Data Platform. This book teaches you to design and implement robust data engineering solutions using Data Factory, Databric...

Full description

Bibliographic Details
Main Author: L'Esteve, Ron C.
Format: eBook
Language:English
Published: Berkeley, CA Apress 2021, 2021
Edition:1st ed. 2021
Subjects:
Microsoft Software
Big Data
Microsoft
Database Management
Microsoft .net Framework
Online Access:
Collection: Springer eBooks 2005- - Collection details see MPG.ReNa
Table of Contents:
  • Introduction
  • Part I. Getting Started
  • 1. The Tools and Pre-Requisites
  • 2. Data Factory vs SSIS vs Databricks
  • 3. Design a Data Lake Storage Gen2 Account
  • Part II. Azure Data Factory for ELT
  • 4. Dynamically Load SQL Database to Data Lake Storage Gen 2
  • 5. Use COPY INTO to Load Synapse Analytics Dedicated SQL Pool
  • 6. Load Data Lake Storage Gen2 Files into Synapse Analytics Dedicated SQL Pool
  • 7. Create and Load Synapse Analytics Dedicated SQL Pool Tables Dynamically
  • 8. Build Custom Logs in SQL Database for Pipeline Activity Metrics
  • 9. Capture Pipeline Error Logs in SQL Database.-10. Dynamically Load Snowflake Data Warehouse.-11. Mapping Data Flows for Data Warehouse ETL
  • 12. Aggregate and Transform Big Data Using Mapping Data Flows
  • 13. Incrementally Upsert Data.-14. Loading Excel Sheets into Azure SQL Database Tables.-15. Delta Lake
  • Part III. Real-Time Analytics in Azure
  • 16. Stream Analytics AnomalyDetection
  • 17. Real-time IoT Analytics Using Apache Spark
  • 18. Azure Synapse Link for Cosmos DB
  • Part IV. DevOps for Continuous Integration and Deployment
  • 19. Deploy Data Factory Changes
  • 20. Deploy SQL Database
  • Part V. Advanced Analytics
  • 21. Graph Analytics Using Apache Spark’s GraphFrame API
  • 22. Synapse Analytics Workspaces
  • 23. Machine Learning in Databricks
  • Part VI. Data Governance
  • 24. Purview for Data Governance

Similar Items