Azure Data Factory and Data Lake – Building a Modern Cloud Data Architecture (AZRDF)
Databases, Data Analytics
This intensive two-day course guides you through migrating and modernizing on-premise ETL into Microsoft Azure. You will design and implement a scalable data platform using Azure Data Factory, Azure Data Lake Storage Gen2 and the Medallion Architecture for layered storage.
You will build a realistic pipeline framework to ingest data from relational databases, CSV files and REST APIs, implement a metadata-driven orchestration for incremental loads, and practice monitoring, security and production best practices with ADF, metadata-driven ETL and incremental loading.
Location, current course term
The course:
Hide detail
-
Basics of cloud data architecture and Azure Data Factory
-
Principles of a modern data platform in Azure
-
The role of an orchestrator and data storage in the data architecture
-
Overview of Azure Data Factory services
-
Description of Integration Runtime and connection methods to data sources
-
Working with Azure Data Lake Storage Gen2
-
Medallion Architecture (Bronze / Silver / Gold)
-
Creating your first ADF pipeline
-
Loading data from a relational database into the data lake
-
Ingesting file data (CSV) into ADLS
-
Metadata-driven ETL framework
-
Orchestrating data pipelines in Azure Data Factory
-
Loading data from multiple sources (SQL databases, files, REST API)
-
Designing a metadata table to control ETL processes
-
Implementing a metadata-driven pipeline (Lookup + ForEach)
-
Incremental data loading
-
Transformations between Bronze and Silver layers
-
Monitoring and managing pipelines in Azure Data Factory
-
Architecture summary and recommendations for production solutions
-
Assumed knowledge:
-
Basic SQL knowledge and general database concepts are expected.
-
Schedule:
-
2 days (9:00 AM - 5:00 PM )
-
Course price:
-
676.00 € ( 817.96 € incl. 21% VAT)
-
Language:
-