Azure Data Factory Training
BI – Flow
- What is OLTP & OLAP?
- How to Load data from OLTP to OLAP?
- What is ETL?
- Who will convert the textual OLAP data to Reports?
- To Whom BI teams share the Reports?
- The end-End flow of BI Environment flow
- Roles & Responsibilities as BI Developer
ETL — Flow
- What is ETL & where it is used?
- ETL V/S SQL
- What is the staging layer?
- What kind of data is stored in the staging layer?
- What is ODS?
- What kind of data is stored in the ODS layer?
- What is a data mart?
- Roles & Responsibilities as ETL Developer
Azure Data Factory Overview at High Level
- What is Azure?
- What is the difference between cloud services & on Premises?
- What is Azure Data Factory?
- What are all the Components required to Configure ADF?
- ADF Stands for?
- What is Linked Service?
- What are Data Sets in Azure Data Factory?
- What is Dataset?
- What is IR?
- Types of IR’s?
- What is Pipeline?
- What are Data flows?
- List of Transformations in Dataflow?
DATA WAREHOUSE CONCEPTS
- What is SCD?
- Types of SCD’s
- SCD Type1/ SCD Type2 & SCD Type3
- Attributes required maintaining Type2
- Use Case to implement SCD Type2
- What is Dimension?
- What is Fact?
- What is Star Schema?
- What is Snowflake Schema?
- What is Data Mart?
- Top to Bottom Approach
- Bottom to Top Approach
- Types of Dimensions
- Type of Facts
- ODS – Operational Data Source
- EDWH – Enterprise Data Ware House
- OLTP V/S OLAP
AZURE CLOUD ACCOUNT SERVICE CONFIGURATION
- Cloud Services V/S On-Premises
- Cloud Services providers in Market
- What is Azure Cloud Service?
- What is a Subscription?
- Types of Subscriptions?
- How to configure a free account?
- How to configure a PayAsyouGo account?
- How to Configure Visual Studio subscription?
- How to verify the cost of the subscription?
- What is Recourse Group?
- How to create Recourse?
- How to configure ADF Recourse?
- What is Storage Account Recourse?
- How to Create Containers in Storage Account?
CONFIGURING SOURCE STORAGE ACCOUNTS & DATABASE
- Configuring Storage Accounts and containers
- Uploading files into the containers
- Blob storage in Azure account
- Coping files from one blob to blob in ADF
- Configuring SQL database introduction to RDBMS
- Reading the data from SQL data from database
- SQL concepts in RDBMS.
- OLTP Db concepts like Create table & inserting the data & updating
- Clauses like Order by, where clause, Group By, having & Distinct
- Select statements using like joins & sub quires
- OLAP functions.
AZURE DATA FACTORY RESOURCE CONFIGURATION
- Steps to Configure ADF (AZURE DATA FACTORY)
- Architecture of ADF
- User Interface of ADF
- ETL flow using ADF
- Creating Linked Service (Source & Target)
- Creating Datasets for source & target
- Creating pipeline to flow the data from source to target
- Integration Run time in Azure data factory
- Azure IR
- Self-Hosted IR
- Azure SSIS
- Publishing the objects to ADF Repository
- Validating pipelines
- Debugging the pipeline
- Monitoring the pipeline
- Working with the activities
- Working on files & tables using activities in ADF
- Working with data flows using transformations
- Executing the pipelines using dataflow
- Transforming the data from source file to table using dataflow
ADF WITH DATA FLOW TRANSFORMATIONS
- Source Transformation
- Alter Row Transformation
- Sort Transformation
- Filter Transformation
- Conditional Split Transformation
- Aggregate Transformation
- Rank Transformation
- Pivot Transformation
- Un-Pivot Transformation
- Join Transformation
- Lookup Transformation
- Union Transformation
- Derived Column Transformation
- Exists Transformation
- New Branch in mapping data flow
- Select Transformation
- Surrogate Key Transformation
- Window Transformation
- Sink Transformation
SCD CONFIGURATION USING DATA FLOWS
- SCD type 1
- SCD type 2
- Incremental load & Delta load from SQL to Blog Storage using ADF
- Azure Key Vault
- Secure secrets
- keys certificates
- Triggers & Event Based Trigger in Azure Data Factory
- Tumbling window trigger in Azure Data Factory
- Schedule Trigger in Azure Data Factory
- Parameters to pass values dynamically
- Passing the parameters from Master Pipeline to Child Parameter
- Parameters V/S Variables
AZURE DATA FACTORY WITH ACTIVITIES
- Copy Data Activities
- Data Flow Activity
- Filter Activity
- For Each Loop Activity
- If Condition Activity
- Switch Activity
- Until Activity
- Get Metadata Activity
- Execute Pipeline Activity
- Lookup Activity
- Pipeline Parameters
REALTIME USE CASES USING AZURE DATA FACTORY
- Copy multiple files from blob to blob in Azure Data Factory
- Filter activity in azure data factory helps to filter the required file
- Dynamic Copy in azure data factory using parameter
- Get Source File Names from Folder Dynamically in ADF
- Get Count of Source files from folder in ADF
- Validate copied data between source and sink in ADF as Unit Testing
- Incrementally copy new and changed files based on Last Modified Date in ADF
- Copy data AWS to Azure
- Migrate AWS S3 Buckets to ADLS Gen2 using ADF
INTERVIEW SUPPORT CLASSES
- One Retail & Real-time Project
- FAQ’s technical & non-technical
- Mock interviews in classes
- Resume preparation class