Azure Data Factory Training

BI – Flow

• What is OLTP & OLAP?
• How to Load data from OLTP to OLAP?
• What is ETL?
• Who will convert the textual OLAP data to Reports?
• To Whom BI teams share the Reports?
• The end-End flow of BI Environment flow
• Roles & Responsibilities as BI Developer

ETL — Flow

• What is ETL & where it is used?
• ETL V/S SQL
• What is the staging layer?
• What kind of data is stored in the staging layer?
• What is ODS?
• What kind of data is stored in the ODS layer?
• What is a data mart?
• Roles & Responsibilities as ETL Developer

Azure Data Factory Overview at High Level

• What is Azure?
• What is the difference between cloud services & on Premises?
• What is Azure Data Factory?
• What are all the Components required to Configure ADF?
• ADF Stands for?
• What is Linked Service?
• What are Data Sets in Azure Data Factory?
• What is Dataset?
• What is IR?
• Types of IR’s?
• What is Pipeline?
• What are Data flows?
• List of Transformations in Dataflow?

DATA WAREHOUSE CONCEPTS

• What is SCD?
• Types of SCD’s
• SCD Type1/ SCD Type2 & SCD Type3
• Attributes required maintaining Type2
• Use Case to implement SCD Type2
• What is Dimension?
• What is Fact?
• What is Star Schema?
• What is Snowflake Schema?
• What is Data Mart?
• Top to Bottom Approach
• Bottom to Top Approach
• Types of Dimensions
• Type of Facts
• ODS – Operational Data Source
• EDWH – Enterprise Data Ware House
• OLTP V/S OLAP

AZURE CLOUD ACCOUNT SERVICE CONFIGURATION

• Cloud Services V/S On-Premises
• Cloud Services providers in Market
• What is Azure Cloud Service?
• What is a Subscription?
• Types of Subscriptions?
• How to configure a free account?
• How to configure a PayAsyouGo account?
• How to Configure Visual Studio subscription?
• How to verify the cost of the subscription?
• What is Recourse Group?
• How to create Recourse?
• How to configure ADF Recourse?
• What is Storage Account Recourse?
• How to Create Containers in Storage Account?

CONFIGURING SOURCE STORAGE ACCOUNTS & DATABASE

• Configuring Storage Accounts and containers
• Uploading files into the containers
• Blob storage in Azure account
• Coping files from one blob to blob in ADF
• Configuring SQL database introduction to RDBMS
• Reading the data from SQL data from database
• SQL concepts in RDBMS.
• OLTP Db concepts like Create table & inserting the data & updating
• Clauses like Order by, where clause, Group By, having & Distinct
• Select statements using like joins & sub quires
• OLAP functions.

AZURE DATA FACTORY RESOURCE CONFIGURATION

• Steps to Configure ADF (AZURE DATA FACTORY)
• Architecture of ADF
• User Interface of ADF
• ETL flow using ADF
• Creating Linked Service (Source & Target)
• Creating Datasets for source & target
• Creating pipeline to flow the data from source to target
• Integration Run time in Azure data factory
• Azure IR
• Self-Hosted IR
• Azure SSIS
• Publishing the objects to ADF Repository
• Validating pipelines
• Debugging the pipeline
• Monitoring the pipeline
• Working with the activities
• Working on files & tables using activities in ADF
• Working with data flows using transformations
• Executing the pipelines using dataflow
• Transforming the data from source file to table using dataflow

ADF WITH DATA FLOW TRANSFORMATIONS

• Source Transformation
• Alter Row Transformation
• Sort Transformation
• Filter Transformation
• Conditional Split Transformation
• Aggregate Transformation
• Rank Transformation
• Pivot Transformation
• Un-Pivot Transformation
• Join Transformation
• Lookup Transformation
• Union Transformation
• Derived Column Transformation
• Exists Transformation
• New Branch in mapping data flow
• Select Transformation
• Surrogate Key Transformation
• Window Transformation
• Sink Transformation

SCD CONFIGURATION USING DATA FLOWS

• SCD type 1
• SCD type 2
• Incremental load & Delta load from SQL to Blog Storage using ADF
• Azure Key Vault
• Secure secrets
• keys certificates
• Triggers & Event Based Trigger in Azure Data Factory
• Tumbling window trigger in Azure Data Factory
• Schedule Trigger in Azure Data Factory
• Parameters to pass values dynamically
• Passing the parameters from Master Pipeline to Child Parameter
• Parameters V/S Variables

AZURE DATA FACTORY WITH ACTIVITIES

• Copy Data Activities
• Data Flow Activity
• Filter Activity
• For Each Loop Activity
• If Condition Activity
• Switch Activity
• Until Activity
• Get Metadata Activity
• Execute Pipeline Activity
• Lookup Activity
• Pipeline Parameters

REALTIME USE CASES USING AZURE DATA FACTORY

• Copy multiple files from blob to blob in Azure Data Factory
• Filter activity in azure data factory helps to filter the required file
• Dynamic Copy in azure data factory using parameter
• Get Source File Names from Folder Dynamically in ADF
• Get Count of Source files from folder in ADF
• Validate copied data between source and sink in ADF as Unit Testing
• Incrementally copy new and changed files based on Last Modified Date in ADF
• Copy data AWS to Azure
• Migrate AWS S3 Buckets to ADLS Gen2 using ADF

INTERVIEW SUPPORT CLASSES

• One Retail & Real-time Project
• FAQ’s technical & non-technical
• Mock interviews in classes
• Resume preparation class