Introduction to DP-203: Data Engineering on Microsoft Azure Training
The DP-203 Data Engineering on Microsoft Azure certification training course from laraonlinetraining offers candidates proper training and relevant study material to prepare and successfully clear the DP-203 exam.
This learning path is designed to help you prepare for Microsoft’s DP-203 Data Engineering on Microsoft Azure exam. Even if you don’t plan to take the exam, these courses and hands-on labs will help you learn how to deploy and manage a variety of Azure data solutions.
Candidates who pass the DP-203 exam will earn the Microsoft Certified: Azure Data Engineer Associate certification.
The DP-203 exam tests your knowledge of four subject areas: designing and implementing data storage, designing and developing data processing, designing and implementing data security, and monitoring and optimizing data storage and data processing.
After completing this course, students will be able to:
- Design and implement data storage
- Design and develop data processing
- Design and implement data security
- Monitor and optimize data storage and data processing
Prerequisites
The prerequisites of DP-203 exam include:
- A candidate must have solid knowledge of data processing languages, such as SQL, Python, or Scala, and they need to understand parallel processing and data architecture patterns.
- Candidate should have subject matter expertise integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building analytics solutions.
Why Should I choose Lara Online Training?
- Laraonlinetraining provide very in-depth course material with real time scenarios for each topic with its Solutions for MS-100: Microsoft 365 Identity and Services Training.
- We also provide production case studies during the training.
- Schedule the sessions based upon your comfort by our Highly Qualified Trainers and Real-time Experts.
- Get the Class recordings, immediately after the session for further Reference.
- Avail flexibility with Normal Track, Fast Track, and Weekend Batches also for MS-100: Microsoft 365 Identity and Services Training.
- Cost Effective and Flexible Payment Schemes.
- 100% Placement assistance
- We provide Assessment and Mock Interviews
Module 1 Design and Implement Date Storage Design a data storage structure
1
Design an Azure Data Lake solution
2
Recommend file types for storage
3
Recommend file types for analytical queries
4
Design for efficient querying
5
Design for data pruning
6
Design a folder structure that represents the levels of data transformation
7
Design a distribution strategy
8
Design a data archiving solution
9
Design a partition strategy
10
Design a partition strategy for files
11
Design a partition strategy for analytical workloads
12
Design a partition strategy for efficiency/performance
13
Design a partition strategy for Azure Synapse Analytics
14
Identify when partitioning is needed in Azure Data Lake Storage Gen2
Design the serving layer
1
Design star schemas
2
Design slowly changing dimensions
3
Design a dimensional hierarchy
4
Design a solution for temporal data
5
Design for incremental loading
6
Design analytical stores
7
Design metastores in Azure Synapse Analytics and Azure Databricks
Implement physical data storage structures
1
Implement compression
2
Implement partitioning
3
Implement sharding
4
Implement different table geometries with Azure Synapse Analytics pools
5
Implement data redundancy
6
Implement distributions
7
Implement data archiving
Implement logical data structures
1
Build a temporal data solution
2
Build a slowly changing dimension
3
Build a logical folder structure
4
Build external tables
5
Implement file and folder structures for efficient querying and data pruning
Implement the serving layer
1
Deliver data in a relational star schema
2
Deliver data in Parquet files
3
Maintain metadata
4
Implement a dimensional hierarchy
Module 2 Design and Develop Data Processing Ingest and transform data
1
Transform data by using Apache Spark
2
Transform data by using Transact-SQL
3
Transform data by using Data Factory
4
Transform data by using Azure Synapse Pipelines
5
Transform data by using Stream Analytics
6
Cleanse data
7
Split data
8
Shred JSON
9
Encode and decode data
10
Configure error handling for the transformation
11
Normalize and denormalize values
12
Transform data by using Scala
13
Perform data exploratory analysis
Design and develop a batch processing solution
1
Develop batch processing solutions by using Data Factory, Data Lake, Spark, Azure Synapse Pipelines, PolyBase, and Azure Databricks
2
Create data pipelines
3
Design and implement incremental data loads
4
Design and develop slowly changing dimensions
5
Handle security and compliance requirements
6
Scale resources
7
Configure the batch size
8
Design and create tests for data pipelines
9
Integrate Jupyter/IPython notebooks into a data pipeline
10
Handle duplicate data
11
Handle missing data
12
Handle late-arriving data
13
Upsert data
14
Regress to a previous state
15
Design and configure exception handling
16
Configure batch retention
17
Design a batch processing solution
18
Debug Spark jobs by using the Spark UI
Design and develop a stream processing solution
1
Develop a stream processing solution by using Stream Analytics, Azure Databricks, and Azure Event Hubs
2
Process data by using Spark structured streaming
3
Monitor for performance and functional regressions
4
Design and create windowed aggregates
5
Handle schema drift
6
Process time series data
7
Process across partitions
8
Process within one partition
9
Configure checkpoints/watermarking during processing
10
Scale resources
11
Design and create tests for data pipelines
12
Optimize pipelines for analytical or transactional purposes
13
Handle interruptions
14
Design and configure exception handling
15
Upsert data
16
Replay archived stream data
17
Design a stream processing solution
Manage batches and pipelines
1
Trigger batches
2
Handle failed batch loads
3
Validate batch loads
4
Manage data pipelines in Data Factory/Synapse Pipelines
5
Schedule data pipelines in Data Factory/Synapse Pipelines
6
Implement version control for pipeline artifacts
7
Manage Spark jobs in a pipeline
Module 3 Design and Implement Data Security Design Security for Data Policies and Standards
1
Design data encryption for data at rest and in transit
2
Design a data auditing strategy
3
Design a data masking strategy
4
Design for data privacy
5
Design a data retention policy
6
Design to purge data based on business requirements
7
Design Azure role-based access control (Azure RBAC) and POSIX-like Access Control List (ACL) for Data Lake Storage Gen2
8
Design row-level and column-level security
Implement data security
1
Implement data masking
2
Encrypt data at rest and in motion
3
Implement row-level and column-level security
4
Implement Azure RBAC
5
Implement POSIX-like ACLs for Data Lake Storage Gen2
6
Implement a data retention policy
7
Implement a data auditing strategy
8
Manage identities, keys, and secrets across different data platform technologies
9
Implement secure endpoints (private and public)
10
Implement resource tokens in Azure Databricks
11
Load a DataFrame with sensitive information
12
Write encrypted data to tables or Parquet files
13
Manage sensitive information
Module 4 Monitor and Optimize Data Storage and Data Processing Monitor Data Storage and Data Processing
1
Implement logging used by Azure Monitor
2
Configure monitoring services
3
Measure performance of data movement
4
Monitor and update statistics about data across a system
5
Monitor data pipeline performance
6
Measure query performance
7
Monitor cluster performance
8
Understand custom logging options
9
Schedule and monitor pipeline tests
10
Interpret Azure Monitor metrics and logs
11
Interpret a Spark directed acyclic graph (DAG)
Optimize and troubleshoot data storage and data processing
1
Compact small files
2
Rewrite user-defined functions (UDFs)
3
Handle skew in data
4
Handle data spill
5
Tune shuffle partitions
6
Find shuffling in a pipeline
7
Optimize resource management
8
Tune queries by using indexers
9
Tune queries by using cache
10
Optimize pipelines for analytical or transactional purposes
11
Optimize pipeline for descriptive versus analytical workloads
12
Troubleshoot a failed spark job
13
Troubleshoot a failed pipeline run
Be the first to add a review.
Please, login to leave a review
Related Courses
30-Day Money-Back Guarantee
Includes
Full lifetime access
Access on mobile and TV
Thank you so much, Mr. Kishore, for the online IT training. That was a great experience with you. I have learned a lot from you. You are a very knowledgeable person and helpful in answering questions. I appreciate your replies to me promptly through text messages or email. I understand you overall.
Asif Jafri
2023-01-09
The explanation is excellent. Trainer explained with live examples and secenarios
Eswar vakada
2023-01-09
Excellent experience. Kishore is a valuable resource, Lara online training should be proud of him.
Nasir Javed
2023-01-09
that was a best practice and I have learned alot new things.
thank you kishore
Muscle Passion
2023-01-09
Thank you so much for give us such a great training I learn a lot and it adds value to my knowledge If got a job I should be able to do it without any trouble thanks.
Mohammad Zahid_Qureshi
2023-01-09
My experience was good in this class ! Mr. Kishore is very knowledgeable and he knows what he’s teaching! He answered all the questions during class and even after the class. He taught everything in detail and with different scenarios so we won’t have hard time when we get our job! 10/10 experience for me!
Muhammad Arsalan Zubair
2023-01-09
Kishore sir is very experienced and he was helpful through out the class
Satya Teja
2023-01-08
