• To provide an overview of the class agenda and the expected learning objectives

Coach welcome apprentices to lesson and run through the session outline and learning objectives. Coach can run an ice breaker from here .

• Introduce Data Warehouses
• Understand the difference between a Data Warehouse and a Database

In this section you will be guiding apprentices through the concept of a Data Warehouse- what it is and the processes it follows. This is followed by an examindation of the differences between a data warehouse and a database which includes defining the processes OLAP ad OLTP. This section is a reminder from Module 2.

• Define Data Integration

In this section you will be defining what data integration means and leading a discussion around why it is important in data management. These concepts will be appearing in the KM1 exam as well. You will also be showing apprentices examples of data sources as well as discussing the benefits of data integration.

• Understand the difference between batch and real time integration

In this short section you will be showing the difference between batch and real time integration so apprentices can appreciate the processes and imagine benefits and challenges of each.

• Understand and explain the data integration life cycle

In this section you will be explaining the end to end process of how someone might carry out data integration, from selecting and profiling the data, understanding various requirements and designing the system to testing to make sure it works. This section incorporates vertical slides so you can lead discussions around each stage.

• Understand the need for data profiling
• Perform a profile on a data set

In this section you will be explaining to apprentices why data profiling is important and some of the information that is gained through the process. There is an example to show and a 10 minute activity where apprentices can create their own data profile from the products csv file.

• Explore various data integration techniques

In this section you will be exploring 5 different methods of using data integration and for each one also address benefits and drawbacks (vertical slides). You can use these opportunities to discuss what processes apprentices may already be aware of and how they are used.

• Consider methods to follow for keeping data integration accurate and safe

When using a data integration process it is important to consider rules and policies to ensure it works and is safe. Guide apprentices to consider what sort of rules around access, ensuring clean data and that the process has worked should be put into place. Link this back to GDPR and why data security is essential.

• Understand the ETL process
• Create a unified dataset following the ETL process

In this section you will be covering extract, transform and load- a process covered back in module 1. ETL is the process behind data integration where data from disparate sources are identified and transformed so they can be loaded into one unified view. There are slides for each part of the process with examples of how the process can be carried out in python using a variety of sources (CSV, JSON, SQL, etc). Dotted throughout the section is a Jupyter Notebook exercise (see data). Give 10 minutes for each part where apprentices pull the different tables from the Iowa Liquor Stores dataset into one table. If apprentices want to stretch themselves, there is an opportunity to extract data from BigQuery as well.

• Consider steps to make your ETL process more secure

This short section is to help apprentices think about how they can make the ETL processs more secure- run the process on a secure server and seek the necessary permissions.

• Consider how data integration can be automated

Another short section to help apprentices think how they can automate data integration and why they should do it.

• Understand the difference between using a licensed product for data integration and coding yourself

In this section you will lead a discussion around whether data integration should be performed manually (like in the ETL section) or by using a licensed software to make the process potentially smoother and automated. Each slide explores the advantages and disadvantages of each. This is followed by a quick view of some integration softwares available.

• Understand why MDM is important

In this section you will explain the process of master data management- how to store and document important data, and why it is important.

• Understand the three testing strategies

Earlier in the session we talk about testing strategies as part of the data integration life cycle. In this section you will walk through the three strategies (TAT, UAT and PST) and explain what each is testing for and how they are used in industry. With TAT there is an exercise where apprentices can write a script and check that it works using unittest in python.

• Recap the session

Recap the day, explain that in the next session we will be exploring data synchronization and gateway.

• Introduce the days learning

Explain that in this session we will be looking at data synchronization before thinking about the end of the course.

• Understand the definition of data synchronization

In this section you will be defining data synchronization. Make sure to stress the difference from data integration.

• Understand the benefits of data synchronization

In this short section you will run through some of the benefits of data synchronization, we will cover the challenges later.

• Describe the four methods of data synchronization

In this section you will be running through the four methods of data synchronization, giving definitions and examples. The two to focus on are file sync and version control. Examples of products for both are given. Lead discussions on how apprentices already use these. There is an exercise at the end for apprentices to discuss how these processes are used in their organisations

• Consider challenges to data synchronization and how to resolve them

In this section you will be exploring some of the challenges to do with data synchronization, exploring each in detail as well how to mitigate for them. Go through each issue and ask apprentices how they can be solved, before showing them.

• Recap the session

Explain that you have covered data synchronization and have now taught everything needed for the KM1 exam. This would be a good time to do a kahoot or mentimeter quiz to reinforce learning. Next you will be going through what apprentices can expect for the rest of the course.

• Understand what to expect from the final couple of months on the apprenticeship

Go through the timeline of when the KM1 exam will be and then gateway. STate that a revision session will be organised before the exam and gateway will be described in more detail in the next section. You will need to know the cohort specific gateway date, this will not be in the slide deck

• Understand the Gateway process

In this section you will inform apprentices about the gateway format and what apprentices need to do to prepare.

• Understand what to expect during the synoptic project

In this section you will help apprentices understand what to expect during the synoptic project as well as walking through the reporting format to help them structure their write up. There is an activity mid way through the section to help apprentices think about what sort of things they should consider during the project.

• Understand what to expect during the EPA interview

In this section you will be explaining the interview process and what they can expect. You will give them guidance on how to respond to questions and structure their answers. There is an activity where apprentices can have a go at asking each other questions ot prepare.

• Recap the day

Take this opportunity to answer any final questions from apprentices. Remind them to complete session attendance logs and update their OTJ.