Data Engineering (2/1 Credits)

Learning Outcomes:

On successful completion of this course, student will be able to: LO1 – define the role of data engineering within the modern data ecosystem and its relationship to analytics and data science; LO2 – describe the fundamental concepts of data architecture, data pipelines, and the data lifecycle from ingestion to consumption; LO3 – differentiate between various data integration and transformation methods, including ETL and ELT approaches; LO4 – apply conceptual models to design simple data pipeline architectures that ensure data quality and reliability; LO5 – evaluate the suitability of different data tools and frameworks for specific business or analytical use cases.

Topics:

1. Introduction to Data Engineering;
2. Overview of Data Warehousing, Dimensional Modeling, and Pentaho Data Integration (Extract);
3. Pentaho Data Integration (Load and Transform);
4. The Data Engineering Lifecycle;
5. Designing Good Data Architecture;
6. Deployment: Data Visualization using Power BI ;
7. Assignment ;
8. Choosing Technologies Across the Data Lifecycle + Pentaho Intro ;
9. Data Generation in Source Systems;
10. Apache Airflow: Job Orchestration ;
11. Review Material;
12. Data Storage Concepts ;
13. Data Ingestion;
14. Queries, Modeling, and Transformation;
15. Serving Data for Analytics, ML, and Reverse ETL;
16. Security and Privacy in Data Engineering;
17. The Future of Data Engineering;
18. Mini Project Workshop;
19. Integration.