DATA MANAGEMENT AND ORGANIZATION (2 SCU)

Learning Outcomes:

On successful completion of this course, students will be able to: LO1 – Describe the basic principles of data management and big data processing; LO2 – Demonstrate HDFS commands and Spark SQL; LO3 – Perform data preparation and machine learning using PySpark; LO4 – Perform Extract, Transform, Load (ETL) with Spark.

Topics:

1. Working with Apache Spark;
2. PySpark for Supervised Machine Learning;
3. PySpark Variable selection;
4. Utility Functions in PySpark;
5. ETL (Extract, Transform, and Load);
6. SQL on Big Data Landscape;
7. Optimizing Spark Applications;
8. Model Evaluation using PySpark;
9. Working with PySpark;
10. Spark Streaming;
11. Data Management & Data Governance;
12. Spark SQL.