STSCI 5065

Big Data Management and Analysis

Course Description

Course information provided by the 2026-2027 Catalog.

Concepts, challenges, and industry trends of big data, with a focus on the Hadoop system. Topics include: basics of the Apache Hadoop platform and Hadoop ecosystem; the Hadoop distributed file system (HDFS); MapReduce or its alternative, a parallel programming model for distributed processing of large data sets; common big data tools, such as Pig (a procedural data processing language for Hadoop parallel computation), Hive (a declarative SQL-like language to handle Hadoop jobs), HBase (the most popular NoSQL database), and YARN; case studies; and integration of Hadoop with statistical software packages, e.g., SAS and R.

Prerequisites background equivalent to: STSCI 4060; STSCI 5060; STSCI 5010; STSCI 4520 or STSCI 4030.

Enrollment Information Enrollment limited to: Applied Statistics Master of Professional Studies (MPS) students.

Last 4 Terms Offered 2026SP, 2025SP, 2024SP, 2023SP

Learning Outcomes

• Comprehend the basic concepts of big data, the Apache Hadoop platform and its ecosystem, and the Hadoop distributed file system (HDFS).
• Be able to use MapReduce, a parallel programming model for distributed processing of large data sets.
• Master Hive, a declarative SQL-like language that presents data in the form of tables to handle Hadoop jobs.
• Master Pig, a procedural data processing language for Hadoop parallel computation.
• Understand components of the Hadoop ecosystem, especially PySpark.
• Know how to integrate the Hadoop system with statistical analysis software/packages, e.g., SAS and/or R.
• Master the basic skills of the CentOS Linux OS.

View Enrollment Information