The PAMCET Learning Team is the institutional curriculum and content group behind course publishing on this platform. PAMCET (Professional and Academic Management Centre for Education and Training) is operated by Digitpen Hub Ltd, a company registered in Lagos, Nigeria. Courses credited to the PAMCET Learning Team are structured, organized, and published centrally by the platform rather than authored by a single named individual instructor — each course's syllabus, learning outcomes, and content are reviewed before publication under this institutional account. For questions about a specific course or to report a content issue, please use the platform's Help & Support or Contact Us pages.
SWE 310: Data Engineering & Scalable Data Pipelines
Architectural principles of large-scale data engineering. Batch vs stream processing, Extract-Transform-Load (ETL) and ELT pipelines, distributed compute frameworks (Apache Spark), distributed messaging, data lakehouse architectures (Apache Parquet, ...
Course Overview
Architectural principles of large-scale data engineering. Batch vs stream processing, Extract-Transform-Load (ETL) and ELT pipelines, distributed compute frameworks (Apache Spark), distributed messaging, data lakehouse architectures (Apache Parquet, ... Curriculum Structure &...
What You'll Learn
- Architect resilient, fault-tolerant batch and streaming data pipelines processing high-volume event streams.
- Write distributed dataset transformations using Apache Spark DataFrames and PySpark.
- Orchestrate multi-stage data dependencies using Apache Airflow Directed Acyclic Graphs (DAGs).
- Manage evolving data schemas using Apache Avro and Schema Registry without breaking consumers.
- Implement automated data quality verification gates using Great Expectations.
Requirements
- SWE 205
- SWE 210
Degree Programme Curriculum Context
This course is a curriculum unit of the Bachelor of Engineering (B.Eng.) in Software Engineering degree framework in Nigeria.
Earn Your Certificate
Showcase your skills with a certificate! Complete the course, submit your project, and earn your certificate. Here's a sample of what you'll receive to show off your accomplishments.

Course Curriculum
Browse complete theoretical foundations, worked calculations, laboratory protocols, and board assessment questions under each topic.
Data Engineering Foundations: OLTP vs OLAP, Data Warehouses, Lakes & Lakehouses: Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 1: Data Engineering Foundations: OLTP vs OLAP, Data Warehouses, Lakes & Lakehouses — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Executive Overvie...
Data Engineering Foundations: OLTP vs OLAP, Data Warehouses, Lakes & Lakehouses: Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 1: Data Engineering Foundations: OLTP vs OLAP, Data Warehouses, Lakes & Lakehouses — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidelines 1. Laborat...
Chapter 1: Data Engineering Foundations: OLTP vs OLAP, Data Warehouses, Lakes & Lakehouses — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Data Engineering Foundations: OLTP vs OLAP, Data Warehouses, Lakes & Lakehouses.
Batch vs Stream Processing Paradigms: Lambda vs Kappa Architectures: Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 2: Batch vs Stream Processing Paradigms: Lambda vs Kappa Architectures — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Executive Overview & Theoreti...
Batch vs Stream Processing Paradigms: Lambda vs Kappa Architectures: Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 2: Batch vs Stream Processing Paradigms: Lambda vs Kappa Architectures — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidelines 1. Laboratory & Comput...
Chapter 2: Batch vs Stream Processing Paradigms: Lambda vs Kappa Architectures — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Batch vs Stream Processing Paradigms: Lambda vs Kappa Architectures.
Columnar Storage Formats: Parquet, ORC, Compression Schemes & Predicate Pushdown: Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 3: Columnar Storage Formats: Parquet, ORC, Compression Schemes & Predicate Pushdown — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Executive Overvi...
Columnar Storage Formats: Parquet, ORC, Compression Schemes & Predicate Pushdown: Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 3: Columnar Storage Formats: Parquet, ORC, Compression Schemes & Predicate Pushdown — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidelines 1. Labora...
Chapter 3: Columnar Storage Formats: Parquet, ORC, Compression Schemes & Predicate Pushdown — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Columnar Storage Formats: Parquet, ORC, Compression Schemes & Predicate Pushdown.
Distributed Computing with Apache Spark: RDDs, DataFrames, Catalyst Optimizer & Shuffling: Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 4: Distributed Computing with Apache Spark: RDDs, DataFrames, Catalyst Optimizer & Shuffling — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Executi...
Distributed Computing with Apache Spark: RDDs, DataFrames, Catalyst Optimizer & Shuffling: Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 4: Distributed Computing with Apache Spark: RDDs, DataFrames, Catalyst Optimizer & Shuffling — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidelines...
Chapter 4: Distributed Computing with Apache Spark: RDDs, DataFrames, Catalyst Optimizer & Shuffling — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Distributed Computing with Apache Spark: RDDs, DataFrames, Catalyst Optimizer & Shuffling.
Data Pipeline Orchestration: Workflow DAGs, Apache Airflow, Sensors & Operators: Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 5: Data Pipeline Orchestration: Workflow DAGs, Apache Airflow, Sensors & Operators — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Executive Overvie...
Data Pipeline Orchestration: Workflow DAGs, Apache Airflow, Sensors & Operators: Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 5: Data Pipeline Orchestration: Workflow DAGs, Apache Airflow, Sensors & Operators — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidelines 1. Laborat...
Chapter 5: Data Pipeline Orchestration: Workflow DAGs, Apache Airflow, Sensors & Operators — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Data Pipeline Orchestration: Workflow DAGs, Apache Airflow, Sensors & Operators.
Schema Management: Schema Evolution, Confluent Schema Registry & Apache Avro: Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 6: Schema Management: Schema Evolution, Confluent Schema Registry & Apache Avro — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Executive Overview &...
Schema Management: Schema Evolution, Confluent Schema Registry & Apache Avro: Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 6: Schema Management: Schema Evolution, Confluent Schema Registry & Apache Avro — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidelines 1. Laboratory...
Chapter 6: Schema Management: Schema Evolution, Confluent Schema Registry & Apache Avro — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Schema Management: Schema Evolution, Confluent Schema Registry & Apache Avro.
Stream Processing: Windowing (Tumbling, Sliding, Session), Watermarks & Exactly-Once Semantics: Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 7: Stream Processing: Windowing (Tumbling, Sliding, Session), Watermarks & Exactly-Once Semantics — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Ex...
Stream Processing: Windowing (Tumbling, Sliding, Session), Watermarks & Exactly-Once Semantics: Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 7: Stream Processing: Windowing (Tumbling, Sliding, Session), Watermarks & Exactly-Once Semantics — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidel...
Chapter 7: Stream Processing: Windowing (Tumbling, Sliding, Session), Watermarks & Exactly-Once Semantics — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Stream Processing: Windowing (Tumbling, Sliding, Session), Watermarks & Exactly-Once Semantics.
Data Quality, Governance & Data Observability (Great Expectations / dbt): Theoretical Foundations, System Architecture & Subtopic Deep-Dive
SWE 310 • 300L • Software Systems Infrastructure Chapter 8: Data Quality, Governance & Data Observability (Great Expectations / dbt) — Theoretical Foundations, System Architecture & Analytical Modeling Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Benchmark: IEEE/ACM CS2023-IM-ENG 1. Executive Overview &...
Data Quality, Governance & Data Observability (Great Expectations / dbt): Worked Engineering Problems, Production Code & Simulation Protocols
SWE 310 • 300L • Software Systems Infrastructure Chapter 8: Data Quality, Governance & Data Observability (Great Expectations / dbt) — Applied Engineering Computation, Production Systems & Verification Course: Data Engineering & Scalable Data Pipelines Curriculum: CCMAS-SWE-310 Accreditation: COREN-SWE-DAT-03 Standard: IEEE / ACM SE Practice Guidelines 1. Laboratory &am...
Chapter 8: Data Quality, Governance & Data Observability (Great Expectations / dbt) — Formative Engineering Quiz
Rigorous formative evaluation assessing theoretical knowledge, mathematical formulations, and engineering principles for Data Quality, Governance & Data Observability (Great Expectations / dbt).
Academic Ecosystem & Connected Pathways
Deepen your studies across connected learning pathways in the PAMCET academic network:
Instructor
PAMCET Learning Team
Institutional Course Curation · PAMCET
Qualifications:
PAMCET Learning Team — Institutional Course Curation, Digitpen Hub LtdMy Skills:
- Curriculum Curation
- Course Content Review
- Learning Outcome Design
- Course Publishing & Quality Assurance
- Platform Content Operations
Reviews
Course Includes
Download Syllabus & Study Guide
Receive the full lecture syllabus, reading list, and milestone schedule for SWE 310: Data Engineering & Scalable Data Pipelines.
By submitting, you agree to receive educational updates. You can unsubscribe anytime. Read our Privacy Policy.

