Python plays a crucial role in data engineering, data science and AI development due to its versatility, extensive libraries such as Pandas and PySpark, and its ability to handle large-scale data processing, making it an indispensable tool for extracting insights and building data pipelines. In this course, participants will gain a solid understanding of Python.

They will acquire the necessary skills and knowledge to utilize Python effectively, from basic syntax to implementing real-world solutions and visualizing the results. During the course participants will get hands-on experience with Pydantic, Pandas, Matplotlib, Seaborn, PySpark, ...

Getting Started with Python

Python is a high-level, interpreted, interactive and object-oriented scripting language. This chapter introduces the history of Python and how to install Python and run your first lines of Python Code. There are quite some editors available for writing Python code but this course focusses on using Visual Studio Code as a code editor for Python. We'll also cover modern Python tooling including uv, a fast Python package installer and project manager.

• Introduction to Python
• Installing Python
• Executing Python Code from the Command Shell
• Python and Visual Studio Code
• Working with packages in Python
• Working with Virtual Environments in Python
• Modern Python tooling with uv
• Interactive development in Jupyter notebooks
• LAB: Installing Python and executing code

Basic Language Constructs in Python

We explore programming in Python by discussing some basic syntax, variables and conditional statements.

• Introduction to Python Coding Syntax
• Declaring and Using Variables
• Data Types in Python
• Conditional Statements
• LAB: Writing basic Python code

Collections and Loops

Collections allow you to store and organize data efficiently, making it easier to handle. Loops help you repeat actions on these collections.

• Collections
• Immutable and Mutable Types
• Loops in Python
• List Comprehension
• LAB: Collections and Loops

Functions and Error Handling

We explore how to structure reusable code, handle unexpected situations and manage resources.

• Functions
• Exception Handling
• Context Manager
• LAB: Functions and Error Handling

Working with Classes and Objects

Python classes provide all the standard features of Object Oriented Programming. Classes can inherit from other base classes, have Constructors for the initialization of objects, and leverage modern Python features like dataclasses for simplified class creation with automatic method generation.

• Introduction to Object-Oriented Programming
• Defining and instantiating Classes in Python
• Working with Constructors
• Instance and Class Variables
• Inheritance in Python
• Abstract base classes
• Multiple inheritance
• Class and static methods advanced usage
• Working with Access Modifiers
• Python dataclasses for simplified class definitions
• LAB: Working with classes and objects

Using and Creating Modules

Modules in Python are reusable code libraries and Python ships with quite a large amount of build-in Modules. Learn how to create and import Modules.

• Introduction to Modules
• Importing Modules
• Creating Modules
• LAB: Using and creating Modules

The Python Standard Library and External Ecosystem

You do not need to reinvent the wheel when coding in Python. Its Standard Library offers a rich collection of built-in modules that simplify common tasks, while external libraries provide specialized tools for modern development.

• Introduction to the Standard Library
• Date and Time Handling
• Mathematical Tools and Randomness
• Managing Files and Data (os, pathlib, json)
• Internet Protocols and Network Communication
• Popular Python Frameworks: Overview of Requests, FastAPI, PyTorch, and BeautifulSoup
• LAB: Functionalities of Standard and Third-Party Libraries

Model Validation with Pydantic

Pydantic is a powerful Python library that uses Python type annotations to validate data and settings management. It provides runtime type checking and automatic data conversion, making it essential for building robust data pipelines and APIs. This chapter covers how to define data models, validate complex data structures, and handle validation errors effectively.

• Introduction to Pydantic and data validation
• Defining BaseModel classes and field types
• Working with built-in validators and custom validation
• Configuration and settings management
• Validation error handling and custom error messages
• LAB: Implementing data validation pipelines in a web API

Unit Testing in Python

Testing is a critical aspect of software development that ensures code reliability and maintainability. Python provides excellent testing frameworks, with pytest being the most popular choice for its simplicity and powerful features. This chapter covers writing effective unit tests, mocking dependencies, and implementing test-driven development practices for data engineering and app development.

• Introduction to unit testing concepts
• Getting started with pytest framework
• Writing test functions and organizing test files
• Mocking external dependencies and APIs
• Code coverage analysis and reporting
• LAB: Implementing unit tests for Python applications

Data Processing and Cleansing using Pandas

Pandas is a Python library which makes loading and transforming data a lot easier. As long as all your data fits in memory, Pandas is your friend.

• What is Pandas
• Introducing Pandas Data Structures
• Reading data with Pandas
• Indexing in a DataFrame
• Creating and deleting columns
• Filtering and Replacing data
• Sorting and Ranking data
• Grouping and aggregating data
• Regular Expressions
• LAB: Working with Pandas

Exploratory Data Analysis and Visualization

Data visualization is a critical skill for data scientists and engineers to communicate insights and identify patterns or anomalies in data. This chapter explores the Python visualization ecosystem, starting from basic plotting in Matplotlib and Pandas, moving to the sophisticated statistical aesthetics of Seaborn, and concluding with interactive, web-ready visualizations using Plotly.

• The need for Visualization
• Pandas Visualization: Quick plotting directly from DataFrames
• Matplotlib: The foundation of Python plotting
• Seaborn: Statistical data visualization
• Plotly: Creating interactive and dynamic charts for the web
• LAB: Data Visualization with Python

Exploring Data Lakes and Apache Spark

Data lakes allows storing large data volumes in their original format, but Pandas doesn’t scale well. Apache Spark enables distributed processing, and PySpark brings it to Python (available in Micorsoft Fabric, Azure Synapse Analytics and Databricks).

• Storing your Data in a Data Lake
• Working with Parquet files
• What is Apache Spark?
• From Pandas to PySpark
• Spark SQL
• LAB: Apache Spark in Databricks

This course combines deep technical expertise with hands-on, instructor-led training, helping you transform knowledge into practical, real-world skills that last.

This course is targeted at data engineers, data scientists and AI developers with no or little experience with Python. Familiarity with programming in general might come in handy.