Apache Spark Cloud Training in Copenhagen, Denmark

This course introduces participants to the core concepts of Apache Spark and its deployment in cloud environments like AWS, Azure, and Google Cloud. Participants will learn how to process large-scale data using Spark’s components (Core, SQL, Streaming, MLlib) and how to set up, manage, and optimize Spark clusters in the cloud. The course combines theoretical foundations with hands-on labs to provide a strong understanding of Spark's cloud integration.

Course Agenda

Introduction to Apache Spark

• What is Apache Spark?

• Spark architecture and components

• RDDs, DataFrames, Datasets

• Cluster modes: local, standalone, YARN, Mesos, Kubernetes

• Cloud deployment overview: AWS EMR, Azure HDInsight, Dataproc, Databricks

Running Spark in the Cloud

• Setting up Spark on cloud environments (AWS EMR / Azure HDInsight / GCP Dataproc)

• Cloud storage integration (S3, Azure Blob, GCS)

• Writing and submitting Spark jobs on cloud clusters

• PySpark and Spark-shell hands-on labs

• Managing Spark clusters and scaling

Spark SQL and DataFrames

• Working with DataFrames and Datasets

• Using Spark SQL for querying structured data

• Integrating with cloud-native data sources (BigQuery, Redshift, Synapse)

• Caching, partitioning, and optimization techniques

• Hands-on exercises with large datasets

Spark Streaming and Machine Learning

• Introduction to Spark Streaming and Structured Streaming

• Processing streaming data in real time from Kafka, cloud storage, and REST APIs

• MLlib: machine learning pipelines in Spark

• Building and evaluating models in Spark

• Deploying ML models in the cloud

Performance Tuning and Monitoring

• Spark job tuning strategies

• Executors, memory, shuffling, and parallelism

• Monitoring Spark jobs using cloud-native tools (CloudWatch, Azure Monitor, etc.)

• Debugging and handling failures

Hands-on Exercises