Databricks Training in Cork, Ireland

The Databricks Training provides hands-on knowledge of building scalable data engineering, analytics, and machine learning solutions using the Databricks Lakehouse Platform. Participants will learn how to use Apache Spark on Databricks, manage data with Delta Lake, build pipelines, perform analytics, and collaborate effectively using notebooks.

Introduction to Databricks & Lakehouse

• What is Databricks and the Lakehouse concept

• Lakehouse vs data lake vs data warehouse

• Databricks architecture overview

Databricks Workspace & Environment

• Databricks workspace overview

• Clusters and compute management

• Notebooks and collaboration

• Workspace security basics

Apache Spark Fundamentals on Databricks

• Spark architecture (Driver, Executors)

• RDDs vs DataFrames vs Datasets

• Spark execution model

• Lazy evaluation and transformations

Data Ingestion & Connectivity

• Ingesting data from files and cloud storage

• Reading structured and semi-structured data

• Connecting to databases and external sources

• Incremental and batch ingestion strategies

Data Transformation with Spark

• DataFrame and SQL APIs

• Data cleansing and enrichment

• Joins, aggregations, and window functions

• Handling large-scale transformations

Delta Lake Fundamentals

• What is Delta Lake and why it matters

• ACID transactions in data lakes

• Schema enforcement and evolution

• Time travel and versioning

• Handling deletes, updates, and merges

Data Pipelines & ETL/ELT

• Building end-to-end data pipelines

• Medallion architecture (Bronze, Silver, Gold)

• Job scheduling and orchestration

• Error handling and monitorin

Performance Optimization

• Partitioning and file optimization

• Caching and persistence

• Query optimization techniques

• Cluster sizing and tuning

Analytics & Reporting

• Using Databricks SQL

• Creating views and dashboards

• Ad-hoc analytics

• Integrating with BI tools

Security & Governance

• Authentication and access control

• Data permissions and table access

• Secrets management

• Data governance best practices

Introduction to Machine Learning on Databricks

• Overview of ML workflows

• Feature engineering basics

• Model training and evaluation

• Introduction to MLflow

Hands-on Exercises

Summary and Conclusion