PySpark – Big Data Analysis in Databricks (PYSPARK1)

Databases, Data Analytics

Do you work with Excel, Power Query, SQL or Pandas but need to process gigabytes to terabytes of data? PySpark is the Python interface to Apache Spark, a big data engine for scalable, distributed processing that can handle datasets larger than one machine's memory and speed up analysis.

The workshop runs entirely in Databricks Community Edition in your browser - no local setup required. You'll learn the DataFrame API and Spark SQL, work with notebooks, clusters and data uploads, and apply familiar SQL skills to scale analyses to large datasets.

THIS TRAINING COURSE WILL HELP YOU:

  • Process large datasets using PySpark DataFrame API
  • Run SQL queries with Spark SQL and combine APIs
  • Read and write CSV, Parquet and Delta formats
  • Perform joins, aggregations, pivots and data transforms
  • Use Databricks notebooks, clusters and export results

WHO SHOULD ATTEND?

  • Data analysts using Excel, Power Query or SQL
  • Python users familiar with Pandas who need scale
  • BI developers working with large datasets
  • Anyone preparing data for analytics at scale

COURSE LOCATION AND AVAILABLE DATES



Choose whether to attend in person in our classroom or join online. You can select your preferred format during registration. Learn more about hybrid training.
Brno + online (optional)
Praha + online (optional)
Private training

Unlock your employees' potential. We can adapt every course in our portfolio to your objectives and participants.
Need training at your premises or want to tailor the content and duration? We will prepare the right solution, in Czech or English.

Request customised training

Course content:

Hide details
  • Getting started with Databricks
    1. What PySpark is and when to use it
    2. Creating an account in Databricks Community Edition
    3. Navigating the environment – workspace, notebooks, cluster
    4. Uploading data to Databricks
  • DataFrame – basic operations
    1. Creating DataFrames
    2. Schema and data types
    3. Selecting columns (select)
    4. Filtering rows (filter, where)
    5. Adding and transforming columns (withColumn)
  • Spark SQL
    1. Registering a DataFrame as a table (createTempView)
    2. Running SQL queries on data (spark.sql)
    3. Combining DataFrame API and SQL
    4. Using SQL functions in the DataFrame API
  • Data sources
    1. CSV files
    2. Parquet – optimal format for Spark
    3. JSON files
    4. Delta Lake (basics)
  • Data processing
    1. Column and type transformations
    2. Handling missing values (null)
    3. Joining tables (join)
    4. Combining datasets (union)
  • Data aggregation
    1. Grouping (groupBy)
    2. Aggregate functions (count, sum, avg, min, max)
    3. Multiple aggregations at once (agg)
    4. Pivot tables
  • Troubleshooting
    1. Reading PySpark error messages
    2. Common errors: data types, missing columns
    3. Data checks and debugging
  • Outputs and exporting data
    1. Saving to files (CSV, Parquet)
    2. Visualizations in Databricks
    3. Downloading results
Prerequisites:
Basic Python (variables, loops, functions); experience with SQL, Excel, Power Query or Pandas is advantageous.
Recommended previous course:
Python – Programming Basics (PYTH1)
Schedule:
2 days (9:00-17:00)
Price per person:
432.00 € ( 522.72 € incl. 21% VAT)

Training and learning environment

IT classroom with desks, computer monitors, and a projection screen at the front.
What does the environment for in-person IT courses look like?We hold training courses in our own classrooms equipped with modern computer equipment for each participant and a large projection screen. Each workstation is fitted with a powerful computer for seamless practical exercises, and the environment supports a calm, friendly atmosphere during training.
IT classroom with desks, computer monitors, and a projection screen at the front.
Do I need to bring my own laptop to the classroom IT course?Our training rooms are fully equipped, so you do not need to bring your own laptop. A modern computer with all necessary hardware and software is ready for you at your workstation.
Sample ICT Pro certificate of successful completion of an IT training course.
Will you receive an official certificate after completing an IT course?Yes, upon successful completion of the training course, you will receive an official ICT Pro certificate confirming your participation and acquired technical skills. You can use it to verify your qualifications or add it to your professional profile.
Exterior view of the COMGUARD office building on Sochorova Street in Brno.
Where can you find our training center in Brno?The premises of the ICT Pro training center are located on the 1st floor of the COMGUARD building at Sochorova 38 in Brno. Positioned on the border of Žabovřesky and Komín, the location is easily accessible by both car and public transport.
Exterior view of the Opatov Park office building in Prague, Chodov.
Where can you find our training center in Prague?The premises of the ICT Pro training center are located on the 1st floor of the Opatov Park building at Líbalova 1, Prague 11 – Chodov. The location provides a comfortable environment for your education with convenient access.
Fully equipped kitchenette with a coffee machine, microwave, sink, and shelves with cups and glasses.
What facilities and refreshments are available to you during training breaks?Our training centers feature a fully equipped kitchenette for a comfortable break during your courses. The facilities include a coffee machine, microwave, kettle, and a wide selection of drinks and snacks.
Coffee station featuring an automatic coffee machine, yellow and white cups, glasses, and syrups.
What beverage refreshments are available during your course?A beverage station with a coffee machine, teas, syrups, and water is at your disposal throughout the entire training session. You can enjoy a coffee or another drink of your choice anytime during breaks.
Table with sweet pastries, fruit syrups, a water dispenser, and glasses set up for training participants.
What light refreshments are provided for you during the courses?During course breaks, participants can enjoy light sweet snacks, pastries, and fruit syrups to mix their own drinks. Fresh water dispenser and plates are also available for your convenience.
Wicker baskets filled with packaged savory snacks, sweet biscuits, and snack bars.
Should I bring my own snacks to ICT Pro training?There is no need to. A wide selection of savory snacks, sweet biscuits, and snack bars is available for you during breaks. Boost your energy for all-day focus right inside the training center premises.
Outdoor wooden terrace with green surroundings by the glass building of the training center.
Can you spend training breaks in the fresh air?Yes, an outdoor terrace is available at our training center. During breaks between learning sessions, you can relax in the fresh air in a pleasant green environment.
Spacious outdoor wooden terrace with seating and views of the surrounding area.
Where can you relax during course breaks?Our training center features a spacious outdoor terrace with seating options. During breaks between learning sessions, you can step outside to relax and recharge in the fresh air.