UII R and Big Data

Course content for the R and Big Data class 2019 at UII.

• course R and Big Data

  • course introduction
  • basics
  • recap tidyverse
  • R web scraping
  • R web scraping advanced
  • introduction to distributed computing (hadoop & spark)
  • connect R and spark (quickstart in the cloud)
    • databricks community edition
    • import sample notebook https://docs.databricks.com/_static/notebooks/sparklyr.html
  • connecting R and spark
  • practical intro to dataframes
    • lab 1.1
    • lab 1.2
    • lab 1.3
  • partitioning, UDF and joins
  • extensions (not part, only mentioned briefly)
    • scala UDF
  • machine learning
    • brief recap machine learning methods
    • distributed ML, scoring, governance
    • https://github.com/harryprince/awesome-sparklyr
    • anomaly detection using h2o
  • streaming
    • streaming and shiny
    • intro watermarking slides
    • streaming advanced (sliding windows, joins)
  • further (advanced) content if time permits
    • applications geospatial
    • graph analytics
    • xgboost
  • capstone project

• faculty talk