Week 6 Image Transfermation - Style Transfer

Presentation: Slides

• What is Style Transfer
• How does it work
• What Neural Networks See
• Different kinds of style transfer
• How to train your own style transfer
• Use the trained model in ml5.js (one image and webcam images)

Coding session:

• Running Style Transfer model in ml5.js, p5 sketch
• Training a new Style Transfer model, Run this Google Colab. Watch this video1, video2, p5 web editor sketch
  • Some Notes:
    • Open the colab, make sure the GPU is enabled: Menu - Runtime - Change runtime type
    • Run through each cell, wait for each cell to finish running, make sure there is no error in each cell's output
    • Step 2 and 3 may take 1 and 2 hours to finish, keep the tab open and active while waiting(Power your computer while waiting)
    • Once step 2(download datasets) finishes, don't re-run it, becase it takes a long to finish
    • While running step 2(download dataset), it might notify you that "Disk is almost full", ignore that

See demos live

• Style Transfer on one image
• Style Transfer on webcam images

Homework

Style Transfer projects

• Cubist Mirror
• Anime/Manga-Style Filter with PoseNet and Style Transfer

Resource:

• ml5js style transfer pre-trained models
• Google Colab
• Training a style transfer model with on Spell, github
• Training a style transfer model with on Spell, video
• Introduction to Spell (for Machine Learning in the Cloud)
• Creating your own style transfer mirror with Gradient° and ml5.js