Some ML Tools

Tools

An overview of some new creative tools that use ML.

  • Teachable Machine
  • RunwayML
  • EbSynth
  • Photoshop Neural Filters

Teachable Machine

Google’s Teachable Machine tool allows you to train a (neural-network-based) image recognition system, in the browser, without code. Note that its models can also be used with p5.js.

Live demo! I show a version I got working with p5.js here. It was easy!

Some of the people contributing to the project include recent CMU architecture alumna Irene Alvarado, and undergrad Design alums, Lucas Ochoa and Gautam Bose. Gautam and and Lucas have made an absurd cereal sorter:

Here’s another project breakdown by Gautham and Lucas, on making a system that tells you when your pancakes are ready to flip:

Shiffman has made a Coding Train tutorial:

Need more? Here’s a tutorial for a Head Tilting Detector.


RunwayML

RunwayML is a startup that packages recent ML research into automatable workflows that artists can use with no coding. It has dozens of offerings, including the following:

RunwayML’s MaskRCNN – Detects objects in an image while simultaneously generating a high-quality segmentation mask:

RunwayML’s AttnGAN – Turn text descriptions of scenes into synthesized images:

RunwayML’s SPADE-landscapes – Generate realistic images of landscapes from sketches and doodles:

RunwayML’s BigBiGAN – Generate similar images based on the input image:

RunwayML’s Background Matting – Real-Time High-Resolution Background Matting:

RunwayML’s HiFill – High resolution image inpainting with HiFill:

Lots of Runway help information here.


EbSynth

Easy style transfer for video. “Teach” the system how it should look by painting over one frame, and it carries the texture forward over time.


Photoshop’s New (2021) Neural Filters