Vision Workshop

From FutureTheater Wiki

Introductory Computer Vision in Processing
A workshop by Golan Levin and Kyle McDonald

---

Contents 1 Artistic Overview 1.1 Notable Works 2 Workshop Materials 2.1 Simple image processing 2.2 Higher-level analyses using OpenCV 2.3 Computer vision in Processing with the Kinect 3 Other Resources

Artistic Overview

Some of the material here is taken from my article, Computer Vision for Artists and Designers.

Notable Works

Here is a compilation of notable interactive artworks that use computer vision.

• Myron Krueger, Videoplace (1974)
• Camille Utterback & Romy Achituv, Text Rain (1999)
• Chunky Move, Mortal Engine (2006)
• Philip Worthington, Shadow Monsters (2006)
• Tmema (Levin + Lieberman), Messa di Voce (2003)
• Klaus Obermaier, Apparition (2005) (Also)
• Art+Com, Jew of Malta (2002)
• David Rokeby, Sorting Demon
• Christian Moeller, Cheese (2003)

Various works by Scott Snibbe:

• Boundary Functions (1998)
• Deep Walls (2003)
• Falling Girl (2008)

Various works by Rafael Lozano-Hemmer:

• Subtitled Public (2005)
• Underscan (2005)
• Standards and Double-Standards (2004)
• Surface Tension (1993)

---

Workshop Materials

Here is the bundle of downloads we will use for today's workshop.

Simple image processing

• Accessing pixels
• Simple thresholding (File/Examples.../Libraries/Video/BrightnessThresholding)
• Simple background subtraction (In download bundle)
• Background subtraction (File/Examples.../Libraries/Video/BackgroundSubtraction)
• Frame differencing (File/Examples.../Libraries/Video/FrameDifferencing)
• Brightness tracking (File/Examples.../Libraries/Video/BrightnessTracking)

And then some applications:

• LaserTag (In download bundle)
• TextRain (In download bundle)
• SlitScanTiny (In download bundle)

Higher-level analyses using OpenCV

• Blob segmentation & Contour extraction (In download bundle)
• Face detection (In download bundle)

For the OpenCV examples, we'll use the OpenCV library for Processing.
Complete instructions are there. I have already included You'll need to download three things from that site:

• (For MacOSX) Download and install the opencv-framework-1.1.dmg
• Download, unzip, and move the opencv_01.zip (OpenCV Library) into your Processing libraries folder.
• Download and unzip the opencv_examples.zip.

Computer vision in Processing with the Kinect

Further possibilities for computer vision are made possible with the Kinect, a "depth camera" that can sense how far away each pixel is from the camera, in addition to its RGB color. Daniel Shiffman from NYU has created a Processing interface for the Kinect; his complete instructions are here: http://www.shiffman.net/p5/kinect/

1. Download the following: https://github.com/shiffman/libfreenect/raw/master/wrappers/java/processing/distribution/openkinect.zip (These materials are also included in the Workshop Bundle for your convenience.)
2. With your Processing app closed (not running), unzip and install the openkinect folder into your Processing "libraries" folder, i.e. Sketchbook/libraries/openkinect
3. Now launch Processing. Check that the library is installed properly by going to: Processing->Sketch->Import Library->openkinect
4. You can now open and run Daniel's examples: Processing->Open...

   • (Sketchbook)/libraries/openkinect/examples/AveragePointTracking
   • (Sketchbook)/libraries/openkinect/examples/PointCloud
   • (Sketchbook)/libraries/openkinect/examples/RGBDepthTest

---

Other Resources

There are too many great resources out there to count, but I happen to find the following sites to be quite useful.

• Fisher, Robert, et. al. HIPR (The Hypermedia Image Processing Reference).
• Fisher, Robert, et. al. CVonline: The Evolving, Distributed, Non-Proprietary, On-Line Compendium of Computer Vision.
• Huber, Daniel et. al. The Computer Vision Homepage.