Weather Control was an installation project programmed in Processing and used Synapse for skeleton tracking and Box2D for physics.


This was one of many Kinect experiments I’ve been up to lately. Pretty excited about the possibilities of using this tool for clients and creating some cool interactive experience at retail.

Choose between Rain, Snow or Sun. The Rain scene has well, rain drops that fall in whichever direction the mouse is pointing. Users control an umbrella that deflects the rain drops. Lifting your hands up causes it to “Make It Rain” as dollar bills fall. The Snow scene has snow flakes that float down unless the user moves closer to the Kinect and then all the snow particles attract to the users hand. The Sun scene lets users bounce falling beach balls. Pressing the “c” key causes the beach balls to turn into crabs. Seemed funny at the time. There are a few other secret modes such as “Kitten Mode” and “Snow Monster”. Good times all around.

Center of Mass tracking is a great tool because it doesn’t require the user to stop and assume a pose for the kinect to track them. A more passive experience, the kinect would automatically recognize humans and calculate the center most position of their mass, which is usually around your belly button. This makes it easy to create projects that track people in malls, on sidewalks in-front of store windows and wherever else you can imagine to have graphics or animations interact with them.

For this kinect hack I used Processing and the SimpleOpenNI and NITE libraries. If you haven’t downloaded them you can do so here. If this is your first time working with processing and kinect I would also recommend you check the book Making Things See. Installing the NITE files requires using the terminal if your on a mac which can be tricky. The Making Things See book walks you through the process easily.

Download the timer class from Daniel Shiffman. This used used to trigger the save and reset each hour. Have fun.

If you want to try it out here’s the processing code:

import SimpleOpenNI.*;
SimpleOpenNI kinect;

Timer timer;

void setup() {
 size(800, 600);

kinect = new SimpleOpenNI(this);
kinect.enableDepth();
kinect.setMirror(true);

kinect.enableUser(SimpleOpenNI.SKEL_PROFILE_NONE);
   smooth();
   noStroke();
   background(0);  //set background to black
   
   //Timer stuff
 timer = new Timer(3600000);
  timer.start();

}

void draw() {
 kinect.update();
//image(kinect.depthImage(), 0, 0);
//Timer call
 if (timer.isFinished()) {
     saveReset();
    timer.start();
  }

IntVector userList = new IntVector();
kinect.getUsers(userList);

for( int i=0; i<userList.size(); i++) {
  int userID = userList.get(i);
  PVector position = new PVector();
  kinect.getCoM(userID, position);

  kinect.convertRealWorldToProjective(position, position);

  float zDepth = map(position.z, 500, 5000, 10, 30);
  float zScale = map(position.z, 500, 5000, 20, 100);
  //println(position.z);
  
  color[]      userCoMColors = { color(255,100,100,zDepth), color(100,255,100,zDepth), color(100,100,255,zDepth), color(255,255,100,zDepth), color(255,100,255,zDepth), color(100,255,255,zDepth) };
pushStyle();
   fill(userCoMColors[userID % userCoMColors.length]);
   ellipse(position.x, position.y, zScale, zScale);
 popStyle();

  }
}
  
void saveReset() {
   saveFrame("images/sketch-###.jpg");
   background(0);  //set background to black
}
 

If your office is like mine then coming into the kitchen just to find the coffee pot empty and no one has refilled it drives people crazy. Well not anymore with this simple coffee pot alarm. Really this is a simple exercise in using a FSR or Force Sensitive Resistor. By using a simple sensor we can measure the force or basically weight of the coffee pot.

What we’re doing here is simply returning a value between 0 and 1024 from the FSR and when the value falls within one of our specified ranges activates an led or combination of led and buzzer. Let the fun begin.

What You’ll Need:

• Arudino
• Force Sensitive Resistor
• Jumper Wires
• 1 Red Led, 1 Orange Led, 1 Green Led
• Piezo Buzzer
• 10k ohm resistor
• 9V batter and connector optional

Here I have my Led’s set as digital outputs with pin 2 = red, pin 4 =orange and pin 7 =green. My buzzer is an analog output on pin 5. Our FSR is also an analog reading on A0. Once you’re ready to go be sure to launch your Serial Monitor (Tools >Serial Monitor) to get text feedback during testing. As you squeeze the sensor we’ll go from “No pressure Getting Refilled” for when the coffee pot has been removed from the sensor to “Coffee Full!” when the pot has been refilled. You may need to adjust your values depending on the weight of your coffee pot at various levels. The FSR I’m using has a max weight of 22 lbs and many are around the 20 lb range.

This project is completely Arduino based. Here’s the code:

/* FSR simple coffee pot alert. 

Connect one end of FSR to power, the other end to Analog 0.
Then connect one end of a 10K resistor from Analog 0 to ground 

For more information see www.ladyada.net/learn/sensors/fsr.html */

int fsrPin = 0;     // the FSR and 10K pulldown are connected to a0
int fsrReading;     // the analog reading from the FSR resistor divider
int LEDR = 2;       //Red Led
int LEDO = 4;       //Orange Led
int LEDG = 7;       //Green Led

void setup(void) {
  // We'll send debugging information via the Serial monitor
  Serial.begin(9600);
 pinMode(LEDR, OUTPUT);
 pinMode(LEDO, OUTPUT);
 pinMode(LEDG, OUTPUT);
 pinMode(5, OUTPUT);    //Piezo buzzer
}

void loop(void) {
  fsrReading = analogRead(0);  

  Serial.print("Analog reading = ");
  Serial.print(fsrReading);     // the raw analog reading

  // We'll have a few threshholds, qualitatively determined
  if (fsrReading < 10) {
    Serial.println(" - No pressure Getting Refilled");   //writes to serial monitor, be sure to open it
    refill();             //function that cycles leds written below
    digitalWrite(5,LOW);
  } else if (fsrReading < 200) {
    Serial.println(" - Empty");
    digitalWrite(LEDR, HIGH);    //Turns Red Led on
    analogWrite(LEDO, LOW);
    analogWrite(LEDG, LOW);
    analogWrite(5,50);          //sounds buzzer, range can go up to 256, but set to 50 so not to loud
  } else if (fsrReading < 500) {
    Serial.println(" - Getting Low");
    analogWrite(LEDR, LOW);
    digitalWrite(LEDO, HIGH);   //Turns Orange Led on
    analogWrite(LEDG, LOW);
    digitalWrite(5,LOW);
  } else if (fsrReading < 850) {
    Serial.println(" - Enough Coffee");
    analogWrite(LEDR, LOW);
    analogWrite(LEDO, LOW);
    analogWrite(LEDG, LOW);
    digitalWrite(5,LOW);
  } else {
    Serial.println(" - Coffee Full!");
     analogWrite(LEDR, LOW);
     analogWrite(LEDO, LOW);
     digitalWrite(LEDG, HIGH);   //Turns Green Led on
     digitalWrite(5,LOW);
  }
  delay(1000);
} 

//Function that runs when no pressure is registered to cycle leds
void refill() {
  digitalWrite(LEDR, HIGH);
  delay(100);
  digitalWrite(LEDR, LOW);
  delay(100);
  digitalWrite(LEDO, HIGH);
  delay(100);
  digitalWrite(LEDO, LOW);
  delay(100);
  digitalWrite(LEDG, HIGH);
  delay(100);
  digitalWrite(LEDG, LOW);
  delay(100);
  digitalWrite(LEDR, HIGH);
  delay(100);
  digitalWrite(LEDR, LOW);
  delay(100);
  digitalWrite(LEDO, HIGH);
  delay(100);
  digitalWrite(LEDO, LOW);
  delay(100);
  digitalWrite(LEDG, HIGH);
  delay(100);
  digitalWrite(LEDG, LOW);
  delay(2000);
}

While the circuit may look a little busy, remember the two lower breadboard rows are 5V and ground running horizontally on the board. This is a rough demonstration so I'm sure you can make it much neater if your going for a finished product. The code is a great jumping off point for other projects as well so get creative and try to add other sensors.

If your fingers get tired from constantly clicking refresh to make sure you haven’t missed out on the latest awesome Agency Spy post then you’re in luck. I’m pleased to announce the Agency Spy Alertometer. Simply follow my easy to construct 94 step process and you’ll have this puppy glowing on your desk in no time. Ok, it’s not actually that difficult.

The Agency Spy Alertometer is pretty cool though. This simple lamp and desktop widget will keep you up to date on the latest agency news by alerting you to new stories by changing colors. Whenever a new article is posted the lamp will change colors based on which agency is mentioned. If an insider leaks breaking news on DDB the lamp will turn yellow, if someone defects at CP+B the lamp will glow orange. If that’s not enough for you it comes with a handy desktop widget that matches the color of the lamp and displays the article title.

The alertometer is a good project for creatives learning how to connect the digital and physical world. A combination of arduino and processing, you’ll dip your toe in the world of code, don’t worry it’s provided and at the same time connect a few simple wires to power your lamp.

This project is in response to the growing demand for hands on interactive art directors. I’m a strong believer that you can’t create truly innovative ideas unless you know what’s possible. The future of the internet will lie in the interaction of online data with physical devices and tinkering with projects such as this is as good a place to start as any.

What you’ll need:

• Download the Processing SDK here
• Download the Arduino SDK here
• An arduino board available at radio shack or online at sparkfun.com
• Breadboard
• A 140 or 270 ohm resistor
• 1 RGB LED (you can use more to make it brighter)
• USB connector for Ardunio (should come with it)
• Jumper wires
• Sweet Agency Spy paper lamp Download here
• Widget background graphic, or create your own

You’ll also need to download the simpleML library for processing to work with XML data here.

Don’t get nervous it’s going to be ok. You can learn how to install a library here.

I’m not going to go super deep, you’ll want to check out the processing and arduino site’s to learn more or check out two helpful books. Getting Started with Arduino and Getting Started with Processing. They were really helpful in creating the alertometer.

So what is going on: Basically we’ve just written a program in processing that grabs the latest Agency Spy Article title. We have the program scan the title and search for words that match our list of agencies. If there is a match we update a color value that we have assigned to that agency title. That color is then passed to the arduino and changes the color of the lamp. This is a super stripped down explanation, but hopefully you get the point. Like I said I’m a creative, not a developer, or maybe a little of both, but I think everyone should be.

You’ll want to upload your arduino code first. Then run your processing sketch ( a sketch is what code is called in processing).

Here’s the Arduino code:

// Arduino Networked Lamp from Getting Started with Ardino
//

#define R_LED 11
#define G_LED 10
#define B_LED 9

int val = 0; // variable to store the value coming from the sensor

//int btn = LOW;
//int old_btn = LOW;
//int state = 1;
char buffer[7] ;
int pointer = 0;
byte inByte = 0;

byte r = 0;
byte g = 0;
byte b = 0;

void setup() {
  Serial.begin(9600);  // open the serial port
}

void loop() {

  if (Serial.available() >0) {

    // read the incoming byte:
    inByte = Serial.read();

    // If the marker's found, next 6 characters are the colour
    if (inByte == '#') {

      while (pointer < 6) { // accumulate 6 chars
        buffer[pointer] = Serial.read(); // store in the buffer
        pointer++; // move the pointer forward by 1
      }

      // now we have the 3 numbers stored as hex numbers
      // we need to decode them into 3 bytes r, g and b
      r = hex2dec(buffer[1]) + hex2dec(buffer[0]) * 16;
      g = hex2dec(buffer[3]) + hex2dec(buffer[2]) * 16;
      b = hex2dec(buffer[5]) + hex2dec(buffer[4]) * 16;

      pointer = 0; // reset the pointer so we can reuse the buffer

    }
  }   

    analogWrite(R_LED, r);  // turn the leds on
    analogWrite(G_LED, g);  // at the colour
    analogWrite(B_LED, b);  // sent by the computer

  delay(100);                // wait 100ms between each send
}

int hex2dec(byte c) { // converts one HEX character into a number
    if (c >= '0' && c <= '9') {
      return c - '0';
    } else if (c >= 'A' && c <= 'F') {
      return c - 'A' + 10;
    }
}

Here's the Processing code:

//Import libraries
import simpleML.*;
import processing.serial.*;

// Variables
String feed = "http://www.mediabistro.com/agencyspy/feed/rss";
int interval = 10;  // retrieve feed every 10 seconds;
int lastTime;       // the last time we fetched the content
color c = (105); //background color variable
String cs; //color sent to arduino
String oldcs; //check old color to see if updated
String headline = "Updating"; //Current headline from blog
//int light = 0;  // light level measured by the lamp
Serial port;
String buffer = ""; // Accumulates characters coming from Arduino
PImage b; //store logo image

XMLRequest xmlRequest;  //request to blog for title

void setup() {
  size(500, 200);
  frameRate(10);    // we don't need fast updates
  fill(255);
  // Creating and starting the request
  xmlRequest = new XMLRequest(this, feed);
  xmlRequest.makeRequest();

  lastTime = 0;

  String arduinoPort = Serial.list()[1];
  port = new Serial(this, arduinoPort, 9600); // connect to Arduino
}

void draw() {
  background(c);

  //background image
  b = loadImage("AlertometerUI.png");
  image(b, 0, 0);

  int n = (interval - ((millis()-lastTime)/1000));

  fill(0); //sets text color

  cs = "#" + hex(c, 6); // Prepare a string to be sent to Arduino

  text("Sending:", 15, 188);
  text(cs, 78, 188);
  text("Next update in "+ n + " seconds", 310, 188);
  text( headline, 38, 135);

  if (n <= 0) {
    fetchData();  //checks for new article post
    lastTime = millis();

    if (oldcs != cs) {
      updateBlink();
    }
  }

  port.write(cs); // send data to Arduino

  if (port.available() > 0) { // check if there is data waiting
    int inByte = port.read(); // read one byte
    if (inByte != 10) { // if byte is not newline
      buffer = buffer + char(inByte); // just add it to the buffer
    }
    else {

      // newline reached, let's process the data
      if (buffer.length() > 1) { // make sure there is enough data

        // chop off the last character, it's a carriage return
        // (a carriage return is the character at the end of a
        // line of text)
        buffer = buffer.substring(0, buffer.length() -1);

        // turn the buffer from string into an integer number
        //light = int(buffer);

        // clean the buffer for the next read cycle
        buffer = "";

        // We're likely falling behind in taking readings
        // from Arduino. So let's clear the backlog of
        // incoming sensor readings so the next reading is
        // up-to-date.
        port.clear();
      }
    }
  }
}

// When the request is complete, prints title to
String netEvent(XMLRequest ml) {
  // Retrieving an array of all XML elements inside "" tags
  String[] headlines = ml.getElementArray("title");
  for (int i = 0; i < 1; i++) {
    // println(headlines[1]);
    headline = (headlines[1]);
  }
  return headline;
}

void fetchData() {

  //make call again to site
  xmlRequest.makeRequest();
  try {
    URL url = new URL(feed);  // An object to represent the URL
    // prepare a connection
    URLConnection conn = url.openConnection();
    conn.connect();
    BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
  }
  catch (Exception ex) { // If there was an error, stop the sketch
    ex.printStackTrace();
    System.out.println("ERROR: "+ex.getMessage());
  }

  // Look for names of agencies or keywords in title
  String[] m1 = match(headline, "Odds");
  String[] m2 = match(headline, "Euro");
  String[] m3 = match(headline, "DDB");
  String[] m4 = match(headline, "BBDO");
  String[] m5 = match(headline, "Leo");
  String[] m6 = match(headline, "Mithun");
  String[] m7 = match(headline, "CP+B");
  String[] m8 = match(headline, "Chiat");
  String[] m9 = match(headline, "Grey");
  String[] m10 = match(headline, "Razorfish");
  String[] m11 = match(headline, "72andSunny");
  String[] m12 = match(headline, "Draft");
  String[] m13 = match(headline, "Digitas");
  String[] m14 = match(headline, "Ogilvy");
 String[] m15 = match(headline, "AKQA");

  // if agency name found, set to appropriate color
  if (m1 != null) {
    c = color(89, 133, 39);
  }
  else {
    c = color(255, 60, 0);  //Set to standard orange if no matching agency in title
  }
  if (m2 != null) {
    c = color(0, 155, 135);
  }
  if (m3 != null) {
    c = color(255, 255, 0);
  }
  if (m4 != null) {
    c = color(255, 72, 0);
  }
  if (m5 != null) {
    c = color(255, 255, 255);
  }
  if (m6 != null) {
    c = color(255, 222, 0);
  }
  if (m7 != null) {
    c = color(255, 66, 0);
  }
  if (m8 != null) {
    c = color(255, 255, 0);
  }
  if (m9 != null) {
    c = color(105, 105, 105);
  }
  if (m10 != null) {
    c = color(230, 255, 0);
  }
  if (m11 != null) {
    c = color(255, 150, 0);
  }
  if (m12 != null) {
    c = color(255, 150, 0);
  }
  if (m13 != null) {
    c = color(255, 255, 255);
  }
  if (m14 != null) {
    c = color(255, 0, 0);
  }
   if (m15 != null) {
    c = color(105, 105, 200);
  }  

  println(headline);
  println(cs);
  oldcs = cs;
}

//simple function for setting a delay. Not that important.
void myDelay(int ms) {
  int time = millis();
  while (millis ()-time < ms);
}

//Blink light when new article found. Not working yet.
void updateBlink() {
  for (int i = 0; i < 10; i++) {
    c = color(255);
    myDelay(500);
    c = color(0);
    myDelay(500);
  }
}

It's time to wire this bad boy up. It's pretty simple. Pin 11 is Red, Pin 10 is Green, Pin 9 is Blue. Check out the photos.
Make sure you're grounded and try not to set anything on fire.

Once your light is shining grab the Agency Spy lamp shade to cover the arduino and enjoy.

About Me:

I'm an interactive creative working in Chicago, with experience at shops such as Euro RSCG, Leo Burnett and Draft FCB. I have a passion for all things advertising and especially how it will be shaped by digital in the future. If you have a second feel free to take a look through the rest of my book.

I’m using a cheap infrared motion sensor attached to an arduino to capture human motion. I’ve placed it inside a cup to limit the area it detects since I’m at work and there are a lot of people moving around. The arduino simply reads if motion is detected or if the motion has stopped. This data is then passed through the serial port and read by processing.

In my processing sketch I’m using a port.readChar(); method to return an integer. 1 for motion detected and 0 for motion stopped. I can then run a simple if statement. If motion detected I trigger a takePicture(); function I wrote.

This is where it gets more interesting. The function uses the computers webcam to take a photo and save it to the desktop. That image is also uploaded to twitpic at the same time, auto-tweeting the person who activated the camera. Most of the time is was just weird pictures of me since I was in front of the camera, but if you aim it towards the sensor you’ll capture the person or cat if that’s your thing who set it off.

Other than misspelling my twitter password and losing a couple hours trying to figure out why it wasn’t working, this wasn’t to bad of a project. There are some awesome libraries out there that make things easy.

For those of you in advertising who are wondering why this us useful, it’s a simple example of how the physical and digital world is going to increasingly collide. The ability for physical devices to interact with the internet and vicev ersa is going to become increasingly important and brands will look to interactive art directors to come up with innovative ways to help them connect with consumers.

To view some terrible pictures of me and other people in the office check out the @Aus10_Bot

OpenGl was used to create a simple 3D cube. A more complex environment could be used but this seems to do the trick. A camera was then placed in the scene to orbit around the cube and view it from different sides. The camera was controlled by the position of my mouse on screen. If I moved the mouse right it turned right, etc. This is pretty similar to how we move through video games and online environments.

The next step was to eliminate the mouse. I did this using Face Detection. The camera on my computer first recognized if a face was present, and then tracked my faces position up and down, left and right. The next part was simple. I replaced the mouses X and Y position with that of my faces position. Now as I moved my head left the cube would turn left.

The problem here is that if I was trying to look around a corner to view the side of the cube I would see the side of the cube instead of the cube’s front surface turning to the left as well. This was solved by reversing the camera’s Z position which was set to orbit the cube. By multiplying the Z position by negative 1 I was able to get a realistic view of the cube. Now when I moved my head to the left I was looking around the cube.

This concept could make for some pretty cool interfaces online and especially on mobile phones with front facing cameras. If you’re up for trying it out I’ve attached the source below.

Download FaceCubePerspective.pde

Note: You’ll need the openGL library to create the cube and the Open CV library for face detection.

Get Open CV here

I’ve finally been inspired to start digging into arduino and am quickly learning there’s a lot more to physical computing than my phone’s accelerometer. As an Interactive art director I’ve made it a point to learn as much about the code side of things as I have the visual design. It’s not that I’m looking to master every language. I’m grateful to have genius developers for that, but to come up with ground breaking ideas you need to know what your capabilities are. This has helped me a lot in the mobile and flash space, but I’ve been pretty limited beyond that.

Physical computing or interaction design is a great place for interactive designers to stretch their imaginations and take all that data they’ve been pushing around into a new dimension.  Arduino is a simple and easy way to start playing with some of those concepts. It’s basically a tiny computer you can connect via usb to send and recieve information from sensors.

The possibilities are endless, but to get your brain going here are some of the sensors and modules you can use to read and display data.

• Photocells – Light sensors that detect how bright it is.
• Thermostats – Temperature check
• Magnetic switch (reed relay) – turns on and off when two magnets come together
• Carpet switch – detects when someone is standing on it
• Range Finder – can use sonar or infrared to tell distance
• Accelerometer – to tell tilt and speed
• Motion sensors- detects movement
• Potentiometer – a dial for adjusting analog input
• LED – lights
• LCD – screen for displaying info and graphics
• Servo motor – like a fan motor or possibly spin a robot wheel

There are a ton more. Check out www.sparkfun.com to buy components like these. Visit http://arduino.cc and http://www.instructables.com to learn more about the programing side.

The image at the top was a project I built from a great starter book called Getting started with Arduino. It uses Processing (another development language) and arduino to read a specified blog. In this instance the Roundarch blog and count the number of times three words appear. I counted the words “flash”, “Roundarch” and “mobile” since I knew they would appear in the text a lot. It then converted those numbers into a hex code to generate a color and lit up the rgb LED to that color. As  new blog entries are written the light will change to reflect the updates. Maybe not super practicle, but if your an interactive AD it definitely can get the imagination going.

So what else needs to happen before we see this. A few things, the constant debate over when we will see flash come to the iPhone and iPad. Have faith, it will come. Once it does, there will need to be a power boost in the 3G network. We may be calling it 4G or 5G, but users will not have patience to wait on these rich flash expereiences if it takes minutes to download. Anyways without futher at do, here is the Wired iPad video.

The time has finally come. My thesis survey is online and i’m ready for the results to start rolling in. I’m asking everyone to please take a moment to complete the survey. I will need all your help to make this a success.

Thank you all so much.

I’m currently working on a survey to collect my primary research for the paper. At the beginning of this project I became very focused on one-to-one communication versus one-to-many as it is now. Basically brands speaking with you as an individual through advertising versus brands speaking to a broad group of people through an ad. I’ve since backed out to look at the bigger picture.

I’m trying to define a few possible directions online advertising may go in the future and to test to see which the public is most comfortable and in favor of as well as what correlations in advertising have been going in these directions over time. The 5 general areas I and with the help of others have come up with at this point are:

• Content Production -(ie. sponsorships, creation of orginal media or entertainment tied to the brand)
• Applications – (ie. utility, utility, the trade personal information for valuable results, interactive)
• Informational – (ie. what we currently have, educational, persuasive)
• Transactional – (ie. Ability to purchase directly from the ad, not linked to online store)
• Conversational – (ie. social streams of advertising)

Through my research I hope to find evidence that will point to the growth of one or multiple trends in these areas and to what degree. You are welcome to throw out ideas and please feel free to direct me to any information you’ve run across relevant to this research.