Overview
HackTheChoir was a two day micro residency produced by iShed and Ian Danby which investigated new ways of thinking about classical concerts that could appeal to all music lovers. The hack was partnered with Bristol Proms and were lucky enough to include the amazing Erebus Youth Ensemble.
The Hack had no limitations on what technology could be used, the only stipulation was that any experiments or products generated during the hack should become part of the open audience performance in Watershed’s Cafe Bar. This post will illustrate one experiment which attempted to gather and display multiple choir members pulses rates in real time using the Amped Pulse Sensors.
Sensors
The Amped sensor sits comfortably either on the earlobe or on the end of a finger. An LED is shone into the skin, then a photocell measures the changes in light intensities. This data is read by the a micro-controller capable of reading analog values.
The sensors come with both a Arduino and Processing sketches which converts raw pulse data into Serial data and then sends the data to be visualised in Processing.
Getting the Data
This was a great starting point but, the Arduino sketch only allows one pulse sensor to be used at a time as it uses intricate timers to calculate the BPM, we needed to use 10 sensors!
Our solution was to utilise the Firmata protocol on a Arduino Mega (this has 16 Analog input pins) and return the raw pulse data into an openFrameworks application that used a modified version of Patricio Gonzalez Vivo’s ofxPulseSensor addon.
This next section walks through our solution.
First, upload the Standard Firmata Sketch onto the Arduino.
The next step is to connect all the Pulse Sensors to the Arduino, for this we used a breadboard to supply the Sensors with 5v and ground points. We then separated the signal wire and plugged it into the analog pins. (As below).
Next we modified the ofxPulseSensor addon, by removing all references to the ofSerial object. Originally this object allowed the application to communicate to the Arduino via the Serial port much like the Processing sketch but, for this project we only needed access to the calculations and return functions. We added void pushRawData(int raw); to the ofxPulseSensor.h file which enabled us to push analog data from the Firmata protocol into the calculation functions via the Signal variable.
//-------------------------------------------------------------
void ofxPulseSensor::pushRawData(int raw)
{
Signal = raw;
}
Next we initialised an ofArduino object inside our ofApp files.
In ofApp.h
void setupArduinoUnit(string SerialPort,int baud);
void setupArduino(const int &version);
ofArduino arduino;
bool bArduinoIsHere;
ofxPulseSensor pulseSensors[10];
In ofApp.cpp
//--------------------------------------------------------------
void ofApp::setupArduinoUnit(string SerialPort,int baud)
{
arduino.connect(SerialPort);
ofAddListener(arduino.EInitialized, this, &ofApp::setupArduino);
bArduinoIsHere = false;
}
//--------------------------------------------------------------
void ofApp::setupArduino(const int & version)
{
ofRemoveListener(arduino.EInitialized, this, &ofApp::setupArduino);
// It is now safe to send commands to the Arduino
bArduinoIsHere = true;
for (int aPins = 0; aPins < 15; aPins++)
{
// Make all the analog pins report
arduino.sendAnalogPinReporting(aPins, ARD_ANALOG);
}
// Listen for changes on the digital and analog pins
ofAddListener(arduino.EDigitalPinChanged, this, &ofApp::digitalPinChanged);
ofAddListener(arduino.EAnalogPinChanged, this, &ofApp::analogPinChanged);
}
//--------------------------------------------------------------
void ofApp::setup()
{
// Open the Connection to the Arduino
setupArduinoUnit("/dev/tty.usbserial-A900ac8s",57600);
for(int i = 0; i < 10; i++)
{
pulseSensors[i].setup(-1);
}
}
This ensures that all of the Analog Pins are set to report and that 10 pulse sensors have been initialised.
Now we return the analog values from the Arduino to the Pulse Sensor objects.
//--------------------------------------------------------------
void ofApp::update()
{
arduino.update();
for(int i = 0; i < 10; i++)
{
pulseSensors[i].pushRawData(arduino.getAnalog(i));
pulseSensors[i].update();
}
}
Displaying the Data
We had two mechanisms for displaying the data. One was a beat display which flashed a red marker every time a heartbeat was detected and the other was the classic Electrocardiograph graphic.
The beat detector worked by comparing mapped analog data to a thresholded value. If the data surpassed the threshold it would alter a boolean state that triggered an animation for a specific sensor.
//--------------------------------------------------------------
void ofApp::getBeats()
{
for (int i = 0; i < 10; i++) { pBeat[i] = false; if (pulseSensor.getAnalog(i) > ofMap(pRoof[i],0,100,0,highRange))
{
pBeat[i] = true;
}
}
}
We then called getBeat(); inside the main update loop.
//--------------------------------------------------------------
void ofApp::update()
{
getBeats();
for(int i = 0; i < 10; i++)
{
if (pBeat[i] == true)
{
pFadeRate[i] = 255;
}
else
{
pFadeRate[i] -=15;
}
}
}
//--------------------------------------------------------------
void ofApp::draw()
{
ofPushStyle();
for(int i = 0; i < 10; i++)
{
if (pBeat[i] == true)
{
color[i].set(ofMap(pFadeRate[i], 0, 255, 0.00, 1.00), 0, 0);
ofSetColor(255, 0, 0);
}
else
{
color[i].set(0, 0, 0);
ofSetColor(0, 0, 0);
}
ofCircle(ofPoint(ofGetWidth()-300,50+(i*100)),40);
}
ofPopStyle();
}
The animation set the colour of a circle to red then gradually faded out to black. Here is what it looked like with an LED.
The Electrocardiograph was generated from incoming pulse data (which was remapped to values between 0 and 100) and a counter which increased every frame. These values are pushed to the end of an array every frame, so that the latest data is always at the end.
To ensure the data points did not go off the screen, we made a small conditional which checked the current value of the counter against the margins of the application window. If the counter reached or exceeded the value then the counter was reset to 0 and the array is emptied of data.
//--------------------------------------------------------------
void ofApp::update()
{
ECH0pts.push_back(ofPoint(pulseTime,ofMap(arduino.getAnalog(0),lowRange,highRange,100,0)));
// Progress the Visualisor
pulseTime +=2;
// If the counter reaches the Screen Width Reset
if (pulseTime >= ofGetWidth()-200)
{
pulseTime = 0;
ECG0pts.clear();
}
}
To draw the graphic we simply iterated through the array and created a polyline from the coordinates in the array.
//--------------------------------------------------------------
void ofApp::draw()
{
ofNoFill();
if (ECG0pts.size() > 0)
{
ofSetColor(0,255,100);
ofBeginShape();
for (int i = 0; i < ECG0pts.size()-1; i++)
{
ofVertex(ECG0pts[i]);
}
ofEndShape(false);
}
}
The results of this experiment showed that choir members pulse rate started to synchronise as soon as they started to sing. In the image below you can see that their approximate heart rates generally became 101 but, what was more interesting was the actual pulse itself. In the Electrocardiograph, sensors 2,3,7,8 and 9 have share a pattern where the peak and troughs are nearly identical.
In future experiments it would be interesting to see what the conductors pulse does? Will it synchronise? Will it surpass the choirs?
