#HackTheSpace: Light Histories

Overview

This post will highlight some of the technical solutions used to create a recent project at the Tate a part of the HackTheSpace event. The project dubbed Light Histories was a 100x75cm panel containing 9x Neopixel Rings that displayed abstracted images from the Tate’s archive in 16bit colour palettes.

Introduction

#HackTheSpace was the first official hacking event produced by the Tate Modern. We were lucky enough to be invited to take part in the inaugural 24 hour hackathon to create artworks from unique datasets provided by the Tate. The Tate specified that there were two categories of hacks, Alpha and Beta. Alpha projects were generated only in the 24 hours of the hack itself and the Betas were allowed to bring their ideas not matter what state they were in to the hack and produced them for the final showing.

Representing Watershed and the Pervasive Media Studio, Arthur Buxton (Colorstories) and I joined forces to create our intepretations of data and how it might be visualised. For those who might not have seen Arthur’s work, he produces stunning visualisations of paintings, images and other media, by abstracting the most prominent colours from images and turning them into a filled rings and other shapes. This process is very labor intensive and requires Arthur to painstakingly calculate the colour palettes of each image then collate them in Illustrator. However, recently he managed to attain funding on Kickstarter to produce a online platform for producing these images from photo albums from social media or photographic sites. Click here to see the Kickstarter Page.

The Very Hungary Catapiller by Arthur Buxton

For me it seemed a shame to pass up the opportunity to work with Arthur’s beautiful aesthetic and try to extend the principle behind the printed media, whilst avoiding screen based displays.

Lights,Camera,Action

We had to decide what data was going to generate the artwork. In initial experiments, we tried everything from earthquake intensities to temperature from round the UK’s cities, even a chronology of how many bicycles have been stolen in Bristol. None of this data suited our purpose. On further investigation we found that the Tate had provided a full list of their collection. We decided to use this data to visualise the entire back catalogue and display how the colour palettes of artists changed over time letting a narrative unfold based on this progression of time.

Scrapper

We utilised the Tate’s search form to generate a simple web scraper that would return image urls from the website.

This was the generic search url.
“http://www.tate.org.uk/art/search-fragment?ap=1&wp=1&tab=artworks&wn=9&ws=date&wv=grid&wot=6”

We customised the url by altering the following variables.

&wp=1 altered the result page number.
&wn=9 altered the number of records per page.
&ws=date ensured the results were in chronological order.

The entire online collection contains around 6,000 records! So we populated an array with 600 urls altering the &wp= variable.

for (int i = 0; i < 600; i++) 

{

  requestArrayOne.push_back("http://www.tate.org.uk/art/search-fragment?ap=1&wp="+ofToString(i)+"&tab=artworks&wn=9&ws=date&wv=grid&wot=6");

}

Screen Shot 2014-10-08 at 12.31.31

Returned Source Page

At application runtime, these urls were fired into the loadNewRequest(string request) which returned the webpage’s source code and temporarily stored it inside a buffer. Using Gumbo the webpage’s source code was scanned and any img tags were processed, this returned the img’s src path which were pushed into an array for use elsewhere in the application.

//--------------------------------------------------------------

void TateRequestData::loadNewRequest(string request)

{

    urls.clear();

    ofBuffer buffer = ofLoadURL(request);



    GumboOutput* output = gumbo_parse(buffer.getText().c_str());



    search_for_images(output->root);



    gumbo_destroy_output(&kGumboDefaultOptions, output);

}

//--------------------------------------------------------------

void TateRequestData::search_for_images(GumboNode* node)

{

    if (node->type != GUMBO_NODE_ELEMENT)

    {

        return;

    }



    GumboAttribute* img;



    if (node->v.element.tag == GUMBO_TAG_IMG &&

        (img = gumbo_get_attribute(&node->v.element.attributes, "src")))

    {

        if(ofIsStringInString(ofToString(img->value), ".jpg"))

        {

            urls.push_back("http://www.tate.org.uk"+ofToString(img->value));

        }

        else if(ofIsStringInString(ofToString(img->value), ".png"))

        {

            // Return the PM Studios logo

            urls.push_back("http://www.pmstudio.co.uk/pmstudio/sites/default/files/pmstudio-logo.jpg");

        }

    }

}

Every 8 seconds the program cycles through the list, loading the next set of images.
This 8 second timer was essential as it gave the program enough time to load the images, perform the CV algorithms and then display the results. In the development of the project we came across numerous issues that were eventually solved by this simple fix.

Color Abstraction

Once the image urls were loaded and the images were downloaded, they were Posterized which returns the most prominent colours in the image. (huge thank you to nkint for ofxPosterize).
Screen Shot 2014-10-15 at 10.35.10Normal Image – Palette – Posterized Image

The addon calculates both the average and clustered histograms. Histograms are essentially calculated representations of the distribution of colour within an image.

//--------------------------------------------------------------

void colorStoryRing::extractPalette() 

{

    // If we have an valid loaded image continue

    if(currentImage.isAllocated()) 

    {

        average = ofxPosterize::average(currentImage, _numberOfSegments);

        averageHistogram = ofxPosterize::getHistogram(average);

        averagePalette.clone(drawPalette(averageHistogram, 200, _numberOfSegments));



        clustered = ofxPosterize::clusterize(currentImage, _numberOfSegments);

        clusteredHistogram = ofxPosterize::getHistogram(clustered);

        clusteredPalette = drawPalette(clusteredHistogram, 200, _numberOfSegments);

        writeNewColors();

    }

}

//--------------------------------------------------------------

void colorStoryRing::writeNewColors()

{

    int num_histograms = 2;

    map<int, int> histograms[] = {averageHistogram, clusteredHistogram};

    clearColors();

    for (int i=0; i<num_histograms; i++) 

    {

        map<int, int> hist = histograms[1];

        for(std::map<int,int>::iterator iter = hist.begin(); iter != hist.end(); ++iter)

        {

            int k =  iter->first;

            ofColor c = ofColor::fromHex(k);

            generateNewColors(c);

        }

    }

}

//--------------------------------------------------------------

void colorStoryRing::generateNewColors(ofColor c)

{

    color.push_back(c);

}

These colours were simply transferred into the ring, which was generated on application load.

To generate the ring, we first specify a centre point and calculate both an array of coordinates for the inner and outer edges of the shape. In this setup routine we added the ability to alter the number of segments the ring would have, this ensured we could synchronise the number of pixels on each Neopixel Ring to the colors returned from the image.

//--------------------------------------------------------------

void colorStoryRing::setupRing(ofPoint pos, int outerRadius, int innerRadius, int numberOfSegments)

{

    _pos = pos;

    _outerRadius = outerRadius;

    _innerRadius = innerRadius;

    _numberOfSegments = numberOfSegments;



    neoRing.setupLedRing();



    for (int out = 0; out < _numberOfSegments; out++)

    {

        float angle = (1.0 * out) * (2.0 * M_PI)/(1.0 * _numberOfSegments);



        // Generate the outer ring coordinates

        float rx = _pos.x + (_outerRadius * cos(angle));

        float ry = _pos.y + (_outerRadius * sin(angle));

        outerPosition.push_back(ofVec2f(rx,ry));

    }

    for (int in = 0; in < _numberOfSegments; in++)

    {

        float angle = (1.0 * in) * (2.0 * M_PI)/(1.0 * _numberOfSegments);



        // Generate the inner ring coordinates

        float rx = _pos.x + (_innerRadius * cos(angle));

        float ry = _pos.y + (_innerRadius * sin(angle));



        innerPosition.push_back(ofVec2f(rx,ry));

    }

    for (int c = 0; c < _numberOfSegments; c++)

    {

        // Generate random colours to begin

        color.push_back(ofFloatColor(ofRandom(0.0,1.0),ofRandom(0.0,1.0),ofRandom(0,1.0)));

    }

}

To display the resulted image, the application simply cycled through the arrays of coordinates creating a series of Quad shapes, that were filled with the colours from the histogram.

//--------------------------------------------------------------

void colorStoryRing::drawRing()

{

        ofPushStyle();

        glBegin(GL_QUAD_STRIP);

        for (int i =0; i < _numberOfSegments; i++)

        {

            if (i == 0) {

                glColor3f(color[1].r,color[1].g,color[1].b);

                glVertex2f(innerPosition[0].x, innerPosition[0].y);

                glVertex2f(outerPosition[0].x, outerPosition[0].y);

                glVertex2f(innerPosition[1].x, innerPosition[1].y);

                glVertex2f(outerPosition[1].x, outerPosition[1].y);

            }

            else if(i == _numberOfSegments-1)

            {

                glColor3f(color[_numberOfSegments-1].r,color[_numberOfSegments-1].g,color[_numberOfSegments-1].b);

                glVertex2f(innerPosition[_numberOfSegments-1].x, innerPosition[_numberOfSegments-1].y);

                glVertex2f(outerPosition[_numberOfSegments-1].x, outerPosition[_numberOfSegments-1].y);

                glVertex2f(innerPosition[0].x, innerPosition[0].y);

                glVertex2f(outerPosition[0].x, outerPosition[0].y);

            }

            else

            {

                glColor3f(color[i].r,color[i].g,color[i].b);

                glVertex2f(innerPosition[i-1].x, innerPosition[i-1].y);

                glVertex2f(outerPosition[i-1].x, outerPosition[i-1].y);

                glVertex2f(innerPosition[i].x, innerPosition[i].y);

                glVertex2f(outerPosition[i].x, outerPosition[i].y);

            }

        }

        glEnd();

        ofPopStyle();

}

And here is the resulted ring.
Screen Shot 2014-10-15 at 10.35.02

Colour Ring

LEDs

The final process was to transfer the colour data from the image abstraction to the LEDs. To do this we used a Fadecandy unit and the ofxOPC library (which is documented in this post).

For those who are unfamiliar with the Fadecandy, it is a small usb powered LED Driver which can output a total of 512 channels across 8 output pins (64 channels of data from a single pin). As our selected Neopixel rings were 16px long, we realistically could have had four rings on a single pin. However, in the interest of safety we decided to split the rings in to three rows of three. This meant that if one row had any fault or corruption of data the other rings would be unaffected. It was also extremely beneficial when it came to mounting and soldering the rings together.
Hackthetate

Wiring Diagram for LEDs

In the interest of time and saving our sanity. We generated a class that would handle the image processing, url requests, LED abstraction and visualise the data. The colorStoryRing class meant that we could setup and initialise all the elements with one command.

To setup the LEDs we needed to ensure they were drawn with enough spacing that the other rings would not corrupt the colour data.

// In ofApp.h

colorStoryRing ColorRings[9];



// In ofApp.cpp

//--------------------------------------------------------------

void ofApp::setupRings()

{

    for (int i = 0; i < 9; i++) 

    {

        // Setup first row    

        if (i <= 2) 

        {             

            ColorRings[i].setupRing(ofPoint(70+(i*100),100),40,20,16);         

        }

        // Setup second row         

        if ((i >= 3)&&(i <= 5))         

        {             

            ColorRings[i].setupRing(ofPoint(70+((i-3)*100),200),40,20,16);         

        }         

        // Setup third row

        if ((i >= 6)&&(i <= 9))

        {

            ColorRings[i].setupRing(ofPoint(70+((i-6)*100),300),40,20,16);

        }

    }

}

Screen Shot 2014-10-14 at 12.09.37

ofxOPC element Spacing

For this project we specifically added extra features to the ofxOPC library that allowed us to chain multiple arrays of colour data together. To write the data to the LEDs check if the opcClient is connected to the Fadecandy then pull the data from the colorStoryRings class.

if (opcClient.isConnected()) 

{        

    opcClient.writeChannelOne(ColorRings[0].neoReturnValue(), ColorRings[1].neoReturnValue(), ColorRings[2].neoReturnValue());

    opcClient.writeChannelTwo(ColorRings[3].neoReturnValue(), ColorRings[4].neoReturnValue(), ColorRings[5].neoReturnValue());

    opcClient.writeChannelThree(ColorRings[6].neoReturnValue(), ColorRings[7].neoReturnValue(), ColorRings[8].neoReturnValue());

}



for (int i = 0; i < 9; i++) 

{

    ColorRings[i].update();

}

And here is the result.
2014-06-14 15.56.27

Abstracted Images next to Real Images

HackTheSpace

LEDs mounted