Digital and Analogue Theatre

I thought now would be a good time to reflect on where the Hauntology project is. Tim Wright, my long time collaborator on Hauntology, Adam Hoyle, co-founder of DoTank Studios, Neal White, my colleague at Bournemouth University and Director of the Office of Experiments, and myself, have recently been working together on a bid for the NESTA Digital R&D fund. We've just had our first knockback, with some interesting feedback, and are now looking at reworking and resubmitting for the next deadline.

The project is centred on the haunted box - the wooden casket which knows where it is and will recite poetry to you if you take it to the right place. Inside the box is a pile of smarts: a GPS sensor, a microphone, a speaker, an SD card full of poetry recordings; hence you can script the journey of the box - when you carry it to the right location, it will read to you, or invite you to record your own words and voice. This is all done using off the shelf components and open source hardware - Arduino, relatively cheap sensors, simple code. I wrote about some of the early stages of The Haunter project on BU's research blog.

Switch wiring diagram

The aim is that these smarts give us a way into the literature and landscape of Hardy country: we've used the Poems of 1912-1913 collection, the poems he wrote following the death of his first wife Emma. You take the box for a walk out of Dorchester, through the fields towards Stinsford, and finish at the graveyard where she is buried - and indeed where Hardy's heart was also interred many years later.

Therein lies a tale: Hardy's body was buried in Poet's Corner at Westminster Abbey, while his heart was returned to his home. While awaiting the funeral, the heart was placed in a casket on the mantelpiece, and legend has it that a hungry cat swallowed the heart before the earth had its chance: the only possible compromise was to put the cat in the casket and bury it too.

Haunted box - Hardy Haunting

The challenge of the project is not in the hardware or software - that's very achievable, and the increasing variety of IoT phenomena out there, whether it's the latest BERG Cloud platform for interconnecting objects with networks, or some of our early inspirations such as Arduniana's Reverse Geocache or Tim's BlakeWalks, makes creating smart objects, which extend media experiences into lots of news spaces, much more easy.

Given that we're adapting 100-years-old poetry by one of the geat writers of English literature, the challenge also does not lie in generating content as such. The 1912-13 collection is full of haunting poems, and the more you learn about the fraught relationship Hardy had with Emma before her death, the more the anguish and nostalgia of the poet works it way out of the lines and into your skin: "Woman much missed, how you call to me, call to me, Saying that now you are not as you were..."


The challenge is in between these two different layers, the smart tools and the mournful poetry: how do you create theatre? The box must be haunted, and mysteriously come to life at just the right moment. The promenade theatre of the walk has to seamlessly work with the labour of carrying the box, and the weight of the poems themselves.

A question arises when we describe the project to people: why a box? why not a phone?

I want to say that the box is the opposite of augmented reality. AR: you pull out your phone, with its shiny screen, its smooth bevels, its obvious 21st-centuryness and you hold it up to the scenery; imposed over the reproduced video of the background scene, you see in the foreground a floating icon, a 3D shape, some text. The peripheral scene recedes, as you are pulled both into the present moment, and the mediated image on the screen. Your vision is absorbed, and the voices of those around you become muted as their presence all but disappears. Your phenomenal experience is fused in the phone; you could be anywhere; you've been plucked out of the place your body has walked to, and are suddenly nowhere. You're in a bubble, a little cybersphere in which you've been engineered into absence.

The box should do the opposite of this. You are with a group of people, with whom you are starting to have unusual experiences. The box has spoken a poem in which the regret of a widower for neglecting his hermit wife has leaked into your own thoughts. The poem has ended and you all walk on, over a stile and into a field. The box is passed from person to person, each murmurs something like - 'do you want to have a go?' Someone goes off on their own with the box and returns saying they have recorded an anonymous thought. You wonder if Emma and Thomas ever walked this route together. You are walking this route with someone, several people, now.

The Box Speaks: Hardy-haunting

If you've been to theatre in the round, you may be familiar with this strange intense and anxious communal feeling: you eye each other uneasily as the actors perform in amongst you - you're unsure whether you're in the scene or invisible. Your only option is to trust the performance, in the same way that you trust in a providential author when you're lost in a book, or trust in a filmmaker that the journey is worth it and the final scene won't consist of pictures of cans of beans instead of the tense finale of the story.

You have to trust each other, yes, but even more you have to trust the box - that it knows what it is doing: you have to believe it is haunted, in the same way that you believe that Superman can fly or Gandalf can cast spells. You must believe it might have been on that mantelpiece, and that microscopic traces of Hardy's fibrous muscular heart might still be found on the inside, and that if you concentrate really hard, you might very well be able to tune into the analogue impressions left by people's hands, their voices, the waves that hit the atoms of the wood and dislodge them forever in an imprint that could be reconstrued and remade into the original sound like the self-destroying grooves of an antique wax cylinder.

Posted by joe

17 May, 2013

theatre, augmented-reality, internet-of-things, story-telling.

... being one drop of water in the great lake ...

"Three weeks ago, when Varya had read the Tsar's manifesto stuck to an advertising pilar on Vasilevsky Island where she lived, she took a tram across the Neva and there on St Isaac's Square saw the German Embassy being sacked by patriots. Everyone was wildly excited, as though what had arrived was not a war but their own long-awaited happiness.
"She felt frightened and lonely. Her life had not been a particularly full one, but at least she had had a sense of belonging, of being one drop of water in the great lake which was the life of Russia. Now it was as though the bottom of that lake had opened up and the water was draining away for ever, swirling and roaring as it went. Before it dried completely she had to hurry, hurry!"

Solzhenitsyn, August 1914, 1971

Posted by joe

07 May, 2013

great, Russia, war.

one day, when everything is over

one day, when everything is over

". . . one must visualize the catastrophe of World War I and what its outbreak and development meant to the cultural consciousness of European humanity The bourgeois society, spoiled by the long period of peace, had developed a belief in progress and a cultural optimism that came to characterize the liberal age. All of this collapsed in the storm of the war, which in the end was completely different from all those that had preceded it. The course of the war was not decided by personal courage or military genius but, rather, by the outcome of the competition between the heavy industries of all the different countries. The horror of matériel battles [Material-schlachten], in which innocent nature, fields and woods, villages and cities were devastated, in the end left those in the trenches and dug-outs with no room for any thought except "one day, when everything is over," as Carl Zuckmayer had expressed it then.

The extent of this insanity outstripped the youth's powers of comprehension. They had come to the struggle with an idealistic enthusiasm and a willingness to make sacrifices, but it soon became clear to the youth on all sides that the old forms of chivalrous - if often cruel and bloody - honor had lost their place. "What remained was a nonsensical and unreal event - one that was also founded on the unreality of the overheated nationalism that had in turn caused the workers' movement, the internationale, to explode. It was no wonder that the intellectual leaders of that time asked, "What has gone astray with our belief in science, with our belief that the world was being made a more humane place and that its safety was being insured by the increasing amount of regulation? What had gone astray with the presumed development of society towards progress and freedom?"

Hans-Georg Gadamer, Existentialism and the Philosophy of Existence, 1981

Posted by joe

24 January, 2013

world-war-one, hans-georg-gadamer, unreal.

The Haunter Field Day

From the NRG / CEMP Location-Based Storytelling Symposium pages:

Tuesday 19th June 2012


Both Joe and Tim are going to be contributing to the Location-Based Storytelling Symposium at Bournemouth University on Monday 18th June.
The next day – Tuesday 19th June – we’ll be going to Dorchester to do some work out in the field. We’d love people to join us.

Our intention is to:

  • Record as much Hardy material as possible out in the field – and capture ambient sound from the route of the walk, snippets of conversation, natural sounds etc.
  • Work with Proboscis to create a Haunter booklet (or perhaps more than one!?), and simultaneously contribute to the very exciting ongoing series of collaborative urban explorations, City As Material: -
  • walk and talk with people from various backgrounds and disciplines in order to understand how The Haunter project can be developed to appeal to the broadest range of user groups and local service providers.

People can walk with us, try out Joe’s beta version ‘haunted’ box, help us record audio, fill in booklet material or simply join the conversation about what we’re trying to do with this project.

It may be also useful for Joe and I to give a short talk at some point on the walk, and perhaps others would like to do the same. If you have some thoughts or a story you’d like to share with the group at some point during the day, do let Tim know in advance at, so we can plan it into the schedule.

Both Joe and Tim are going to be contributing to the Location-Based Storytelling Symposium at Bournemouth University on Monday 18th June.

The next day – Tuesday 19th June – we’ll be going to Dorchester to do some work out in the field. We’d love people to join us.


The Plan for the Day


View Where The Night Rooks Go in a larger map

You can join us at any point during the day. To track us down, follow us on Twitter at @moongolfer.

  • 10:30: Coffee at the Dorset County Museum
  • 11:00: Start the walk proper at the Hardy statue at the top of High W Street, following the Yellow Route. Joe & Tim to introduce the project.
  • 12:15: Arrive at Gray’s Bridge on the London Road. Here, we may split into two groups, one walking on the Red Route, the other on the Blue Route.
  • 13:30: Arrive at Stinsford Graveyard.
  • 13:45: Late lunch break – you’ll need to bring your own food & drink or we can repair to the Kingston Maurward Gardens, which has a cafe
  • 14:45: We’ll take the Black Route on the map via Lower Bockhampton, up to the Heath and on to Hardy’s Birthplace (sadly closed on Tuesdays).
  • 16:15: Back to Stinsford via the Green Route and on to Max Gate (also closed on Tuesdays) via the Blue Route. A chance for final audio recordings.
  • 17.45: Finish at the Thomas Hardy Pub: DT1 2AB. Disperse at leisure.

Posted by joe

31 May, 2012

field-day, haunter.

The Haunter - Tech Spec draft 1

The Haunter is a public art project that involves people being asked to carry a wooden box along a prescribed route from the centre of Dorchester out to the Hardy family plot in the graveyard at Stinsford.

The hand-crafted box will be packed full of smarts that allow it to speak, to release recitals of the famous Poems of 1912-13 at various points on the walk, and play out snippets of imagined conversation, music, monologues, extracts from novels, letters and biography.

The box will capture a record of each trip it takes, allowing every person who carries it to leave behind their own ʻhauntingʼ for others. Eventually an archive of Hardyʼs writings, recordings of poems, places and people, and images of the box in use, will be used to create interactive resources.


I've broken the architecture of the device into 3 areas: affordances, resources and outputs.

The affordances are 'inputs' which can effectively act as triggers for the different outputs. Outputs may draw on resources. E.g.: when a specific button is pressed in a specific place (2 affordances), a specific audio file (1 resource) will be played through a speaker (1 output).

The relationships between affordances, resources and outputs should be configurable - I'm calling this 'scriptability' - the ability to configure different combinations of affordances to trigger different combinations of outputs and resources, using a simple structured 'pluggable' script / mark-up language.

The resources and the mark-up should be 'pluggable' - that is, they should be easy to change by swapping different data in and out.



  • Swappable media
    • an SD card, up to 32GB
    • storage of pre-prepared audio files or new audio recordings
    • location of pre-prepared interaction mark-up script
    Use scenario: For The Haunter, we intend to use 21 poems, each of varying length; we'll pre-record them out in the field; we may also want to include in the pre-recorded audio library
    • Several extracts from Hardy novels
    • Music and historical stuff from the same period
    • Ambient natural sounds from around the walk from different times of year
    • Ambient natural sounds from Beeny Cliff and other Cornish sites mentioned in the poems
    The Haunter will also be used to capture audio clips while the box is in use, some of which may eventually be used in subsequent derivations of the project, such as iPad apps, online archives, etc. In case of limited space, these could in principle be transferred off the storage media in between each use of the box, but it would be better to be able to only have to transfer new recordings once a day - or even not at all?
  • Power
    • the power source for the device
    • lightweight but powerful enough to be used to run the electronic components, an incidental uses of servo motors, a speaker, LEDs, etc, for up to about 6 hours.
    Use scenario The Haunter box can be loaned out to groups of walkers, perhaps up to two or three times in a day, for walks lasting up to 3 hours. Ideally, it would not require recharging in between uses. Alternatively the battery could be easily changed between uses so that spent batteries can be recharged while the box is being used.


  • Geofencing
    • ability to detect location to adjustable levels of granularity / accuracy
    • waypoint recording with timings
    Use scenario: Audio needs to be played out in specific locations with an accuracy of, say 10-20m. In each location the system can randomly pick from a set of, say 3-4 audio files that are assigned to that location. In this way, there remains a certain variety in terms of what each set of walkers hears. In practice, it would be useful to be able to adjust the level of granularity required. Each walk can be reconstructed from the stored record of co-ordinates and timestamps, eventually allowing a composite visualisation / archive of the walks the box has been taken on.
  • Switches
    • up to 4 configurable switches
    • 2 toggle switches (non-locking SPST)
    • 1 (or 2?) touch sensitive sensors
    Use scenario: Two switches on the underside of the box are slightly proud of the casing. When the box is put on a flat surface, both buttons are held in for more than a second, allowing an audio clip to be played ("don't leave me here!”). The box is taken to a particular place and the user presses their finger on a touch sensitive sensor on the box, which triggers a sound, or the opening of a secret compartment.
  • Separability
    • connection to a duplicate device
    • ability to detect when other device is / is not present
    Use scenario: The Haunter box can be separated (by the unlocking of a hidden mechanism within through a servo, perhaps), and for the two halves to be taken on separate routes; and then united again at a location further on, such as the graveyard where Hardy is born. Each half of the box can contain a full version of the device with its own SD card of audio and scriptable interactions, which will only occur if the box separates. The different halves of the box must be able to detect the presence or absence of the other: perhaps each half has a male/female connector with small voltage across it allowing it to act as a switch, which is toggled on and off when the connectors are plugged in or removed.


  • Audio playback and recording
    • retrieval and playback of audio on demand
    • audio archived in mp3 or wav depending on capacity / access to codes etc
    • a lightweight but powerful speaker
    • on demand capture and storage of audio
    • a strategically placed microphone
    • geo-tagging of audio, either through XMP data or custom text/xml records
    Use scenario: The box plays pre-recorded audio at specific locations, or in response to specific manipulations of the box, such as pressing sensors or triggering buttons. At other locations, it records audio, which can then be made available as a playback resource in subsequent interactions.
  • Servo controller
    • ability to control a servo mechanism
    • ability to adjust the behaviour of each servo - i.e. turn an arbitrary number of degrees in either direction, or spin back and forth to power a vibration mechanism
    • up to three different servos
    Use scenarios: The box is taken to a specific place where the users are invited to record some audio; the box vibrates as a signal that recording is about to begin. The box is taken to a place where, if the correct button is pressed, a small locking mechanism inside the box is unlocked, allowing a secret compartment to open, or the box to separate.
  • LED controller
    • ability to turn on and off a light source
    • 2 different LEDs which can be toggled on and off / flashed
    Use scenario: The box has a pinhole view into a section of the box that lights up so that you can peer in and see an image inside the hollow compartment. Another LED sits in a sunken bevel in the box, and lights up when a audio clip is about to be played, and flashes when about to start recording, etc.
  • Camera relay
    • small built-in camera
    • ability to capture and store a still image
    Use scenario: A camera takes a picture when you are holding the box in the right place. A way of taking random snapshots or a view via fish eye wide lens. Images are then stored on the SD card. The images are used later in online archives, etc.


  • Interaction mark-up / scripting language
    • a mark-up language in which code blocks represent possible combinations of affordances, which, if all evaluate to true, trigger specific outputs / resources.

E.g. if done using an literal-like language, it might look something like this (first draft), where ʻaffordancesʼ are the conditions, and ʻactionsʼ are the commands to execute if the conditions are met:

codeblocks: [
	affordances: { 	// specifies criteria which must be met
		geofence: { 	// the geofence defines a geographical point and
// a radius within which the current position must be
lat: 50.7317, 
			lon: -1.8786,
			accuracy: 20m
		booleans: { 	// booleans inside 'affordances' test 
// for criteria which must be met

			bournemouth_event: 0
		switches: { 	// 4 switches, all of which are off:

			{state: 0, time: 0},
			{state: 0, time: 0},
			{state: 0, time: 0},
			{state: 0, time: 0}
	actions: { 		// specifies actions which are taken 
// if all affordance criteria are met
		playback: /mnt/mp3/bournemouth.mp3,
		booleans: { 	// booleans inside 'actions' set values 
// for other code-blocks to test for. 

			bournemouth_event: 1
	affordances: {
		geofence: {
			lat: 51.5171,
			lon: 0.1062,
			accuracy: 1000m
		switches: { 	// four switches, the 3rd of which is on 
// and has been for 2 seconds

			{state: 0, time: 0},
			{state: 0, time: 0},
			{state: 1, time: 2000},
			{state: 0, time: 0}
		booleans: {
			london_event: 0,
			bournemouth_event: 1
	actions: {
		playback: /mnt/mp3/london.mp3,
		servo: vibrate,
		record: 10000ms,
		booleans: {
			london_event: 1
	affordances: {		// true if no geodata present
					// for 5 mins or more
		geofence: null,
		time: 300000ms
	actions: {
		playback: /mnt/mp3/lost.mp3

The parser for this data structure would have a loop function which repeatedly iterates over each code block until it find one whose affordances are all true. When it finds one, it sets a lock, stopping any other interactions to be evaluated or executed, performs the behaviours defined in the 'actions' block, and then releases the lock, continuing the loop.

So in this case, if the box is within 20 metres of the pier in Bournemouth, an mp3 called 'bournemouth.mp3' will be played, and a boolean records that this codeblock has been executed. When the box is taken to central London, and a user presses and holds down button number 3, as long as the event in Bournemouth has already been triggered, an mp3 called london.mp3 will be played, and then a servo motor instructed to power a vibration mechanism, and finally a 10 second clip of audio will be recorded and stored.

In each case the boolean values record whether events have been triggered before, allowing unnecessary repeats to be avoided, or certain orders of events to be adhered to, etc.

The final codeblock in the example defines what happens when no geodata is available for more than 5 minutes; an mp3 called 'lost.mp3' is played.

The effect of this design to enable every combination of affordances, including previous device actions, to be 'addressable', and used as criteria for triggering subsequent outputs. The only limit to the complexity of the possible 'branches' of interactions is the capacity of the device.

The suggested design of the scripting language above is flat: it may be more economical to allow a nested structure - such as:

	// affordances to be met are listed
	// this code block is followed if affordances are met
		// nested affordances go here
		// nested actions go here
		// etc
	// this code block followed if affordances not met

  • A (non-exhaustive) list of desirable code functionality:
    • being able to detect the length in time of different states (e.g. having no geo-data, or being at a particular geofenced location for a certain amount of time, etc).
    • ability to play a random track, rather than a pre-defined one
    • ability to refer to an audio track recorded by an earlier codeblock



We anticipate developing a full-spec version (1.0 below) of the box for use in the summer of 2013; in the summer of 2012 weʼd like to work with a prototype version (0.1 below) to test the experience.

Version  0.1  (2012)


The minimum spec for this version consists of:

  • Swappable media
    • Storage of up to 32GB
  • Power
    • Enough capacity to power the device for up to 3 hours
  • Geofencing
    • Accuracy of up to 10-20m
  • Switches
    • 2 toggles, 1 touch sensor
  • Audio playback / recording
    • ability to play audio through a speaker; capture audio through a mic and store it for playback in the same session

Desirable features in version 0.1:

  • Servo controller
    • 1 servo controller which can be adjusted to control a small lock

Version  1.0  (2013)


The final version should also have the following capabilities:

  • Power
    • Enough capacity to power the device for up to 6 hours
  • Geofencing
    • Ability to store waypoint / timing record on the SD card for subsequent use
  • Servo controller
    • 3 servo controllers which can be adjusted to control two small locks and operate a vibration mechanism
  • LED controller
    • ability to control 2 LEDs
  • Camera relay
    • ability to control a small built in camera, and store captured images
  • Separability
    • ability to house 2 duplicated devices in the box and detect when each is present or absent

Posted by joe

28 May, 2012

the-haunter, technical, specifications.

The Haunter scoping draft


The Haunter is a poetry walking tour based on the 1912-1913 poems of Thomas Hardy. A specially designed, hand-crafted wooden box and accompanying booklet guide the user around a specific route taking in the outskirts of Dorchester and the countryside towards Stinsford.

At specific points the box can be triggered to play back pre-recorded readings of scripted narrative or Hardy poems, or to record audio snippets on location. The box is geo-aware such that specific interactions can be geo-fenced. It also has ergonomically unobtrusive buttons / toggles which can be optionally used to enable / disable / start / stop / or pause playback or recording.

Within the box there are two separable and self-contained devices with playback / recording functionality and independent power supplies; the box itself can be divided in two so that each section can be carried away in different directions. Each device can detect when the other is not present, and this in itself can be used as a controller for specific scripted interactions.

When not in use the box can be docked so that its power source can be re-charged, and any locally-recorded audio can be retrieved and synched with an online repository. The repository will be used to develop related artefacts ('transmedia') such as supporting website, phone / tablet apps, etc.

Mood / atmosphere

An exhibition of the concept was displayed at the NRG symposium on non-human narratives, alongside a collection of artefacts from the concept development process.

Prototype details

Audio playback

The Adafruit Arduino Wave Shield which has an onboard SD card slot. The WaveHC library can play back uncompressed .wav files stored on the FAT16 formatted card - though this library has read-only access to the card.

Audio Recording

I've found another library - WaverRP (untested) - which can apparently record and playback audio. In terms of sound quality, we are not aiming for high quality - as in the audio we have already captured on location, we expect to encounter wind noise, encode in mono, etc.

Wet mead by joeflintham

GPS receiver

The prototype uses the EM-406 GPS receiver, but if I were buying now I'd probably be looking at the LS20031. The NewSoftSerial and TinyGPS libraries can be used to read off data for subsequent geo-fencing.


Ideally, the device design will allow for a modular architecture. The component essentially parse a data structure stored on the SD card; this data structure describes the scripted behaviour of the box in a bespoke mark-up language.

A series of geo-fenced locations can be pre-defined, which can trigger playback of specific audio files or recording of audio at given locations; these interactions can be stored as boolean values, allowing some sort of basic non-linear scripting of subsequent sequences. e.g. in pseudo-code:

if geo-fenced location 1 has been visited
and geo-fenced recording action 2 has been completed
then trigger geo-fenced location playback 3

This architecture would in theory not only allow various combinations of conditional story-lines to be scripted, but for different geo-fenced locations to be specified, different audio files to be used, etc, without any changes to the device's firmware.


  • Essential:
    • Geo-fenced audio playback
  • Pretty much essential:
    • Geo-tagged audio recording
  • Desirable:
    • Two self-contained but separable units
    • Modular architecture allowing re-use in other locations / with other audio

Posted by joe

13 February, 2012

the-haunter, scope.

Building an audio device #3: prototypes, interactions and code

The box at its most simplest should be able to start, stop and pause audio tracks in a playlist. The code below does this based on a few simple interactions.

In this prototype, there are two switches, wired into the underside of the box. Put the box down on a surface and both buttons are toggled on. Lift it up and they are toggled off. These interactions fire the goToSleep() and waken() functions, which in this case play respective salutary and valedictory audio clips.

While the box is being carried, both buttons on the underside are in the off position. Pressing one of the buttons pauses or unpauses any audio clip that is currently playing (togglePauseButton(). Pressing the other button loads up and plays the next audio clip in the list (skipToNextFile()).

You can see the prototype in use during this little vignette of our trial run:



/* adapted from */

#include "WaveUtil.h"
#include "WaveHC.h"

SdReader card;							// This object holds the information for the card
FatVolume vol;							// This holds the information for the partition on the card
FatReader root;							// This holds the information for the filesystem on the card
FatReader f;							// This holds the information for the file we're play

WaveHC wave;							// This is the only wave (audio) object, 
										// since we will only play one at a time

void setup() {
  Serial.begin(9600);   				// set up serial port
  pinMode(2, OUTPUT);					// Set the output pins for the DAC control. 
  pinMode(3, OUTPUT);					// These pins are defined in the library
  pinMode(4, OUTPUT);
  pinMode(5, OUTPUT);
  pinMode(13, OUTPUT);					// pin13 LED

  if (!card.init()) {         
    while(1);                           // then 'halt' - do nothing!
  card.partialBlockRead(true);			// enable optimize read - some cards may timeout.
  										// Disable if you're having problems

  uint8_t part;							// Now we will look for a FAT partition!
  for (part = 0; part < 5; part++) {	// we have up to 5 slots to look in
    if (vol.init(card, part)) 
      break;							// we found one, lets bail
  if (part == 5) {						// if we ended up not finding one  :(
    sdErrorCheck();						// Something went wrong, lets print out why
    while(1);							// then 'halt' - do nothing!
  if (!root.openRoot(vol)) {			// Try to open the root directory
    									// Something went wrong,
    while(1);							// so 'hang' - do nothing!
										// Whew! We got past the tough parts.

/* device state flags */

int paused = 0;	 						// flag for whether we've activated a pause button
int awake = 0;							// flag for whether the device is awake

/* button state flags */

int buttonValue_1 = 0;					// value of button one's analogue input
int buttonState_1 = 0;					// 'on' or 'off' value of button one's state
int buttonValue_2 = 0;					// value of button two's analogue input
int buttonState_2 = 0;					// 'on' or 'off' value of button two's state

int bothButtonsOn = 0;					// default start value is all buttons off
int buttonOneOn = 0;					// these values help to track 
int buttonTwoOn = 0;					// the state of buttons over time 
int bothButtonsOff = 0;					// over time 

/* normal play mode for the device is to iterate over a predefined playlist */
const int NumberOfFiles = 9;			// number of files in the playlist 
char* fileList[NumberOfFiles] = {		// ordered list of filenames in the playlist 
  "F005.WAV",							// not only there does he see this sight, tim
  "F007.WAV",							// it's so easy to miss someone, tim
  "F009.WAV",							// his visitor, becka
  "F006.WAV",							// you always want to be somewhere else, tim
  "F012.WAV",							// lament, tim
  "F020.WAV",							// the haunter, becka
  "F024.WAV",							// we used to stand on the bridge, tim
  "F021.WAV", 							// the phantom horsewoman, becka
  "F001.WAV",							// a curl of hair, tim
int filePointer = -1; 					// reference to the file which should 
										// be played at any time
char* fileToPlay;						// reference to the name of file to play

/* device control functions */

void togglePauseButton() { 				// handles the pause button toggle

	if (paused < 1){					// state is 'unpaused'
		if (wave.isplaying){			// check if a wave is playing
			wave.pause();				// if it is, pause it
			paused = 1;					// store 'paused' state
	} else if (paused > 0){				// state is 'paused'
		wave.resume();					// resume the paused wave
		paused = 0;						// store 'unpaused' state

void skipToNextFile() {					// skips to next file in playlist

	filePointer++;						// increment the file reference
	if (filePointer == NumberOfFiles){	// reset if we exceed 
		filePointer = 0;				// the length of the playlist
	if (wave.isplaying) {				// already playing something, so stop it!
	fileToPlay = fileList[filePointer];	// pick the file to play
	playfile(fileToPlay);				// play it


void goToSleep() {
	if (awake) {						// if we're only now being sent to sleep,
		fileToPlay = "F027.WAV";		// pick a specific 'goodbye' sound
		playcomplete(fileToPlay);		// and play it
	awake = 0;

void waken() {

	if (!awake) {						// if we're only just waking up,
		fileToPlay = "F026.WAV";		// pick a specific 'greeting' sound
		playcomplete(fileToPlay);		// and play it
	awake = 1;


void loop() {

	/* capture input */
	buttonValue_1 = analogRead(0) / 4; 	// read analogue pin 0
	buttonValue_2 = analogRead(1) / 4; 	// read analogue pin 1
	if (buttonValue_1 > 128) { buttonState_1 = 1; } else { buttonState_1 = 0; } 
	if (buttonValue_2 > 128) { buttonState_2 = 1; } else { buttonState_2 = 0; } 
										// convert to boolean
	if (buttonState_1 > 0 && buttonState_2 > 0) { 	
										// both buttons pressed

		if (bothButtonsOn) {			// both buttons have been held in 
										// so device has been put down
			goToSleep();				// send to sleep
		} else { 						// first detection 
			/* - do nothing for now */
		bothButtonsOn 	= 1;			// remember button state for the next pass
		buttonOneOn 	= 0;			// all other buttons parameters are false
		buttonTwoOn 	= 0;		 
		bothButtonsOff 	= 0;
		delay(1000);					// wait a second

	} else if (buttonState_1 > 0) { 	// only button 1 pressed
		if (buttonOneOn) {				// button has been held in

			/* - we could add 'hold button one down' functionality here */
		} else {
			togglePauseButton();		// on first press, button 1 pauses / resumes

		bothButtonsOn 	= 0;		
		buttonOneOn 	= 1;			// remember button state for the next pass
		buttonTwoOn 	= 0;			// all other buttons parameters are false
		bothButtonsOff 	= 0;	

	} else if (buttonState_2 > 0){		// only button 2 pressed
		if (buttonTwoOn) {				// button has been held in
			/* - we could add 'hold button two down' functionality here */

		} else {
			skipToNextFile();			// button two skips to next track

		bothButtonsOn 	= 0;		
		buttonOneOn 	= 0;		
		buttonTwoOn 	= 1;			// remember button state for the next pass 
		bothButtonsOff 	= 0;			// all other buttons parameters are false

	} else {							// neither button pressed
		if (bothButtonsOff > 0){		// both buttons off for a second pass...
										// we have been lifted up!
			if (!awake){				// if we haven't already been woken up
				waken();				// wake up
		} else {						// we must be inside the first pass
										// of both buttons being switched 'off'

			delay(1000);				// await the second pass
		bothButtonsOn 	= 0;		
		buttonOneOn 	= 0;		
		buttonTwoOn 	= 0;		 
		bothButtonsOff 	= 1;	 		// remember button state for the next pass
										// all other buttons parameters are false


void playcomplete(char *name) {			// Plays a full file from beginning to end 
										// pause is disabled
  playfile(name);						// call our helper to find and play this name
  while (wave.isplaying) {
  										// do nothing while its playing
  										// now its done playing

void playfile(char *name) {
  if (wave.isplaying) {  				// see if the wave object is currently doing something
										// already playing something, so stop it!
    wave.stop(); 						// stop it
  if (!, name)) {  			// look in the root directory and open the file
										// do nothing, put an error message here
  if (!wave.create(f)) {   				// OK read the file and turn it into a wave object

  };	  						// ok time to play! start playback

int freeRam(void) 						// this handy function will return the number of bytes
										// currently free in RAM, great for debugging!   
  extern int  __bss_end; 
  extern int  *__brkval; 
  int free_memory; 
  if((int)__brkval == 0) {
    free_memory = ((int)&free_memory) - ((int)&__bss_end); 
  else {
    free_memory = ((int)&free_memory) - ((int)__brkval); 
  return free_memory; 

void sdErrorCheck(void)
  if (!card.errorCode()) return;		// no error; get on with it!
  while(1);								// there's an error with the card
  										// so 'hang'

Posted by joe

10 February, 2012

audio, video, playlist, prototype, playback, code.

Spirit Tracks - Warren Ellis

I interrupt this period of silence to point at a startling and beautful series of posts from Warren Ellis which are both book notes and talk cues hanging from a skeleton.

"It is a book about how everything is electric. It is also a book about cities, and a book about broadcasting, and a book about science fiction. And it is a book about ghosts."
"Electronic Voice Phenomena: the idea that the dead are speaking to us through radio, somewhere down deep in the medium wave, around 29 megacycles."
"Steven Shaviro, in his book CONNECTED, talks a bit about the Russian sf novel ROADSIDE PICNIC, saying that it, like all science fiction, actually exists to cast a shadow over the present ... He says of science fiction, "It shows us how profoundly haunted we are by what has not yet happened.”"
"It’s 1925. A ghosted image appears on John Logie Baird’s small, flickering test screen. A spectral, actually quite disturbing face grinning out through the swirling ether of the electronic beyond. Stooky Bill."
"Ghost hunters, like the people of this digital-cities conference I am currently giving this talk to, are very technical people. Back home, they roam the abandoned houses and haunted places of Britain with electromagnetic field readers, convinced that ghosts produce an electromagnetic field ... Of course, twenty years before they started doing that, William Burroughs was asserting in science fiction that the human soul is an electromagnetic field. Thereby haunting the future of haunting."
"UFOs leaking out of the earth. UFOs as stress imagery: the stress of living, the stress of the event, turning electromagnetic noise into pictures of something ghostly and alien. The future oozing up through cracks in the ground like a ghost from its grave."
"Take one street. A digital spirit track. Antenna ghosts whispering at us. All of us experiencing an ethereal presence in the street. Because our brains are being magnetically boiled by a constant blooming of experience inducing fields from everywhere."
These keep coming - go read them now: Spirit Tracks, Warren Ellis

Posted by joe

18 May, 2011

electricity, city, ghost, spirit, sci-fi.

The Haunter: A Story, Distributed

In the last few weeks I've had the chance to explore what I think The Haunter wants to be. I presented a paper at the Non-human Narratives symposium at BU two weeks ago, and exhibited some of the artefacts from The Haunter as a 'work/walk-in-progress'; and a few days before, I went haunt-walking around Hardy's homelands with my friend and colleague Jo. We used a draft booklet that Tim wrote for us, in which he guided us through Hardy's poetry, the landscape and the possible voices of the box; as he manoeuvred and ventriloquised us around the words and places from afar (he was in New York at the time) we read and spoke and walked and looked and recorded and thought.

You will see a man waiting by a bridge

Mind, body, world, and 'what a mind can be'.

At the symposium David Herman's talk, which focussed on the telling of animal stories in graphic novels, argued that to get closer to thinking about animal-perspectives, we need to get past a Cartesian notion of mind, and explore the Umwelt (Uexküll) of the living being, which is much more like a developing lifeworld engaged with the affordances (JJ Gibson) of the environment.

To do this we should eschew narrative perspectives and knowledge practices which are predicated on notions of 'inner' experiences and 'outer' appearances, as though there are clear boundaries between mind and world. It is precisely this dualistic enframing of human consciousness which has led us to think that we 'think' while the rest of the universe of animals and plants and minerals are automata - instinctual, algorithmic, viral, crystalline.

Instead, we should try to approach mind or consciousness as something that living - even non-living - things do (Noë) by virtue of their being in the world. This presents the challenge of our own escape attempts: to make the imaginative leaps necessary to see the world as much as possible in terms of what affordances and perceptual cues are relevant and meaningful to the lifeworlds of non-human things, rather than to anthropomorphise them. David argued that while some stories treat animals as vehicles for allegories of the human, there are stories at the other end of the spectrum, which try to dwell or be led by the imperatives of the non-human: to consider the affordances and cues which are meaningful to dogs, molluscs, doorknobs.

My mind is not a flickering, immaterial light encased inside impermeable glass, (dark or not); not a butterfly in a diving bell; not a ghost in a machine. My mind, like the minds of animals and trees and bricks and stars, is distributed between my sensations, my body, my environment, my 'directedness'. I have no grounds for regarding myself as having a greater 'directedness' (the technical philosophical meaning of intentionality) than a tree or a rock or the lizard under the rock by the tree.

Tree / sky

Machines, humans and agency

These thoughts chimed with my arguments in the paper I presented at the symposium, in which I drew on Andrew Pickering's work The Mangle of Practice to suggest that I have no more right to regard my apparent authorial intentions as regards writing to be of a different order to the intentions of my hands, the pens, the paper, the typewriter, the scissors, the words, or the imaginative live-borne world of the diegesis itself. We are all locked in a dance - maybe a struggle - of competing, conflicting, consonant, collaborating effort. Our agency is distributed amongst the affordances and possibilities of the lifeworlds we are all directed towards. I write with my machines, my chips, my pencils, pads and props - we are co-authors, no-authors.

What is the consequence of this wrecking, pulling down of the author to which I seem so often to return? Her death at the end of writing permits me to read her words as detached from her 'intentions' - this is the intentional fallacy, which we accept. However her ever-continuing death before she has written a word seems to command us to judge every interpretation as blind, serendipitous, lucky, emergent - unintended. Adrift in a galaxy of language we weave those words closest to us into a weakly cohering assemblage that we invite others to play in before it falls apart (Barthes). Capturing the flow of the world as it presents itself to us is always a re-presentation of the accidental, the found - the cut-up (Burroughs). Isn't every act of authorship, then, a composition of found things into a arrangement that we optimistically hope is intelligible? Must we also then persuade ourselves that we made it all by hand anew, rather than filtered the lovely ephemera from the strand-line into our own little puppet-show on the beach?

No sign

Distributed story

In every story there are actors and players, affordances and cues, experiences and meanings. The question is how did they get there? Were they placed there by an all-knowing author, the voice of providence, fabricated from words well-crafted and deliberated? Sometimes, yes, no doubt. But why not sometimes, make the players and actors out of field gates and antique boxes? Affordances and cues from poetry and code? Experiences and meanings from walking and views, people and memories, hardware and digital media?

Such a story rejects the narrative as something that comes fully formed from inside the mind of the writer, and instead chooses to see the narrative as something written in the assembling of a miscellany of things - human and non-human. Routes and instructions, history and words, wooden objects and immaterial sounds and images. Here is a scene, say: but in this passage we remove a sentence, and replace it with a place in the world where you must stand and look. And here: a reliance on words is replaced with a space - on paper, on a screen, in a venue, in the mind.

We can think of the mind as the homunculus inside the man, whose stories too are composed from within, and articulated only once their meanings are decided. But like JJ Gibson's conception of the mind as an ecological system, where cognition is situated and arises in the symbiosis of multiple agents and contextual affordances - so we can think of a story which is never fully composed, but always readjusting to the arrivals of newcomers - responding, reflexive, provocative, adaptive. With every new occasion, new relations form, the story changes - even the hundred-year-old story of a man of words and letters, neglectful of his wife, who discovers a renewed sense of yearning after her unexpected death.

Old grave


So maybe what I mean (!) is that The Haunter is a composition of lovely ephemera, found on the strand-line; a composition assembled into a 'minded' experience by every wanderer who adds to the scene their own hands and feet, head and body, memories and aches, thoughts and feelings, words and actions. Not only has the audience become a 'wanderer': they are now too various to label: walking haunting lovelorn speaking mourning reading holding looking writing listening composing recording obeying exploring remembering deciding hoping.

Most importantly though, they are in the story, and they will write it as it unfolds. They will materialise the meanings that haunt the menagerie of parts that make the story; and in turn they will haunt the work themselves, leaving behind new compositions, traces of agency and ghosts of their own directedness.

Posted by joe

10 May, 2011

narrative, distributed, umwelt, affordance, mind, perception, cartesian-dualism, body, situated-cognition, eco-psychology.

Surrounded by hidden ghosts

We ask ourselves, what is being haunted, and who is haunting it?

Literature is a combinatorial game that pursues possibilities implicit in its own material, independent of the personality of the poet, but it is a game that at a certain point is invested with an unexpected meaning, a meaning that is not patent on the linguistic plane on which we were working, but has slipped in from another level, activating something that on that second level is of great concern to the author or his society. The literature machine can perform all the permutations possible on a given material, but the poetic result will be the particular effect of one of these permutations on a man endowed with a consciousness and an unconscious, that is, an empirical and historical man. It will be the shock that occurs only if the writing machine is surrounded by the hidden ghosts of the individual and of his society.
Cybernetics and Ghosts, Italo Calvino, 1967

The writing machine haunts you with your own hidden ghosts.

Posted by joe

19 April, 2011

haunting, ghosts, machine, writing, calvino, italo.

Building an audio device #2: buttons, switches, logic, fingers

An audio playback device needs some switches. The circuit I've documented here so far can either be on or off, and switching it on is equivalent to a complete reset, or a hard reboot. We need to give the device some affordances - controls that allow the human hands grasping the device to provoke it into non-linear action, give a sense of some sort of response to the fingers pressing the haunted objects.

We want to add some buttons to an Arduino-based circuit. A 'button' circuit for a board is clearly documented with a nice step-through at the Arduino site, and my design is a variant of this most basic circuit, but allows for two parallel push-buttons, rather than just one.

The first design for the 2-toggle-button circuit worked up to a point:

Wiring diagram

Each of the 'pins' in the diagram is wired to a unique pin on the Arduino board; however, if you look carefully you'll see that both switches here complete a circuit for either pin: so both switches work, but it is not possible to distinguish between them programmatically from the Arduino.

So we go on to design two:

Switch wiring diagram

In this circuit, each switch only completes a circuit for one of the pins at at time, and therefore each pin is individually readable from code - I can now detect which button is pressed.

Here's the correct circuit soldered onto a strip-board:

Switch circuit

This circuit is wired to the Adafruit Wave Shield. You can see that signal wires (blue for button one and yellow for button two) are wired to, respectively, pins 7 & 11 on the shield. Red is wired to +5v, white to GND.

Wave shield wiring

The switches are basic, non-locking SPST (single pole, single throw) push-buttons - black = ND91Y, red = FH59P. This means that each switch is on only as long as the button is pressed in, and the button must be held in to be 'on' - release it and the switch is 'off'.


This actually gives us 4 button-states in all, plus scope to extend these four states by capturing the amount of time each one lasts - how long a button is pushed in. The four states are

  • 1) both buttons off,
  • 2) button one on,
  • 3) button two on and
  • 4) both buttons on.

The time a button is held in might also tell us something about how the hand is understanding the device, so we can play with:

  • a) holding button one in,
  • b) holding button two in, or
  • c) holding both buttons in,

with a default state of both buttons off (identical to state 1 listed above).

The wires going off to the left lead to the switches; the wires to the right return to the Adafruit Wave Shield, which is plugged into the Arduino board:

Corrected toggle circuit

Let's adapt the code from the basic Arduino 'Button' script so we can use two buttons instead of one. The original script has a button input on pin 7, and an LED output on pin 13 which is toggled on and off. We'll just add a button input on pin 11 and toggle the same LED output on and off from both button inputs:

 Turns on and off a light emitting diode(LED) connected to digital  
 pin 13, when pressing a pushbutton attached to pin 7 or to pin 11. 
 * Note: on most Arduinos there is already an LED on the board
 attached to pin 13.
 created 2005
 by DojoDave 
 modified 17 Jun 2009
 by Tom Igoe
 modfified 18 April 2010
 by Joe Flintham

// constants won't change. They're used here to 
// set pin numbers:
const int buttonPin_1 = 7;     // the number of the first pushbutton pin
const int buttonPin_2 = 11;     // the number of the second pushbutton pin
const int ledPin =  13;      // the number of the LED pin

// variables will change. 
// We'll set different variables for the state of both pins 
// to illustrate how we tell each button apart
// even though we go on to perform the same action
// upon activation of either button
int buttonState_1 = 0;         // variable for reading the first pushbutton status
int buttonState_2 = 0;         // variable for reading the second pushbutton status

void setup() {
  // initialize the LED pin as an output:
  pinMode(ledPin, OUTPUT);      
  // initialize each pushbutton pin as an input:
  pinMode(buttonPin_1, INPUT);     
  pinMode(buttonPin_2, INPUT);     

void loop(){
  // read the state of each pushbutton value:
  buttonState_1 = digitalRead(buttonPin_1);
  buttonState_2 = digitalRead(buttonPin_2);

  // check if either pushbutton is pressed.
  // if either is pressed it's corresponding 'buttonState' value will be 'HIGH'
  if (buttonState_1 == HIGH || buttonState_2 == HIGH) {  
    // turn LED on:    
    digitalWrite(ledPin, HIGH);  
  else {
    // turn LED off:
    digitalWrite(ledPin, LOW); 

In a future post I'll go on to codify all the different states that two buttons can be in at any one time, and how they will regulate the flow of the audio device; and also return to the subject of using WaveHC and the Adafruit Wave Shield to queue and play back audio.

In the meantime, I reflect on why an old antique box, haunted by spirits of the past would have red and black plastic buttons sticking out of it...


The box is picked up and put down. Its underside either rests permanently on a surface (put down) or is caressed by fumbling fingers (lifted up). Its workings should be largely unseen, and the box (at least, this box) should remain closed... ... It occurs to me that buttons that also acted as feet, if placed sensitively, could detect prolonged depression of both buttons (put down), while fingers touching the underside of the box might find such protuberances and discover their action serendipitously...

Posted by joe

19 April, 2011

logic, switch, diagram, wiring, design, circuit.

Theory, incidentally

Incidentally, recently I used the hauntology project as a device to work through some ideas about how theories are useful to practitioners. This was a first year undergraduate lecture aimed at media production students who might be somewhat bemused by the insistence of university lecturers that making a video, designing a website or writing a script might be augmented by engagement with theoretical ideas like political economy or postmodernism.

I know many people who make creative things with serious intent shy away from theory, but I maintain that thinking through ideas that can help you to reframe your approach to your work is profoundly useful.

In this lecture I used the ideas of liminality, diegesis and performativity to illustrate some of the challenges I think about in developing hauntology projects. I can't overstate how useful I find it to pick up these ideas, like a carpenter picks up a hammer, and bonk my work with them.

Posted by joe

15 April, 2011

theory, reflection, academia.

Building an audio device #1: arduino, waveshield, speaker, battery

I'm building a haunted box, for a project I'm working on with Tim Wright, called 'The Haunter'. The idea is to bring together poetry, walking and digital media. Tim has written more about the project, explaining some of the background about Hardy's heart.

So far I have an audio playback device:

Audio playback circuit

Let's go through it piece by piece. First up you can see a mylar speaker (VC89W):

Mylar speaker

I've gone for mylar over a paper speaker because we intend to take the device out in to the wilds. Although mylar isn't exactly unbreakable, it's pretty resilient. You can tell this because I dropped it a few yards outside the Maplin store I bought it in (see how one corner is bent out of shape). Next time I've got a baby strapped to my chest, I'll accept the offer of a bag for my handful of gizmos.

The speaker wires are soldered to an Adafruit Wave Shield, plugged into an Arduino board:

Arduino / Adafruit Waveshield

The Wave Shield is a kit which can play uncompressed (i.e. .wav) audio files from a (max 1GB, FAT16/FAT32 formatted) SD card. You can see the SD card in its sleeve on the top of the PCB. It even has a volume knob and a 3.5mm audio jack out if you prefer to use headphones. It requires quite a lot of soldering, but, hey, you get audio playback for $22.

The shield is plugged into an Arduino Duemilanove board - in this case I'm using one which came with the ARDX Arduino Experimentation Kit from Oomlout, which I acquired when the good folks from Howduino came down to Poole for a weekend of physical computing and hacking.

Then there's a battery housing case:

Battery housing

I dwell on this detail, simply because I've soldered a 2.5 power plug (HH62S) to the wires from a battery box (L90AN) which happens to have an on/off switch on it, so I don't have to install / remove the battery when I need to use the device or conserve energy.

That's the hardware. As for the software, well, the device is currently running one of the example scripts. The WaveHC library provides all the tools you need to access the SD card and read and play wav files. Included in the library download are examples, such as daphc.pde - a script which simply finds all the wav files on the SD card and plays them in turn, looping through all the files indefinitely. I reproduce it here in full. I'll return to some of the details of programming in a later post.

 * This example plays every .WAV file it finds on the SD card in a loop
#include "WaveHC.h"
#include "WaveUtil.h"

SdReader card;    // This object holds the information for the card
FatVolume vol;    // This holds the information for the partition on the card
FatReader root;   // This holds the information for the volumes root directory
WaveHC wave;      // This is the only wave (audio) object, since we will only play one at a time

uint8_t dirLevel; // indent level for file/dir names    (for prettyprinting)
dir_t dirBuf;     // buffer for directory reads

 * Define macro to put error messages in flash memory
#define error(msg) error_P(PSTR(msg))

// Function definitions (we define them here, but the code is below)
void play(FatReader &dir);

//////////////////////////////////// SETUP
void setup() 
  Serial.begin(9600);           // set up Serial library at 9600 bps for debugging
  putstring_nl("\nWave test!");  // say we woke up!
  putstring("Free RAM: ");       // This can help with debugging, running out of RAM is bad

  //  if (!card.init(true)) { //play with 4 MHz spi if 8MHz isn't working for you
  if (!card.init(true)) {         //play with 8 MHz spi (default faster!)  
    error("Card init. failed!");  // Something went wrong, lets print out why
  // enable optimize read - some cards may timeout. Disable if you're having problems
  // Now we will look for a FAT partition!
  uint8_t part;
  for (part = 0; part < 5; part++) {   // we have up to 5 slots to look in
    if (vol.init(card, part)) 
      break;                           // we found one, lets bail
  if (part == 5) {                     // if we ended up not finding one  :(
    error("No valid FAT partition!");  // Something went wrong, lets print out why
  // Lets tell the user about what we found
  putstring("Using partition ");
  Serial.print(part, DEC);
  putstring(", type is FAT");
  Serial.println(vol.fatType(),DEC);     // FAT16 or FAT32?
  // Try to open the root directory
  if (!root.openRoot(vol)) {
    error("Can't open root dir!");      // Something went wrong,
  // Whew! We got past the tough parts.
  putstring_nl("Files found (* = fragmented):");

  // Print out all of the files in all the directories. | LS_FLAG_FRAGMENTED);

//////////////////////////////////// LOOP
void loop() 

/////////////////////////////////// HELPERS
 * print error message and halt
void error_P(const char *str)
  PgmPrint("Error: ");
 * print error message and halt if SD I/O error, great for debugging!
void sdErrorCheck(void)
  if (!card.errorCode()) return;
  PgmPrint("\r\nSD I/O error: ");
  Serial.print(card.errorCode(), HEX);
  PgmPrint(", ");
  Serial.println(card.errorData(), HEX);
 * play recursively - possible stack overflow if subdirectories too nested
void play(FatReader &dir)
  FatReader file;
  while (dir.readDir(dirBuf) > 0) {    // Read every file in the directory one at a time
    // Skip it if not a subdirectory and not a .WAV file
    if (!DIR_IS_SUBDIR(dirBuf)
         && strncmp_P((char *)&[8], PSTR("WAV"), 3)) {

    Serial.println();            // clear out a new line
    for (uint8_t i = 0; i < dirLevel; i++) {
       Serial.print(' ');       // this is for prettyprinting, put spaces in front
    if (!, dirBuf)) {        // open the file in the directory
      error(" failed");          // something went wrong
    if (file.isDir()) {                   // check if we opened a new directory
      putstring("Subdir: ");
      dirLevel += 2;                      // add more spaces
      // play files in subdirectory
      play(file);                         // recursive!
      dirLevel -= 2;    
    else {
      // Aha! we found a file that isnt a directory
      putstring("Playing ");
      printEntryName(dirBuf);              // print it out
      if (!wave.create(file)) {            // Figure out, is it a WAV proper?
        putstring(" Not a valid WAV");     // ok skip it
      } else {
        Serial.println();                  // Hooray it IS a WAV proper!;                       // make some noise!
        uint8_t n = 0;
        while (wave.isplaying) {// playing occurs in interrupts, so we print dots in realtime
          if (!(++n % 32))Serial.println();
        sdErrorCheck();                    // everything OK?
        // if (wave.errors)Serial.println(wave.errors);     // wave decoding errors

I grabbed the archive of wav files from Adafruit's 'examples' page, in which a voice announces all digits from 0 to 9 plus a 'point', and wrote them to the SD - which, remember, must be no more than 1GB, and FAT16/FAT32 formatted. First time I bought a 1GB SD card, the poor guy in Maplin, uncomprehending, tried to sell me a better deal - more space for a lower pound per gig. I left him rueing the obstinance of pig-headed customers who are unable to see the idiocy of their narrow-minded purchases. Resist being upsold.

Here's the result:

The next steps are to: a) trigger audio playback through interactions (e.g. switches, sensors, etc), b) install the device in a pleasing old wooden box and c) colonise the SD card with haunting voice audio. Thanks for reading... jx

Posted by joe

15 April, 2011

audio, electronics, adafruit, waveshield, development.

Box sketches

Sketches of boxes and their possible affordances:

Open box

Open Box
Open Box

The open box: this box is triggered by the opening of the lid. Requires a flex sensor or photodetector, plus a controller / transmitter.

Old engraved box on Flickr
Old engraved box by joeflintham on Flickr

Closed box

Closed Box
Closed box

The closed box with switch: this box does not open - its sensor must be activated by some other mechanism - a toggle switch, perhaps. Requires a sensor, such as a switch or potentiometer, and a controller / transmitter.

Jewellery Box on Flickr
Jewellery Box by joeflintham on Flickr

Rotated box

Rotated Box
Rotated box

The orientation-aware box: manipulations of the box are detected via orientation. Requires an accelerometer, possibly augmented with a magnetometer (compass), and a controller / transmitter.

Glyph-lidded Box on Flickr
Glyph-lidded Box by joeflintham on Flickr

Audio playback box

Audio playback box
Audio box

The audio playback box: an audio output, either to speaker or to headphones, plays back pre-recorded sounds. Requires a local storage device and a controller - e.g. battery-powered Arduino board.

Jim finds an empty bottom drawer... on Flickr
Jim finds an empty bottom drawer... by joeflintham on Flickr

Mic'ed-up box

Mic'ed-up Box
Mic'ed box

The mic'ed-up box: a microphone records live audio stream - built-in mic for ambient / omni-directional recording, or a headset for close-mic recording. Requires audio stream processing, either recorded to a local storage device or wireless / radio transmission to nearby controller / computer.

When this light is on... on Flickr
When this light is on... by joeflintham on Flickr

GPS box


The GPS-aware box: a GPS sensor monitors the geo-location of the box, and either transmits this information in real-time to a nearby controller / computer, or records the change in position over time for later retrieval. Requires a GPS sensor and a controller / transmitter / local storage.

Globe on Flickr
Globe by joeflintham on Flickr

USB box


The USB box: a USB port provides a facility for external devices to read from or write to the box. Requires a battery-powered controller with USB port, plus local storage device for reading / storing data.

Box of electronics on Flickr
Box of electronics by joeflintham on Flickr

Plain box

Plain Box
Plain box

The plain box: it does nothing, but in relief against each other box, it forms the pattern for every other box.

Posted by joe

05 March, 2011

box, idea, interaction, sketch.


Memory-box: memories and boxes

Audio recording: Shoe-box - mp3 file, 01:53, 4.32MB

Posted by joe

23 August, 2010

recording, interview, shoe-box, audio.


Serendipity can't just be the happy accident in which all the players and actions turn out well; it can't just be the chance occurrence that is the most pleasing, distinct from all the other possible outcomes which were thankfully avoided. If you threw the box in the air to see where the contents land, it would be foolish to be disappointed that their chaotic scattering is not as aesthetically pleasing as you'd like, or that the random assemblages, unlucky breakages and strange dispersals are less satisfying than you'd hoped. The unfamiliar new configurations in which you find yourself newly involved are as they are: they bustle a life into the world; they invite you to enter in amongst them; but they will not necessarily wait for you to overcome your hesitance to find the meanings that haunt the things in the world as it is.

As I write these words, I hold the squat pen in my hand. It is disposable, plastic, blue. The paper is in a notebook, bound inside a hard-cover, a remaindered, mass-market line-end. I picked them up this morning when a need to write this came on at the end of a car-boot sale exploration. As I suddenly started hunting for paper and pen, I might well have had in my mind's eye some yellowed pad, a miraculously preserved antique moleskin, or some other imaginary canvas awaiting my discovery; maybe an antique pen, complete with wooden barrel and ink-pot, or a hard-leaded pencil, ready-sharpened and decorated with minuscule scenes of distant lands reflecting the long voyage it had taken to reach my hands. Holding out for such romantic finds might seem admirable; uncompromising; indicative of a self-esteem worthy of such fastidiousness: yet it might also have been fruitless, and these words would have fizzled out as their emanations through my mind ran down and vanished along the grooves of my brain.

I saw a little jewellery carton, small and square, felt-lined; likely it had long ago been home to a once-new necklace. Now it lay open on a paste table and held some cheap worn brooches, a chain interspersed with ancient fake pearls, some little plastic beads no longer connected to their cord, so old they were cracking, and an earring, its oval lobe dulled with age. I paused as I glanced at it and then looked up at the man on the stall. He was old, hair combed madly back, the quiff a flat shelf like a crested penguin, his brylcreem dried up in the hot morning sun; painstakingly giving the prices of the bric-a-brac as two elderly women went through his stock.

I walked on, wondering what I could be looking for. Eventually I came to a collection of old boxes, laid out neatly on a table, the proprietor cross-legged in a camping chair, reading a paper over his carefully trimmed beard, looking for all the world as though he and his stall had been transplanted, copy-perfect, from a dusty antique shop to a sunny field. "Yes, that's nice, isn't it?" he said as I examined a wooden box, the lid decorated with symmetrical patterns in the metal trim. "It has some character. You can have that for six-fifty." I'm not good at haggling. "Ah-" I continued to inspect the box, turning to its underside, then opening it, trying to coax the lock mechanism to move with my thumbnail. "I'll take six for it," he pursued. "I'll give you six-fifty," I replied.

As I continued the slow stroll among the aisles of the market, I kept thinking about the little jewellery box and its nondescript contents. I wondered what sequence of events had led to their being thrown together, those old bits of cheap feminine fetishes and their cardboard container with its soft padded lining. Perhaps, I thought, the old man with his rock-dry, dragged-back hair was clearing out the excess of paraphernalia that eventually clutters up a home, the rag-bag, willy-nilly accumulation of odds and ends, accreting in corners and drawers as their ageing human operators fail to dispose of anything that might have some sort of value or place in memory.

The box looked though it had been lifted straight from its home, exactly as it had long lain. How many years had it taken for those items to find their way to that resting place, rather than the next drawer along, or the second one down? Whose fingers had squeezed the beads from their snapped cord, as it must seem, with the hope that they'd some day be sewn together on a new band, to be worn again once more? This old man alone on his stall, laying out for perusal and sale the no longer needed knickknacks, ornaments and decorative leftovers: how had he come to the conclusion that it was now time to vacate this junk and see if it might bring him a few quid? Was he on his own now, I wondered, briefly reminded of the elderly 80-year-old who'd sold me his car a while ago, saying he didn't need it because he couldn't turn his head to look over his shoulder any more, and anyway, he was a widower now? Were these objects on his stall the flotsam he had finally managed to allow himself to part with, now he'd had time to mourn his wife? The more precious things, of course, he'd kept, so he'd have her most evocative belongings around him; but now - perhaps he was moving? or attempting a new start with a clear-out and a clean-up - he was ready to let go of the bits and bobs that she herself had allowed to sleep in forgotten corners before she herself finally slept for the last time.

But how ludicrous to race away with such fantastic ideas. She might well be at home now, the stall-holder's putative wife; not dead, but cleaning the spare room that for the first time in decades had in recent days been emptied of the gradual accumulations and built-up layers of junk that she and her old man had only now managed to gather up, sort through and dispose of. Perhaps they had grasped the nettle together; dusted off some of the shelves; occasionally sat on the bed or the chair, slowly filing through box and drawer. Perhaps they ask each other from time to time about the items they find, deciding what to keep, what to sell and what to throw away; and as they filter all the old strata of life, they might remember together as they stumble through the evidence and detritus - scenes from the past that are familiar, or scenes dusty with disuse, memories across the spectrum - warm, joyful, awkward, painful. Maybe regrets might leak into consciousness, or equally, satisfaction. Who could know, just from glancing at all these worlds on the paste tables and blankets, what memories they were caught up with, what stories they played in?

And all of these stalls, the rows of picnic tables, doubling as counters for the wares that their vendors had dug out from the recesses and crowded old rooms, attics and garages, the homes of people and of things - weren't they all being freed from their buried past, like disinterred grave goods, or unmoored from their hidden creeks and allowed to wash away into an ocean - flotillas of plastic, paper, wood and metal either bobbing away or sinking to the depths, detached from all that had given them meaning? Even now as they lay on their tablecloths and formica tops, they still bore the traces, the fingerprints, even the dust overloaded with sense and the last living remnants of the dead hands that had held them, lives that had carried them along, hearts that had felt and imbued these charmed objects with significance. This field was a mortuary for the paraphernalia that gave people's live structure, the gifts people exchanged, the tools they built with, utensils they prepared food with, the signs and images with which they decorated their homes, the idols before which they knelt - here was the whole sum of their material world, their heirlooms, their heritage: bequests from lives that were no more, a bazaar of surfaces hiding their own importance, a web of relations disappearing forever.

I realised I needed to buy that little box of irreparable and ageing jewellery and started to make my way back to the brylcreemed man and his stall. What would I say, I wondered, as I anticipated his - what? sadness at the box's departure? confusion at my unexpected and inscrutable interest in an old pile of trinkets? I imagined myself asking for the box, and everything in it, just as it is, and how I'd answer his puzzlement by explaining that it just seemed so pleasing, this box that had somehow come to be where it was, with the earrings and chains and brooches it held - it couldn't have been expected, it couldn't be created, or recreated, or staged, or anything other than what it was - some excess of life that had been washed up on this table, in this sunshine, another chain in the sequence, another step on its journey, another sentence, image, meaning, in its story.

"Hi... this box - how much for the box and all the things in it?"
"This box? I'd like to take this box and everything in it." He fingered his way through the broken beads and sad anomalies, checking for anything valuable.
"Fifty pee."
"Here... thanks..."
"Thank you."

Posted by joe

19 July, 2010

jewellery, objects, bazaar, car-boot, box.

Live hauntings #three

Sounds from the Second Salon showing of hauntology.

Posted by joe

08 October, 2009

haunted-story, audio, recording, installation.

Exhibit number 3: Second Salon

Wednesday was the launch night of Second Salon in which the hauntology project is currently being exhibited. This is the third time it has been displayed, following the two Screengrab09 shows in Brick Lane and Bournemouth. Each exhibition teaches me something new about this project, and the launch night taught me more about audience participation.


In the previous shows I created a small poster which explained that the exhibit invites you to interact with it: variously worded instructions along the lines of 'pick up the picture, explore the drawers, find the casket, leave a haunting'. Something about this troubled me though - not least the fact that by labelling the interactions, I was in some sense removing the possibility of surprise: if you are invited to pick up the picture, then you ought to expect something to happen when you do so. I wondered if this might anaesthetise some aspects of what should be a spooky ghost story. So I thought I'd use the 3 hours of the show just to test out some ideas about how to deal with the balance between instruction and exploration.

I initially set up the plinth and chest with no instructions at all, to see what people did with it. The plinth concealed the computer controlling the exhibit; the chest stood next to the plinth; some leads trailed from the underside of the chest and disappeared into the plinth; finally, a set of headphones rested on the plinth.

Hauntology and plinth

There is clearly a grammar about exhibitions: plinths as a rule are not a part of the artwork itself (though Banksy's Bristol show and even the various uses of the fourth plinth in Trafalgar Square play with that idea). And of course, headphones are not generally understood to be for looking at alone. So people had no hesitation to stand by the plinth, and try on the headset.

Another part of the lexicon of the gallery, however, is a kind of permission culture: you must be given permission to touch things, because the norm is to not touch (or in some galleries, speak, laugh, smile, look at each other, or enjoy anything at all). Some exhibits clearly embody an invitation - Olu Taiwo's camera installation in Avatars of Being (also in the Second Salon show) clearly invite the audience to enter the space of the work and see themselves move and twist in the infinite corridor of mirror space produced by the filming of the projection of the filming. But such embodied interaction does not require that the participant touch anything. I quickly found that unless they knew to do so because they had seen the work before, no-one was willing to pick up the picture, open the drawers, find the casket, leave their voice. Without permission to pick up the picture, people merely perplexedly listened to the ambient sounds of hauntology, but never triggered the narrative elements, since these do not start until the picture frame is physically removed from the top of the chest.

Shaun listens

I had however prepared a small poster with some notes which, rather than provide instructions, instead gave some ambiguous hints that the exhibit should be interacted with: 'Don the headpiece, touch, move, explore, haunt'. This was an effort to provide permission to touch and interact with the exhibit, while still keeping enough ambiguity that the consequences of specific affordances (such as the start of the narrative in response to picking up the picture frame) might still be surprising. The poster also had a picture of the chest with the top drawer open, showing the casket inside, demonstrating that there were things to find.

The drawer

I was a little surprised, though, to find that the words were still being interpreted in an unexpected way - though I know I should not be surprised by unexpected interpretations of ambiguity... The hints seemed to provoke people to open the drawers, and touch the chest and picture frame, and even the fairly obvious infra-red sensor fixed to the side of the chest. All of this is excellent: the permission problem seemed to be solved. However, the words 'touch, move' were interpreted fairly literally. Sometimes, people would understand the instruction 'move' to refer to themselves - they tried walking around the chest and plinth. Sometimes I think the combination of 'touch, move' was understood to mean 'touch and move the picture frame' - so that some people tried to slide the picture frame around the top of the chest - an action which I had not anticipated and so is not detected.

So two things seem to stand out from these observations: the first is not so much a problem as an inspiration provided directly by the audience which I would not have had myself; the second is the problem of ordering the interactions to ensure they are part of the story.

Shaun and casket

The first issue is actually a nice idea which arose on the night in conversation with Lena and Lizzie. The user's action of sliding the picture frame around the top of the chest is actually reminiscent of the way a ouija board is used - the glass is pushed around the board until the group finds the place it should sit. Perhaps rather than endlessly search for the correct but ambiguous wording to encourage interaction, I can just tweak the settings so that, as well as the lifting of the picture frame, simply moving it across the chest, ouija fashion, seance-like, will suffice. There are some technical difficulties here, which I'll perhaps go into another time; I also quite like the more fully engaged and embodied action of holding a picture frame in your hands, and turning it between your fingers, rather than simply sliding it across the chest. But it's something to investigate, anyway.

The second outstanding issue from observing the interaction - the ordering of interaction and participation - is a problem because although hauntology is designed to change in response to audience interaction, I think there need to be boundaries to the nature of those changes. There is a core narrative, albeit made ambiguous by both an element of randomness in order, and by the inherent openness of possible interpretation: who is Michael? who is speaking? who is the picture of? who is 'she'? etc. The casket records the input from the headset's microphone when it is opened, and those recordings are then randomly overlaid onto the core narrative in subsequent replayings, and so any imaginative response to the work is then incorporated into it. However, it is possible (and common in the circumstances I've described above, in which the core narrative is not triggered by the lifting of the picture frame) for the audience to record themselves without ever hearing the core narrative: hence they never get to enter the diegesis the piece invites them to contribute to.

While such recordings are certainly legitimate 'hauntings' - permanent and retrievable records of the unique times and places in which the chest, picture and casket have found themselves - they are not and cannot be legitimate responses to the ghost story. But there is a conundrum here which I can't really answer adequately: a recording of the kind in which the user leaves their odd questions - 'er, what, I'm being recorded..? arrgh' which are fairly frequent - may be left both by people who have not triggered the narrative, but also by people who have heard the ghost story too. I am making a conceptual distinction between the value of the two (the former are welcome, the latter are problematic) which is independent of the type, style, nature and quality of the recordings themselves. This is, strangely, quite like the Gettier problem.

Shaun recorded

Of course I can close the loophole - only record when the narrative has been triggered, prompting those who find the casket to continue looking for a way to make the light turn on - but that doesn't answer the philosophical question about what is a legitimate contribution to the diegesis of a participatory narrative. Of course, as the maker of the work, I am free to impose my vision on the logic of the piece. But as someone interested in the consequences of opening the process of authorship, I also need to find a good way of dealing with this dilemma: my selfish authorial pride in the sanctity of the diegesis, in conflict with the open-armed embrace of anarchic, carnivalesque participation.

Posted by joe

03 October, 2009

authorship, ambiguity, diegesis, surprise, gallery, participation, interaction, observation, exhibition.

Exhibit display #three

Poster display from 3rd hauntology showing at Second Salon, KUBE Gallery, 30 September 2009.

Posted by joe

30 September, 2009

poster-display, pdf, haunted-story, installation.

Ghosts in the machine

With the launch of Second Salon approaching, I dusted off the chest, the chips and the code; unwrapped and unmothballed the hauntology bric-a-brac ready for exhibition. Putting the intricate wires and controllers and sub-routines back into place ... almost impossible, and once achieved - without effect. Nothing would work.

What gremlins had found their way into the innards of the bits and pieces! Had some entropy creeped in, as if it were an instrument gone gradually out of tune? The flex sensor was inverted, its values sky-rocketed beyond the thresholds in the code. The LED, switched, came on when it should go off, and blinked out when I expected it to remain lit. The wires - were they switched? The code, been tampered with?

What has haunted these objects since the last time they creaked into place, wound into the machine assemblage, and flickered into diegetic existence? I am collaborating not only with ghosts but with random acts of agency by inanimate materials.

Posted by joe

28 September, 2009


Second Salon announced

Kube LogoSecond Salon Invitiation

The Second Salon exhibition at the KUBE Gallery Poole has been announced - this is the show in which I'll be exhibiting hauntology work alongside a number of other artists.

"The second show at the gallery by local informal art collective The Salon, featuring works by 11 artists from across the South of England exploring traditional and new media. Exhibition by: Stephen Bell, Jeannie Driver + Mike Blackman, Joe Flintham, Peter Hardie, Poor Photographer, Mark Shufflebottom, Lizzy Sykes + Cathy Seago, Olu Taiwo & Sarah Thompson. The Second Salon show brings together both finished and works in progress, and open-to-the-public events include physical computing workshops, performances, talks and audience involvement. "

Read more

Posted by joe

09 September, 2009

second-salon, exhibition.

Embedded audio hacking

Today I took receipt of two Adafruit wave shields - kits which let your Arduino read and play audio from an SD card.

These kits will be used in the Second Salon exhibition which starts later this month in which I'm showing the hauntology work. I'm planning to rig them up to sensors which will detect movement in the gallery and play sounds in response.

In particular I'm looking at using several sensors dispersed through the building, which when triggered together will fire a set of sounds through the building's tannoy and into the ears of not only the gallery visitors, but the people getting on with their working lives elsewhere in the building.

More details and code here as the piece is constructed.

Posted by joe

03 September, 2009

architecture, arduino, audio, physical-computing.

Intellectual development #1

Part of a series about trying to make sense of #hauntology

I was trying to think about how to make interactive narrative immersive and engaging. I had recently led a 6-part lecture series on narratives for media students, and so was suffused with diegesis, sjuzet, structure, framing, metonymy; oozing Midas-like drips of narrative thinking on everything I touched.

I also had many discussions with my colleague Jim Pope, whose doctoral work had explored the 'vernacular' response to non-linear narrative and hypertext, and showed that a common response to what academics and critics might acclaim as continuing a modern/postmodern development in literature was actually bemusement and alienation. The 'common reader' found hypertext narrative hard to engage with, uninvolving and dissatisfying. Jim recently wrote about this work at interjunction and I implore you to read his hard-earned thoughts instead of my groundless, buffeted words.

In this context I found myself thinking that one of dominant topics in my teaching area, (what we easily call 'interactive narrative' without worrying too much about what that might really mean), might actually be an oxymoron: that a narrative is something provided by a narrator - a voice of providence, a hidden logic, an overarching guide, to wit, an author; and that interactivity is the effacement of the author, the rejection of a providential God who guides and oversees, and an acknowledgement of the tangible, workable, universally masterable, demotic, participatory, revolutionary, anarchic interception of the narratee.

The listener speaks, the spectator mounts the stage, the viewer is seen, the user creates, the reader writes, the audience become the makers, the recipients participate, the public revolt. These role-changes may have been prefigured as the death of the author in the sense of interpretive freedom, but not in the sense that the author must concede even their original skill: the conception of the story and the control of the plot. The writerly text made manifest is not simply that numinous text which is brought to life in every act of reading: it is much more - it is patricide and regicide, it is author reduced to facilitator, god reduced to resource manager, maker reduced to supplier.

In short, interactive narratives are like palpable obscures and darknesses visible: conceptual possibilities designed to confound us by their actual impossibility. To experience a story is to be a fish caught in the net: to tell a story is to weave the webbing for the catch. They are opposites, and there is no overlap, only repulsion.

Posted by joe

16 June, 2009

haunted-story, meta, narrative, interactivity, oxymoron.

Live hauntings #two

Sounds from second showing at Screengrab09, Bournemouth Media School.

Posted by joe

07 June, 2009

recording, audio, haunted-story, installation.

Exhibit display #two

Poster display from 2nd hauntology showing at Screengrab09, Bournemouth Media School, 5 June 2009.

Posted by joe

06 June, 2009

installation, haunted-story, pdf, poster-display.