The Haunter - Tech Spec draft 1
The Haunter is a public art project that involves people being asked to carry a wooden box along a prescribed route from the centre of Dorchester out to the Hardy family plot in the graveyard at Stinsford.
The hand-crafted box will be packed full of smarts that allow it to speak, to release recitals of the famous Poems of 1912-13 at various points on the walk, and play out snippets of imagined conversation, music, monologues, extracts from novels, letters and biography.
The box will capture a record of each trip it takes, allowing every person who carries it to leave behind their own ʻhauntingʼ for others. Eventually an archive of Hardyʼs writings, recordings of poems, places and people, and images of the box in use, will be used to create interactive resources.
Specifications
I've broken the architecture of the device into 3 areas: affordances, resources and outputs.
The affordances are 'inputs' which can effectively act as triggers for the different outputs. Outputs may draw on resources. E.g.: when a specific button is pressed in a specific place (2 affordances), a specific audio file (1 resource) will be played through a speaker (1 output).
The relationships between affordances, resources and outputs should be configurable - I'm calling this 'scriptability' - the ability to configure different combinations of affordances to trigger different combinations of outputs and resources, using a simple structured 'pluggable' script / mark-up language.
The resources and the mark-up should be 'pluggable' - that is, they should be easy to change by swapping different data in and out.
Resources
- Swappable media
- an SD card, up to 32GB
- storage of pre-prepared audio files or new audio recordings
- location of pre-prepared interaction mark-up script
Use scenario: For The Haunter, we intend to use 21 poems, each of varying length; we'll pre-record them out in the field; we may also want to include in the pre-recorded audio library
- Several extracts from Hardy novels
- Music and historical stuff from the same period
- Ambient natural sounds from around the walk from different times of year
- Ambient natural sounds from Beeny Cliff and other Cornish sites mentioned in the poems
- Power
- the power source for the device
- lightweight but powerful enough to be used to run the electronic components, an incidental uses of servo motors, a speaker, LEDs, etc, for up to about 6 hours.
Use scenario The Haunter box can be loaned out to groups of walkers, perhaps up to two or three times in a day, for walks lasting up to 3 hours. Ideally, it would not require recharging in between uses. Alternatively the battery could be easily changed between uses so that spent batteries can be recharged while the box is being used.
Affordances
- Geofencing
- ability to detect location to adjustable levels of granularity / accuracy
- waypoint recording with timings
Use scenario: Audio needs to be played out in specific locations with an accuracy of, say 10-20m. In each location the system can randomly pick from a set of, say 3-4 audio files that are assigned to that location. In this way, there remains a certain variety in terms of what each set of walkers hears. In practice, it would be useful to be able to adjust the level of granularity required. Each walk can be reconstructed from the stored record of co-ordinates and timestamps, eventually allowing a composite visualisation / archive of the walks the box has been taken on.
- Switches
- up to 4 configurable switches
- 2 toggle switches (non-locking SPST)
- 1 (or 2?) touch sensitive sensors
Use scenario: Two switches on the underside of the box are slightly proud of the casing. When the box is put on a flat surface, both buttons are held in for more than a second, allowing an audio clip to be played ("don't leave me here!”). The box is taken to a particular place and the user presses their finger on a touch sensitive sensor on the box, which triggers a sound, or the opening of a secret compartment.
- Separability
- connection to a duplicate device
- ability to detect when other device is / is not present
Use scenario: The Haunter box can be separated (by the unlocking of a hidden mechanism within through a servo, perhaps), and for the two halves to be taken on separate routes; and then united again at a location further on, such as the graveyard where Hardy is born. Each half of the box can contain a full version of the device with its own SD card of audio and scriptable interactions, which will only occur if the box separates. The different halves of the box must be able to detect the presence or absence of the other: perhaps each half has a male/female connector with small voltage across it allowing it to act as a switch, which is toggled on and off when the connectors are plugged in or removed.
Outputs
- Audio playback and recording
- retrieval and playback of audio on demand
- audio archived in mp3 or wav depending on capacity / access to codes etc
- a lightweight but powerful speaker
- on demand capture and storage of audio
- a strategically placed microphone
- geo-tagging of audio, either through XMP data or custom text/xml records
Use scenario: The box plays pre-recorded audio at specific locations, or in response to specific manipulations of the box, such as pressing sensors or triggering buttons. At other locations, it records audio, which can then be made available as a playback resource in subsequent interactions.
- Servo controller
- ability to control a servo mechanism
- ability to adjust the behaviour of each servo - i.e. turn an arbitrary number of degrees in either direction, or spin back and forth to power a vibration mechanism
- up to three different servos
Use scenarios: The box is taken to a specific place where the users are invited to record some audio; the box vibrates as a signal that recording is about to begin. The box is taken to a place where, if the correct button is pressed, a small locking mechanism inside the box is unlocked, allowing a secret compartment to open, or the box to separate.
- LED controller
- ability to turn on and off a light source
- 2 different LEDs which can be toggled on and off / flashed
Use scenario: The box has a pinhole view into a section of the box that lights up so that you can peer in and see an image inside the hollow compartment. Another LED sits in a sunken bevel in the box, and lights up when a audio clip is about to be played, and flashes when about to start recording, etc.
- Camera relay
- small built-in camera
- ability to capture and store a still image
Use scenario: A camera takes a picture when you are holding the box in the right place. A way of taking random snapshots or a view via fish eye wide lens. Images are then stored on the SD card. The images are used later in online archives, etc.
Scriptability
- Interaction mark-up / scripting language
- a mark-up language in which code blocks represent possible combinations of affordances, which, if all evaluate to true, trigger specific outputs / resources.
E.g. if done using an literal-like language, it might look something like this (first draft), where ʻaffordancesʼ are the conditions, and ʻactionsʼ are the commands to execute if the conditions are met:
codeblocks: [
{
affordances: { // specifies criteria which must be met
geofence: { // the geofence defines a geographical point and
// a radius within which the current position must be
lat: 50.7317,
lon: -1.8786,
accuracy: 20m
},
booleans: { // booleans inside 'affordances' test
// for criteria which must be met
bournemouth_event: 0
}
switches: { // 4 switches, all of which are off:
[
{state: 0, time: 0},
{state: 0, time: 0},
{state: 0, time: 0},
{state: 0, time: 0}
]
}
}
actions: { // specifies actions which are taken
// if all affordance criteria are met
playback: /mnt/mp3/bournemouth.mp3,
booleans: { // booleans inside 'actions' set values
// for other code-blocks to test for.
bournemouth_event: 1
}
}
},
{
affordances: {
geofence: {
lat: 51.5171,
lon: 0.1062,
accuracy: 1000m
},
switches: { // four switches, the 3rd of which is on
// and has been for 2 seconds
[
{state: 0, time: 0},
{state: 0, time: 0},
{state: 1, time: 2000},
{state: 0, time: 0}
]
},
booleans: {
london_event: 0,
bournemouth_event: 1
}
},
actions: {
playback: /mnt/mp3/london.mp3,
servo: vibrate,
record: 10000ms,
booleans: {
london_event: 1
}
}
},
{
affordances: { // true if no geodata present
// for 5 mins or more
geofence: null,
time: 300000ms
},
actions: {
playback: /mnt/mp3/lost.mp3
}
}
]
The parser for this data structure would have a loop function which repeatedly iterates over each code block until it find one whose affordances are all true. When it finds one, it sets a lock, stopping any other interactions to be evaluated or executed, performs the behaviours defined in the 'actions' block, and then releases the lock, continuing the loop.
So in this case, if the box is within 20 metres of the pier in Bournemouth, an mp3 called 'bournemouth.mp3' will be played, and a boolean records that this codeblock has been executed. When the box is taken to central London, and a user presses and holds down button number 3, as long as the event in Bournemouth has already been triggered, an mp3 called london.mp3 will be played, and then a servo motor instructed to power a vibration mechanism, and finally a 10 second clip of audio will be recorded and stored.
In each case the boolean values record whether events have been triggered before, allowing unnecessary repeats to be avoided, or certain orders of events to be adhered to, etc.
The final codeblock in the example defines what happens when no geodata is available for more than 5 minutes; an mp3 called 'lost.mp3' is played.
The effect of this design to enable every combination of affordances, including previous device actions, to be 'addressable', and used as criteria for triggering subsequent outputs. The only limit to the complexity of the possible 'branches' of interactions is the capacity of the device.
The suggested design of the scripting language above is flat: it may be more economical to allow a nested structure - such as:
case:
{
// affordances to be met are listed
},
true:
{
// this code block is followed if affordances are met
case:
{
// nested affordances go here
},
true:
{
// nested actions go here
},
false:
{
// etc
}
},
false:
{
// this code block followed if affordances not met
}
- A (non-exhaustive) list of desirable code functionality:
- being able to detect the length in time of different states (e.g. having no geo-data, or being at a particular geofenced location for a certain amount of time, etc).
- ability to play a random track, rather than a pre-defined one
- ability to refer to an audio track recorded by an earlier codeblock
Versions
We anticipate developing a full-spec version (1.0 below) of the box for use in the summer of 2013; in the summer of 2012 weʼd like to work with a prototype version (0.1 below) to test the experience.
Version 0.1 (2012)
The minimum spec for this version consists of:
- Swappable media
- Storage of up to 32GB
- Power
- Enough capacity to power the device for up to 3 hours
- Geofencing
- Accuracy of up to 10-20m
- Switches
- 2 toggles, 1 touch sensor
- Audio playback / recording
- ability to play audio through a speaker; capture audio through a mic and store it for playback in the same session
Desirable features in version 0.1:
- Servo controller
- 1 servo controller which can be adjusted to control a small lock
Version 1.0 (2013)
The final version should also have the following capabilities:
- Power
- Enough capacity to power the device for up to 6 hours
- Geofencing
- Ability to store waypoint / timing record on the SD card for subsequent use
- Servo controller
- 3 servo controllers which can be adjusted to control two small locks and operate a vibration mechanism
- LED controller
- ability to control 2 LEDs
- Camera relay
- ability to control a small built in camera, and store captured images
- Separability
- ability to house 2 duplicated devices in the box and detect when each is present or absent