The Art of Noise
Hackers have always been attracted by audio, fascinated on the one hand by the intricacies of compositional structure and on a more granular level, obsessed with how sound itself is created.
Computers create an ideal platform for both areas of investigation, skilled as they are in dissecting such structural qualities. It's hardly a coincidence that Johann Sebastian Bach's work features so heavily in the work of pioneers such as Hofstadter. And ever since the Italian Futurists in the mid twentieth century embraced the speed and violence of new technologies, making sound with machines has exerted a fascination on all manner of artists. When computer hardware and languages became sufficiently advanced in the early 60s to suit the modest needs of audio, parallel developments in audio theory and computer science spawned a new and still relatively youthful field of artistic research, which now with the growth of free software does seem to be undergoing a renaissance.
Given that most audio or compositional processing is less computationally demanding than, say, work with video, this field does have a longer, if not richer, history. It's common knowledge that as the intelligent computer HAL slowly dies in Kubrick's 2001, the machine regresses through the history of its ilk, to sadly sing "Daisy, Daisy, give me your answer please," the first song ever sung by a computer. Coded by pioneer Max Matthews at Bell Labs, coincidentally the birthplace of UNIX, "Daisy Bell" was generated using MUSIC IV, a Fortran-based music programming language. Music IV was obviously only one iteration of the flexible language MUSIC-N. As Matthews states, "Computer performance of music was born in 1957 when an IBM 704 in NYC played a 17 second composition on the MUSIC I program which I wrote."
MUSIC-N sowed the seeds for open sourced languages such as Csound or environments such as SuperCollider, which are still heavily used and developed by artists today. What's surprising is that a flexible, extendible language-based approach to the creation of sound has been around since the very birth of digital audio. And it's possible to argue that such a methodology, a way of working with art and code which stresses the creation of new structures and environments, is inextricably tied up with the philosophy and development model of free software.
Artists and computer mavericks shared ideas, code and ways of working across diverse institutions from Bell, MIT (Massachusetts Institute of Technology, another landmark within the history of free computing), and Stanford in the USA, to the legendary IRCAM (Institut de Recherche et Coordination Acoustic/Music) in Paris. But even as early as the mid 1970s, with the release of funky micros such as Commodore's KIM, artists and developers could embrace a less centralised way of working, untied from institutions; a development model which stressed community and openness. Indeed, the less demanding nature of audio computation has clear benefits, as with the rise of the low cost PC, pretty much all but the most ambitious audio works could easy be realised on the humble, home PC or live performance-oriented laptop. And such untethered environments are the perfect spawning ground for an open source approach.
Artistic digital audio is a youthful field in which the most exciting work in recent years has embraced a free software model for experimentation, production and distribution. It's an alternative mediascape in which artists have no need of proprietary models, favouring community and experimentation over commercial concerns. It's all about creativity rather than a workaday tool-based approach.
Nearly all the apps we'll investigate either push some artistic or audio boundaries, or present novel ways of looking at how to work with code and artistic material, questioning how we and machines interact and co-create. In common with most academically researched material, nearly all seriously experimental and extendible audio apps and environments have been released with source code, if not strictly under a free software license. The big two more traditional music lab apps, Csound and CLM (Common Lisp Music) are good examples of audio research tools which can easily be located with source on huge academic repositories such as Planet CCRMA (pronounced karma). Planet CCRMA, collated by Stanford University Center for Computer Research in Music and Acoustics where pioneer Max Matthews now resides, provides a great opportunity to witness the breadth and range of readily available open source audio apps.
The repository, which more or less amounts to a complete GNU/Linux multimedia distro, replicates the research and production environment used by artist-developers at CCRMA on a daily basis. And Agnula, a tailor-made audio distribution funded by the European Commission, is also well worth checking out. There can be little doubt that free software artists have no end of apps and environments to play with. And though newbies may complain that free software is troublesome to install and configure, Darwin and the excellent Fink project have led to a burgeoning community, as OS X users join the free software fold. The benefits are clear and well worth repeating.
Free software stresses flexibility, independence and community providing future-proof, reusable and extendible artistic solutions for an impatient, fast-moving audio scene. Free software is all about throwing away the rulebook; after all computer science and digital audio should have no constraints at such a youthful age. In recent years, as the scene has snowballed, ad-hoc groupings of artists based around specific languages, environments or approaches have quickly formed. Grass roots organisations such as Bek in Norway, V2_labs in Holland and Bootlab in Berlin have solidified such groups, providing resources and support for artists and developers. And although Pd (Pure Data) and SuperCollider have probably the widest followings, plenty of artists are coding their own apps in a range of languages.
Free software audio artists share a healthy range of geek interests and a passion for playing live, pushing the envelope well past a lone guy with laptop scenario to embrace all manner of noise producing devices and computational machinery. With ap's computer controlled record decks and neural synths, Mumbai Streaming Attack's recent TRAMJAM sequencing samples using 32 trams in Vienna, and coding poetry slams in Denmark, the free software audio scene obviously has little in common with the dull Cubase clones and tired software synths produced by proprietary models. The history of computer audio is all about free innovation.
Hacker-artist's favourite environment, Pd, really does have roots stretching far back into the history of computational audio creativity. Based around the Max, and later Max/MSP, extendible visual scripting environment for audio creation, and authored by the very same Miller Puckette, Pd takes up where the proprietary Max/MSP left off, thanks to an incredibly lively developer community and free software model.
For users unfamiliar with the Max/MSP or Pd environments, this visual model can easily be described as allowing the interconnection of various boxes which can pass both data and control messages. As you'd imagine from its very name, Pure Data, the flexibility of Pd lies in treating audio solely as data which can be manipulated, mixed and generally thrown around with a huge number of maths, control and generator objects able to change data pathways, structures and events. With such a flattening of audio as simply raw data, the possibilities are endless, with additional extensions also able to manipulate data as graphical content or even through advanced neural net architectures.
Collections of connected boxes are called patches, and these can quite easily hide further sub-patches which simply pass data in both directions to the main patch. Patches are composed of connections between object boxes, messages, GUI boxes such as VU meters or sliders and comment boxes for documentation purposes. It's a reasonably hierarchical approach but most artist's do adapt Pd to suit their own methodologies, either through extensions which can, say, add language scripting possibilities, or simply through the abstractions provided by sub-patching.
Go forth and multiply
The easiest way to understand Pd is quite simply to get up and running, and begin to play with the huge range of objects available. Under most GNU/Linux distros, Pd is trivial to install, with few dependencies and uncomplicated configuration. It may well be worth messing with the low latency possibilities in recent kernels, or patched earlier kernels, and Pd does play well with ALSA and JACK, but a base setup is easy to achieve with readily available walk-through tutorials. Pd is suitably platform agnostic, also running under both Windows and OS X. Documentation is intelligently embedded within the Pd package, and commented Pd patches are easily accessible from within Pd.
Examples range from simple oscillators through more complex control and audio patches, to advanced Fast Fourier Transform (FFT) examples, converting data from the time domain to the frequency domain for further analysis and manipulation. Reference patches are also included which document every single object within the base Pd package from sine and cosine generation objects, through control, network and piping objects, to MIDI and sampling objects. Simple patches may use only a small range of these, but it is easily possible as we'll see to create seriously spidery patches which pack in hardcore networked and control functionality.
And with ad-hoc groupings of Pd-heads sharing ideas and code, extensions have grown up around Pd which allow less technical users to more simply coerce Pd towards their own particular artistic vision. Tom Schouten and Yves Degoyon, occasionally working together live as BULT, are prime movers on the Pd coding scene, between them producing both graphical extensions for Pd, and a huge number of intriguing externals.
On the audio side these include Icecast and SHOUTcast externals, vocoders, compressors and some seriously complex math and DSP objects. Schouten has also recently released the highly adventurous packet forth (pf) package, which can be used both with and without Pd and Schouten's PDP multimedia extension. Pf throws together the Forth language, Pd, Lisp, Unix scripting and hardcore DSP and it could easily be considered as a multimedia glue language which can be used with a variety of interfaces. Schouten has recently prototyped an inferior pf interpreter process under GNU Emacs, which also opens up some serious live coding possibilities.
At first glance the popular SuperCollider environment does appear to offer a radically different approach to audio work than Pd, though a good few Pd-heads, such as Farmers Manual, do divide time between them and both apps do share common roots and an environmental coding approach to audio. SuperCollider, or SC, can easily be viewed as a textual code-based environment, in contrast to Pd's obviously graphical model. Both approaches have advantages and disadvantages, particularly when it comes to live work, but it's easy to see that Pd and SC could quite easily compliment each other.
Indeed, SuperCollider can quite simply be interfaced with Pd, to provide the best of both worlds. Pd appeals to the artist who wants throw together, to jam with, disparate data sources and control, maybe interfacing complex hardware, whereas SC is perhaps more geared towards generating intricate compositional structures.
SuperCollider has a long history centering primarily on the work of one James McCartney, previous to its now open sourced SC3 iteration. It's interesting to see our old friend Max pop up once more, with SC1 morphing out of McCartney's Max object Pyrite, though SC itself is more about MSP style DSP work than pure Max control. SC also owes a good deal to the even more venerable MUSIC-N apps. It was MUSIC III which introduced the concept of a unit generator, a subroutine that would create a specific kind of sound, which is so central to SC.
The program consists of two applications, a client which is the language itself (sclang) and a server, scsynth, which handles the DSP work and sound synthesis. With a highly flexible programming language engine, and supremely networkable architecture, again making use of OSC (see below), it's not too tough to see how far you could push SC. Sclang has morphed considerably through various versions, borrowing differing features, syntax and models from languages such as Smalltalk, Scheme, C and even J along the way, but the core of math operators, oscillators, noise generators, filters, controls, delays, samples, event spawning and I/O wrapped up in an extreme object oriented model with messages and classes remains as the SC approach.
Unit generators, such as diverse objects which process or generate sound, are an essential part of SC, and the controln generator unit class, for example, allows for external control sources, graphical sliders or programmatic processes to control parameters. Examples would include MouseX and MouseY classes. Plentiful online tutorials, wikis, documentation and a fresh, active community are on hand to provide examples and support, though at first SC may well seem like a tough cookie to crack, with a code-based approach seemingly at odds with most artists' methodology.
Getting up and running with SC under GNU/Linux may also present some problems for the less experienced user, though quite thorough documentation is available. JACK and ALSA are absolute necessities, and low latency audio is preferable. With these in place, SC3 can easily be checked out from CVS and compiled. Somewhat lengthy configuration of elements such as JACK inputs is necessary before scsynth is up and running, and sclang itself does need a startup file for class and environment variables, but all this is well documented. Online material is also available for those wishing to extend SC with plugins, and the intriguing architecture of SC should make of this an interesting exercise for the artist-hacker.
Coding as performance
Given that SuperCollider presents a heavily interpreted language, which also makes use of OSC to send messages to the core sound server, live coding in sclang is a real possibility. Live coding pushes performance barriers, exposing audiences to changing, raw code and providing a much needed live virtuosic element for the hacker-artist. After all, given the brittle qualities of code, live programming is a dangerous proposition. Performing without a safety net is always exciting. And what better performance platform than the flexible GNU Emacs itself.
SCEL (SuperCollider Emacs Lisp) provides a complete mode for working with SC on GNU/Linux. As you'd expect, there's syntax colourisation and decent indentation, but also an all important interpreter interface. Alternatively OSC messages can also be sent to the server app from the command line, using sendOSC, or from clients such as RSC, a pre-release Scheme SuperCollider client. SC has recently established a culture of live coding, modifying active algorithms for live audio.
Indeed, an international organisation TOPLAP (Live Algorithm Programming and a Temporary Organisation for its Promotion), has recently been set up by several artists involved with both self-coded apps and SC. Their manifesto, the Lubeck04, calls for the exposure of code itself, and a more fun, performative aspect to live, computer generated audio. Some big names from the world of free sound back up these claims, with Alex McLean, formerly of Slub, representing the Perl and command line approach and both Nick Collins, working as Sick Lincoln, and Julian Rohrhuber pushing forward on the SC side.
Julian also provides the intriguing Just In Time Programming-Library (JITLib), which comes free with the standard SC distribution.
Yet another approach to live coding, which perhaps places less emphasis on the algorithmic and stresses the machine virtuosic is provided by ap's ap02 and gdapp apps, both used together in live performance. Gdapp, or generic data app, maps out coded nodal structures which can just as easily be recoded by humans or machines on the fly.
Nodes, which manipulate raw data of any kind obtained from a range of interfaces or the filesystem itself, also contain microcode which can be re-coded and executed live on the gdapp virtual machine layer. And recently, ap have extended notions of environmental code and an open source development model into the realm of hardware. Their ap0201 project was installed in the California desert in April of this year, and features ap02 virtual machine software embedded within wirelessly networked, solar powered machines.
Ap0201 devices run 24/7, converting audio data input from an extruded microphone into code running on the virtual machine. Code is executed within a complete computational instruction set, which can also access other ap0201 nodes or display chosen material on a small LCD panel. According to ap (Martin Howse and Jonathan Kemp), it's all about embedding code within and as environment.
Storm in a teacup
From the often messy patches of Pd to the supreme interoperability of SuperCollider, in part thanks to OSC, it's all about throwing it all together; the good old UNIX philosophy of absolute connectivity between small apps and elements which can elegantly be recombined towards a greater whole. JACK and OSC should both be given adequate credit as enabling open source apps, through providing an infrastructure which allows software to communicate and present differing, custom interfaces or ways of working.
OSC is particularly interesting in that it does enable communication across so many apps such as Csound and Pd, as well as being easily implemented within other languages. The networked protocol which defines OSC is well documented, and allows for a good deal of flexibility in sending messages. Equally interesting is the audio equivalent of ascii video, low level work with audio which exploits core Unix features such as FIFOs (named pipes for data), or uses the versatile netcat tool for networked audio. Extensions exist to pull such data into Pd, opening up new worlds of interconnect and interoperation between shell and apps.
Without even touching on issues of free distribution, it's already easy to appreciate the richness of the open audio community. Few GNU/Linux users will be unfamiliar with the dyne:bolic live CD, which packs in a wealth of audio apps requiring little if any setup. Dyne:bolic is very much focused on net radio uses and streaming efforts, with the powerful MuSE app as cornerstone of any free audio studio.
Alongside repositories such as Planet CCRMA, dyne:bolic is all about easing the learning curve for artists and developers new to the free software scene. This is an essential task, which both new and seasoned artist-hackers alike can easily contribute to, with blogs and shared patches or code. And although we may have examined the work of artist-hackers across a rather artificial divide of audio and video work, it is necessary in attempting to assess the differing histories and technological demands for future users. Video is perhaps more difficult to work with both due to technical constraints and perhaps a less physical response.
Thomas Grill, Viennese performer and composer, has contributed some ground breaking work to the Pd community, providing for full integration of the Py_thon scripting language with Pd, and also with his flext plugin, unifying Max/MSP and Pd externals. His dyn~ external creates some intense possibilities for self-coded, generative work, allowing for the dynamic creation and connection of standard PD objects and abstractions. Austria definitely does seem the place to be if you're interested in Pd performances or workshops.
Farmers manual, well known electronic sound artists based in Vienna, are also active on the Pd scene, producing both performance and code, including externals integrating Pd with OSC (OpenSound Control). Artists such as Erich Berger have also taken part in chaotic Pd Stammtisch, or conventions, in the Austrian city of Graz. And more recently, Mumbai Streaming Attack made fantastic use of the timings inherent within the Viennese tram network, to produce a masterful electronic jam concert orchestrated by a huge Pd patch produced by Marc Widmer amongst others. TRAMJAM-Vienna RushHour created a highly networked streamed performance using the live tram schedule (through interfaced tram door openers) of 32 tramlines, re-mixing and re-broadcasting live electronics from 32 tramjammers. And all this acoustic mayhem was provided, courtesy of the supremely flexible, highly playful, free Pd.
Though most artists we've explored have worked with some success in integrating visual material with live audio, few have examined such a novel approach to their parallel development as has Erich Berger in his recent Tempest piece with GEM (a graphical extension to Pd). Tempest is inspired by security work based on decoding or recoding electromagnetic radiation produced by a good deal of electronic equipment. Here's how it works. A computer monitor emits such waves at very high frequencies which can easily be picked up by a short wave AM radio. These waves can easily be decoded to recreate the screen image, or inversely screen images can be created to generate certain tones. The latter quality was first investigated by Pekka Riikonen and Erik Thiele, with his GPLed Tempest For Eliza, which cleverly broadcasts Beethoven's piece Elise. Berger takes the concept one step further, cleanly wrapping up audio and visuals with a neat, effortless solution which has no need of window dressing such as beat matching. It's a wonderful example both of openly sharing ideas and technologies to arrive at novel approaches, as well as demonstrating how far the free software world is from the banality of proprietary approaches to audio.
Music And Sound Software