Tweeting the Digital Humanities with Arduino (and Raspberry Pi!)

Over Spring Break, I had an idea for a digital humanities art installation. Picture this: an obtuse, old-fashioned metal box covered in dials and knobs sits inside a glass case. A spool of receipt paper slowly unwinds from the box, covered in dots and dashes. These markings are Morse code, representing real-time, authentic tweets straight from Twitter. Over time, the printed paper grows larger and larger, slowly filling up the case.


The Concept

I wanted to find a compelling way to give weight and dimension to the data that’s being constantly shared over Twitter. Twitter connects a tremendous number of people instantly, and those people are all sharing short, digestible chunks of information. But because it all takes place virtually, we rarely consider the staggering amount of data that’s being generated minute by minute. By putting tweets to paper in real time, the data is placed in a physical space, where it will literally “pile up.” As time goes on, the paper will eventually begin to fill up the case, obscuring the device itself. In doing so, I hope to convey the idea that we are overwhelmed by the ‘connectedness’ of our everyday lives. It becomes impossible to escape the flood of data we constantly generate. For this reason, I intend to have the device seek out tweets containing the phrase “big data”. Big data is typically used to describe the challenge of managing enormous datasets (generated by sites like Twitter), but it is increasingly used to refer to how the wealth of data we have about consumers can be harnessed to provide technological goods and services. For example, my Android smartphone figured out which dorm I live in just by recording data from my phone’s GPS. My location data alone was enough to tell Google where I sleep at night. The fact is, every time someone logs into Facebook, buys something on Amazon, or does a search on Google, they signal some kind of data about themselves. And finding ways to harness that data yields a myriad of new ways to reach out to consumers.

Although the idea of big data tends to make people rather uncomfortable (and justifiably so), I still happen to think it’s pretty cool. As a computer scientist, I’m fascinated by how people share information, and how technology increasingly facilitates our ability to do so. For this reason, I also want the piece to convey a certain degree of anachronism. I see Twitter as being analogous to a modern, massive telegram service. Even century ago, a telegram was certainly the best way to send someone a message quickly. And just like a tweet, a telegram necessarily had to be short, often requiring special abbreviations or creative punctuation. By putting the tweets in Morse code, I wanted to get at the idea that Twitter is just a modern tool to fill the same role as a telegraph. And by putting it inside of an antique housing, I wanted to remind people how far we’ve come in our ability to share information in a short period of time.

Additionally, because the device tracks tweets in real time, it could easily work as an interactive piece. By tweeting anything with the phrase or hashtag “big data”, viewers could see their message printed to Morse code almost immediately.


The Plan

The device will center around the Arduino platform, a versatile microcontroller commonly used for homebrew electronics projects. There are a wide variety of peripherals (sensors, motors, LCD screens, etc.) that can be programmed to work with Arduino, making it possible to build nearly any kind of device you could want. For this project, I’ll be using a thermal printer (the same kind that are commonly used to make receipts) and a ethernet shield to enable connectivity to Twitter. There is a thriving open-source community behind Arduino, so code libraries and example projects are readily available. That will reduce a lot of the trial-and-error from the coding process, because I can look to other examples of how to retrieve tweets and implement a thermal printer.

Practically speaking, the device will work by repeatedly checking an RSS feed for tweets containing the phrase “big data”. When a new tweet is found, it will be read, converted to Morse code, and printed. This is a substantially simpler solution to my initial plan for implementation, which would have required setting up a twitter bot to constantly monitor and retweet all other “big data” tweets. And although it was fun learning how to create a Twitter bot, I found out I could circumvent the need for one by easily creating an rss feed for a custom Twitter search.

I also struggled at first with deciding what kind of housing to use. I considered constructing a simple wooden box, but I have no knowledge of carpentry. I’ve since decided on using an hold ham radio, similar to the one pictured below.

Ham radios are readily available for cheap on eBay, and come in a variety of sizes. It should be easy to open one up and hollow it out, although it would be nice if it were possible to keep some of the meters working just for show. Finding a way to cut neat, precise holes in the housing for cables and paper output is still a challenge I have not worked out a solution for.

Another challenge is powering the device and keeping it connected to the internet. The thermal printer and Ethernet shield will draw a lot of power, so using a batter pack is not an option. And a wireless shield is far too expensive, so the device will also require a dedicated Ethernet cable. This means the device will have to be constantly plugged into a wall outlet as well as an Ethernet port, which will limit the portability of the device and may prevent it from being displayed prominently.

Finding a Name

I’ve been considering names for the device, and I think I’ve settled on Gramograph. It directly relates to the idea of a telegraph, but with a novel meaninglessness. After all, semantically speaking a gramograph would mean something like a ‘message-writer’. And that’s exactly the sort of diffuse communication service Twitter provides. But as an art installation, I’d like to call the piece Why Don’t You Write, You Rascals? The phrase was the first ever message transmitted over a commercial telegraph, and I think it captures a common sentiment against Twitter and social media in general. People are often worried that the prevalence of social media in our modern day lives comes at the expense of authentic, meaningful communication. It’s extremely easy to send a tweet, but that ease means that a tweet can never be as meaningful as a letter or even a handshake. And perhaps the fact that social media is designed to make communication effortless discourages thoughtful discourse. In any case, social media is here to stay, and will continue to shift the way we communicate and form relationships. Consider that a major aspect of the younger generation’s social sphere occurs exclusively online- now that is some very big data indeed. Through this piece, I hope to give viewers a small glimpse of the massive amount of data we all provide about ourselves on a daily basis, and one example of how that data can be harnessed.


Update as of 4/26/2013

Twitter recently changed its authentication process to a service called Oauth in an effort to increase security. Unfortunately, this makes the process of receiving tweets from an embedded device much more complex. And while there is a Twitter library for Arduino that manages Oauth, frustratingly it only supports posting Tweets, not retrieving them. This has proven to be a significant obstacle for development, however I believe I have finally found a solution (albeit albeit a less elegant one than I had initially hoped). There is an open-source Java library available called Twitter4j, which manages Oauth easily and even supports retrieving Twitter searches. Unfortunately, it was not designed to run on an Arduino, and making it do so would be well beyond my expertise as well as the scope of this project. But it will run on a different (but similar) microcontroller calledRaspberry Pi using an environment called Processing (which, happily, is closely related to Arduino). My plan is to have Raspberry Pi do all of the interfacing with Twitter. It will use Twitter4j to gain Oauth access and retrieve a file containing raw html search results for #BigData. The Raspberry Pi will send this file directly to the Arduino via the hardwired serial monitor. The Arduino will parse the feed to get the desired content (namely, the tweets themselves), convert them to Morse code, and print them via the thermal printer. This solution is more expensive and adds another degree of complexity, but it is highly feasible.