Pathable : Event App Technology Blog

Data Analysis of Twitter at Events

I was puzzled. All the blog posts, webinars and tweets I was reading told me that Twitter and live events went together like peanut butter and jelly. I knew, in fact, that Twitter’s “big break” came at SXSW 2007, the annual Woodstock-Sundance of the high-tech / indie music / film worlds.

So how come when I actually read the Twitter history from various events, it seemed like mostly hiya chatter? In the Twitter echo chamber, this can sound like a raucous din, but in the cold light of day, are a couple dozen tweets from a handful of people really valuable?

I consider myself a Twitter skeptic and a data junkie. Don’t get me wrong: I use Twitter every day, I have three separate personae that I manage, and, in fact, I took a break to send a tweet while writing this sentence.

That said, I see a lot of questionable Twitterthusiasm out there, particularly as it relates to live events. I’ve attending more than my fair share of webinars that say they’ll discuss “5 Secrets for Using Social Media at Events” and read more than my fair share of blog posts purporting to describe the “innovative ways Twitter was used at X event”. Event managers (or technology vendors) will breathlesslly extol the value they saw in their wall-sized Twitter projections.

Sadly, so many of the “expert webinars” talk very abstractly about how valuable social media is, insisting that anyone who doesn’t use it is missing out, but fail to describe exactly what the measurable effects are. The blog posts and articles are similarly lacking in concrete facts or data: a few anecdotes about how people discussed the keynote in real-time and little else.

So I’m left asking myself: what exactly are people tweeting about as it relates to live events? How often? What value are they getting out of it?

Before I swerved into technology, I was involved in research psychology, and my motto was “show me the data”. So, hoping to answer my own questions, I took a stroll down data lane.

To better understand how Twitter is actually being used at a live event, I asked our intern, Jeff Losek, to analyze all 797 tweets associated with a particular event, Wordcamp SF. Wordcamp is a self-organized “unconference” centered around the use of WordPress, an open-source blogging platform (in fact, the platform on which this blog is built). The attendees (over 700 of them from 32 different countries) tended to be tech-savvy and relatively cutting edge. As such, this is a “best case” analysis of the use of Twitter.

Clearly, there are many, many events out there that would not see as much or as diverse use of Twitter as this event, but I thought it made more sense to analyze a “best case” event, based on the belief that where the cutting edge leads today, the rest of the world will follow tomorrow (or next year).

Methodology

We collected all tweets containing the event’s hash tag, #wordcampsf. Based on an initial analysis of the content of the messages, we categorized them into one of the following buckets:

  • I am, want to or can’t attend WordcampSF
  • Announcements or questions related to Wordcamp SF
  • I am preparing for or on my way to WordcampSF
  • I am at the event / a session
  • Talk to me or meet me if you are interested in a particular topic
  • Comments / quotes about a particular speaker
  • Here’s what I’m doing / feeling (not directly related to event)
  • Tweets directed at an individual
  • Retweets

Results

Here’s what we found:

Pie Chart of Tweets at WordcampSF event

A few interesting points jump out from this chart:

  • Tweets that are not directly relevant to the vast majority of event attendees (“Here’s what I’m doing / feeling”, “talking directly to someone else”) make up about 1/3 of the tweets sent.
  • Tweets that are useful to people who can’t physically be at the event (“Comments / Quotes about speakers”, “Announcements / Info / Questions related to event”) make up more than 1/3 of the tweets
  • Tweets that report people’s intended or actual location make up around 1/6 of the tweets (“Traveling to”, “At the event / session”)

This is actually reassuring. While there are a large number of tweets that I would consider “noise”, a signal to noise ratio of 1:3 is actually pretty good, especially considering the ease with which one can ignore “noise” in Twitter. Even the tweets that aren’t actionable or educational, such as people announcing their excitement around attending, have value: they advertise to non-attendees that the event is happening and to attendees that the sender is going to be a participant who would like to be included in discussions. We shouldn’t undervalue the impact of “buzz”.

Events have a lifecycle: attendees’ behaviors before the event is very different from their behavior while the event is going on is different from their post-event behavior. How do these differences reflect in Twitter use? In particular, is there value to the wall-projections of Twitter that we sometimes see at events?


These graphs shows how the different types of tweets changed over the event’s lifecycle (the event itself occurred on May 30th, 2009). The graph to the right shows the absolute volume of tweets, the graph above each category as a percentage of the total tweets for that day. For this analysis, we removed “Retweets” because the categorization doesn’t reflect the content of the tweets. Our take-aways?

  • The proportion of “noise” tweets are relatively low leading up the event, and only bloom after the event is complete (when the overall number of tweets decrease).
  • There is a steady stream of tweets with useful information or questions regarding the event that continue for several days post-event.
  • The types of tweets that it would be useful to see on a projection, comments about speakers or sessions, people’s location at the event, and annoucements or questions about the event, make up significant number of the overall tweets during the event itself.

I find this chart reassuring. It shows that although there is a background of chatter and noise, a significant amount of the communication going on over Twitter is useful and relevant, the type of information that I’d like to at least be peripherally aware of.

Our final question was “who is sending these tweets”. For this analysis, we wanted to see how the tweeting was distributed across people.

What do we learn from this chart?

  • While 258 total people sent at least one tweet, 20 people account for more than half of those. That’s consistent at a high-level with the “long-tail” notion of user-generated content (i.e., a large number of people contribute small amounts of content, but that content in aggregate accounts for a large proportion of the total content). The numbers, however, don’t fit cleanly in the 80/20 90/10 buckets that are often cited. Instead, it’s more like 50/50 (50% of the content is accounted for by a small number of high activity contributors, 50% by everybody else).
  • Most people tweet 6 times or less total about the event over the 9 days analyzed, less than a tweet / day.

On the whole, I find this data analysis curbs my skepticism. If I had to summarize:

Many people are tweeting about things relevant to an event. Prior to the event, they’re asking questions, requesting meet-ups and advertising their attendance and at the event itself, they are discussing the active talks and providing timely information about what’s happening at the event itself.

As noted early on, this is a “best case” example of how Twitter is used at an event, but it doesn’t seem overly optimistic to expect that these patterns will grow across the rest of the event industry over time. Given that Pathable has made some key investments in Twitter integration recently, including a “which of my Twitter friends are also at this event” feature and real-time Twitter updates from attendees, I’m encouraged that we’re on the right track.

18 Responses

  1. Sean says:

    Interesting work, Jordan & Jeff.

    I don’t suppose the Twitter API allows you to determine how many people actually read these tweets, does it? That would speak more directly to their impact/utility. Looking at the follower data might be another angle on this.

  2. Jen says:

    Interesting study. I personally think a 1:3 noise ratio is too great. Just goes to show, that as powerful as Twitter can be, it is self-weakened by misuse.

  3. Catherine says:

    Check out the tweets related to a major social media/social change event in Toronto this week: Net Change Week #ncwk
    I’m volunteering as a “session blogger” and will be using Twitter as the application of choice. I followed the stream yesterday – it was quite interesting and confusing. The challenge of course is to sort out the noise from the real content and make sense of the many conversations happening simultaneously – but then that’s what Twitter is all about isn’t it?

  4. This is great info- now if you apply it to things like “tweeting surgery” or Twebinars- what is the utility/ROI there?

    Personally, I think we have to look at twitter as we would look at any other channel of communication, and think about:

    1) is this information I am communicating for me or for an audience?

    2) Is this information that works for delimited text (twitter) or is it more useful in audio/video/long text format?
    3) Are we using this as a broadcast channel or a communication channel in this instance, or both?

    Twitter functions well for me as a crowdsourcing tool- friends pointing me to information that might be relevant or important, that I may not have looked at otherwise. It’s a great way to keep up with friends in an ambient attention way, help people out with problems, find resources and answer questions- all of these things can happen if you have an extensive network.
    However, there are times twitter is nothing more than a loud cocktail party, where I hope serendipity will lead me towards something interesting.

    I find event tweets with links and resources can be helpful, but the “Guess where I am now?” stuff adds to the noise, as much as I am guilty of the same behavior, mostly to allow friends and family to know what I’m doing so they know I won’t be at home, or not to look for me online.

    I can’t thank you enough for this- it helps spell out how twitter is used with events in the most useful way I’ve seen yet.

  5. Julius says:

    Great analysis guys,

    although very misleading link back to Event Manager Blog about visualizing twitter at events.

    One thing for sure is that we never fostered the get rich quick scheme that a lot of technology vendors/bloggers/turned to be SM pro do. We have been striving to storytell our audience how poor concepts are a complete fail no matter how much technology you want to embed in your event.

    We feel that the post has been used out of context, disregarding the 2 years long conversation about the topic. This is unfair, specially since we have no business objective linked to the blog.

    Therefore we are a bit sad to appear as implicitly extolling event managers on how cool is to visualize twitter is and the counterpoint of your interesting study.

    We definitely like the way your argument goes on and conclude but next time fish a bit better in the growing crap you will find out there talking about Social Media and Events to explain what is not cool.

    With affection and respect

    Julius

  6. Please accept my apologies Julius, the link wasn’t meant to imply that you were endorsing those visualizations (breathlessly or otherwise). I was looking for a good summary of the various visualizations out there (because referencing them without linking to them seemed lame) and you had the best list.

  7. Sean, unfortunately, there isn’t a way to know how many people read the tweets. Follower data is a good indicator, as you suggest, but still imperfect: most people will use a standing search on a hash tag (e.g., through search.twitter.com, or through a Twitter client) to keep track of _everyone_ who mentions the hash tag. Twitter doesn’t, to my knowledge, let you know how many people are searching for a term, only how many people are using it.

  8. Jeff Hurt says:

    If the only tweets you’re reading from an event are Hiya chatter, then you’re not following the right events. I’m not buying what you’re selling.

    I’m seeing a lot of meetings and events with Twitter discussions that are content-rich and relevant. I don’t think one study of Wordcamp Twitter stream is adequate research to make your claims or adequate research for a Webinar on Social Media and events. Plus I’ve not seen any Webinars on social media and events and since you’re not an event planner, I question why you’re doing one.

    We’ve used Wiffiti for two years successfully during our events with our attendees where our audience sends question & answer to the presenter while projected on a screen. We didn’t have the noise as you are describing it. It was extremely successful and our attendees have asked to continue doint it.

    Similarly, I virtually attended an event from World Innovation Forum in May that had 20+ official bloggers tweeting and there were more than 30 pages of relevant tweets. There was little noise and these people were doing it right.

  9. Jeff, I appreciate your feedback but find it confusing. The bulk of the post above provides data that there IS valuable activity in Twitter, not that there isn’t. It feels like you didn’t read past the second paragraph. Did you get to the part where it says “this data curbs my skepticism”?

  10. Jeff Hurt says:

    I did get past that statement which is in the next to last paragraph of a really long post. My frustration is that you’ve taken one event and say it is a “best case” scenario for events. Those are your words, not mine. You used an “unconference” and one that did not intentionally plan to use Twitter for the event. Your example shows your lack of respect for meetings professionals and particularly those that integrate social media effectively into their events.

    At the beginning of your post, you made some broad statements that you’ve read your fair share of blog posts about how to use Twitter at events intimating that bloggers & planners don’t know what they’re doing. You make some really hard negative comments about people talking about social media and events.

    I don’t appreciate it when people make broad sweeping statements and then profess they know a better way. You also linked to a great meetings and event professional as an example of negative post about projecting Twitter during an event. You have no idea what we event professionals are talking about regarding social media and took that post out of context. You’re not using social media to listen to us meeting professionals. You’re using it to sell your product. That’s why I don’t trust you.

  11. jschwa says:

    Harsh words, Jeff. I hope we have the opportunity to meet in person, I’d like to change your negative opinion.

    I said “best case” meaning “this event is likely to show Twitter working very well, and can’t necessarily be extrapolated beyond events that are similar to it.” I think you’d agree that one should be careful about extrapolating unconference data to conferences, which is the point I was making.

    I stand by my statement that there are a huge number of snake oil salesman out there saying they know how to teach the “magic of social media”, but when you attend their webinars or read their blog posts, it boils down to very little meaningful content.

    That does NOT mean that ALL bloggers and planners don’t know what they’re doing, it just means that there is a social media “gold rush” going on and there are some hucksters out there taking advantage. The presence of bad examples does not imply the absence of good examples.

    As for the projection, I clarified that above. I was linking to his list, not commenting on his opinion of it. I realize that the clarification was also at the end of a really long blog post, so you may have missed it, as well.

    You say: “You have no idea what we event professionals are talking about regarding social media and took that post out of context. You’re not using social media to listen to us meeting professionals. You’re using it to sell your product. That’s why I don’t trust you.”

    The post above barely mentions Pathable. Your comment reads to me like you responded to this without realizing that at the end of my “really long blog post”, I present a conclusion OPPOSITE what you thought I was saying, and now you don’t want to backtrack.

    The conclusion of the post is that Twitter provides a lot of supplementary value before, during and after an event. I think you agree with that statement, so I’m having a hard time understanding what your complaint is about.

  12. […] an events social networking company, has posted an analysis on the use of Twitter at WordCamp SF. The above chart shows how 797 tweets each categorized by a […]

  13. […] an events social networking company, has posted an analysis on the use of Twitter at WordCamp SF. The above chart shows how 797 tweets each categorized by a […]

  14. David Ing says:

    Your analysis of signal-to-noise of 1:3 for attendees at the conference is interesting. One man’s signal is another man’s noise, because my presumption is people who follow me on Twitter are following me as a person, rather than Twitter devotees who follow a hashtag.

    As one of the people who didn’t attend this conference, but might have followed a person that did — I am a WordPress fan, although not in SF — I’m sure that I would have a different signal-to-noise calculation as a distance follower.

    P.S. I discovered this blog post via Friendfeed aggregation of Tim O’Reilly’s Twitter, pointing to his blog post, which shows how convoluted this whole microblogging environment has become.

  15. […] an events social networking company, has posted an analysis on the use of Twitter at WordCamp SF. The above chart shows how 797 tweets were categorized by a […]

  16. […] of the related content, according to Pathable, is more about people and their experience at the event or its sessions. About a third run along […]

  17. […] le #hashtag afin de voir ce qui se dit et ainsi vous pourrez réagir, répondre, retwitter.  Voici ici une analyse des hasthag à la suite du Wordcamp à San […]

  18. […] interesante sobre el uso de twitter en los eventos: Data analysis of Twitter at events. Y un post de El Caparazón titulado: Twubs, herramienta especializada en el seguimiento de […]

Leave a Reply