A Candle in the Dark

Flowchart

2011-12-26T08:05:00.000-08:00

Cool flowchart, not sure I would ever be able to make these...

The Information Diet

2011-08-17T12:43:00.001-07:00

We are swimming in data these days, but what has that done for us lately? Are we any smarter or do we remember less? With all of this data, we have no fixed methodologies of how to deal with it, how to process it and how to communicate it. Without a methodology we are left scattered, disjointed, confused with fixed bass points to agree upon in short supply. My new project is in response to said ideas; it explores simple ideas in data design to make the world's data sets slightly more useful. Please come visit and leave your feedback.

The United Clusters of America

2011-07-18T08:52:00.000-07:00

The visualization in the NYT depicting call data segmentation attracted alot of criticism. Like others have stated, I have no idea how this clustering algorithm was implemented and why they chose as many groups as they did and not 5 or 100. Regardless, it is an interesting idea to visualize clusters of people (cultures) within the U.S. based on some algorithm and data set. Keep in mind there are so many data sets that describe a social connection or a behavior to potentially choose from (facebook, twitter, netflix, trulia). As these data sets are explored, we would be able to quantify more exactly how culture varies across the United States and what similarities we share.

See Say by Eric Fischer

2011-07-16T11:09:00.001-07:00

Eric Fischer most recent stream of photos on Flickr consists of erie and beautiful visualizations of places twitter users (blue) and flicker users (orange) upload their content. Regions of white are where users have uploaded both photos and tweets. What emerges from this is a discovery of the most picturesque (photo worthy) regions compared to just regions of lots of activity (twitter chatter). Things that were obvious: the beach, the bay, the golden gate bridge, alcatraz and golden gate park are the most popular places to take photos are viewed in a new light. South of the city, everyone is just tweeting (probably about how they can't wait to go to the city and take photos!).

What I have yet to find is exactly how this was done; what was the size of the data set, over what time range, during what hours of the day. Twitter is notoriously hard to pull large quantities of data from, so I'm curious to see what effort was involved. It would also be cool to visualize data this way over time: are there more pictures taken in the summer, do more people tweet in the evening? Regardless of the mystery, this is some great work by Eric Fischer.

Ask Obama

2011-07-15T15:54:00.001-07:00

On Tuesday, July 5, President Obama had his first "Townhall" meeting on Twitter where anyone using the hashtag "#askobama" could ask the president a question. Tens of thousands of twitter users submitted over 100 thousand questions on a wide range of topics (charts here). Over 10 thousand tweets were retweeted by other users.

Beneath this collection of tweets is a retweet network where the most influential messages by the most influential users gets propagated across the network. For example the tweet:

Poker is not a crime. Do you support licensing and regulating online #poker? #askobama

was the most RTed tweet, with over 400 RTs. Other top tweets were about jobs, legalizing marijuana and the U.S. deficit. Of the users @mentioned and RT in other users tweets, these are the users that appeared most:

townhall, 8163,
barackobama, 2900
whitehouse, 2095
iowahawkblog, 1020
twitter, 773
reppaulryan, 624
uschamber, 513
jack, 477

This RT network can also be visualized. Shown below is the RT network of the 100 users that were RTed the most. Clearly, Barack Obama and the Townhall channels make up the center of this graph while other influencers surround them. The arrows indicate the direction of information as in A->B means A was RTed by B. Its interesting from this perspective that there is a hierarchy of RTs: some appear more like "sources" of information: "drewhampshire", "iowahawkblog" while others, with many arrows pointing in, appear as sinks: "ben_howe", "assignmentdesk1", "digiphile" and "anncensky". Surely the sinks are still some of the most RTed users, they are surely sources for other less important users down the line.

Religions of the World

2011-07-06T20:15:00.000-07:00

Christianity is the most popular religion in the world, followed by over 2 billion or nearly 1/3 of the entire population. Buddhist make up about 6% of all religous followers while Judaism falls well short of 1%. Nonreligious people, including agnostics and atheists, currently account for 16% of the human race.

Volatility per Day of Week

2011-07-05T09:19:00.000-07:00

Volatility is a measure of price variation in a financial market. It is used to determine the risk of a financial decision. It often also signifies a mood about the financial market: higher volatility often means more anxiety, more worry and more erratic, aggressive behavior. The figure shows the average volatility per day of the week calculated over the last 60 years for the S&P 500. Not suprisingly, Monday is the day with the highest volatility before descending to the lowest point on Thursday (before making a small return by Friday).

Aspect Ratio of Charts

2011-07-04T16:47:00.000-07:00

It always seems natural when drawing a chart to make a rectangle, golden ratio perhaps. I explored whether the number of data points / number of bars in a chart makes any difference on the optimal aspect ratio of a chart. It seemed to me, from looking at these for any where from 3 to 20 data points, a squarer chart always seemed better. Do you agree?

New Color Palette

2011-07-04T16:34:00.000-07:00

A grande strategy for colors is to use those found in nature: blues, yellows and grays of sky and shadow. Don't use multiple colors to represent the same kind of data. Don't use colors on the opposite side of the color wheel in a multiple bar chart. Don't use red for positive numbers (red is strongly associated with losses).

City Sweat

2011-07-04T11:19:00.000-07:00

Summer time can get very hot, especially in cities where alot of people in close proximity can all contribute to the sweat factor. What are the sweatiest cities in the United States? In the chart above, we show the population density on the x axis and the total population on the y axis. The points are cities and the color is proportional to how how on average July gets. From this perspective, New York is the sweatiest cities having both a large population with a large population density as well as a warm July temperature of 83 (compared to San Francisco at 68).

Toy Advertisement Clouds Per Gender

2011-06-20T12:20:00.001-07:00

The most common word used for boys toys and girls toys. Discussion here.

Cool Maps

2011-06-11T20:29:00.001-07:00

More than any type of data visualization, there have been some really amazing maps published in the last two years that publish some really amazing views of data.

These maps include, in no particular order

Public Reports
The 2008 Election
Campaign Contributions
Gas Prices
Unemployment

By the New York Times

Geography of Race
Netflix Preferences
Another view of race from Wikipedia
Crime

By Trulia
Trulia's Days on Market
Crime

Cool projects
Anywhere you can go in 10 minutes:
The growth of Costco
The growth of Walmart
Congress
Twittervision
Basketball Players
FlickerVision
Hate

Urban Growth

For more cool maps, be sure to read mapscroll

Viral Coefficients

2011-06-07T06:55:00.001-07:00

Here's a cool chart I made awhile back to help understand "virality". Virality is defined as the expected number of children a user invites as a function of time (over a user's lifetime). The number of children a user is expected to spawn in on the t_th day can be modeled as V(t)=alpha*t^(n)and then this function is integrated up to T to get total invites up to that day. Below is a grid in which n and alpha are varied (n: -1.05 to -1.35 and alpha from 0.05 to 0.3). The chart shows the rate of acquisition of children in the first 7 days. There is also a V(7) and V(21) value below that and on the bottom is total expected number of children as T goes to infinity. The boxes are color coded by this total virality. It makes a good picture of the difference in total virality when the parameters, n and alpha, change slightly.

Infographics Everywhere

2011-06-05T13:57:00.000-07:00

I found this "piece of trash" lying on the street: a wedding itinerary from May 2010. Wherever you are, Debbie and Tim, best wishes and lovely design on your wedding itinerary.

Price Per Square Foot Heat Map

2011-05-26T08:32:00.000-07:00

I found this heat map of price per square foot in San Francisco that Zillow posted awhile back. It was a realy intriguing map, but a little low on resolution to gather any block speck specific information. I blew the image up, made it transparent and laid it over a more detailed map of San Francisco to the best of my ability. A little intriguing: price per square foot along 24th or Valencia in the Mission was some of the lowest in the city. Really? Likely, this data was aggregated over some time range that goes far back. Not sure. But it just spawns the idea of how awesome this would be: Imagine looking at a movie of the price per square foot heat map in the city, watching some regions / neighborhoods value increase while others descend. Also would be interesting to see a heat map of very recent (last month or year) and compare with the longer term average. Also volume and inventory. These would all be indicators of trends in san franciscan's perception of value for these geographical regions.

The Best Visualizations

2011-05-24T08:28:00.000-07:00

There is a nice discussion thread on Quora about the best data visualizations of all time. You can tell from my comment, I have a slightly different perspective. Most people fail to see the larger picture of information turned into pictures and ignore some of the greatest data visualizations of all time. Still, there is something to say about the new genre of data visualizations that stems from the new forms of data collected and accessible on the internet. Here is just of a "modern" data visualization that is absolutely amazing: a historical record of the NBA dunk contest.

Trends in MLB

2011-05-23T13:57:00.001-07:00

While triples per game has descended from 0.5 at the turn of the century to just over 0.2 in present day, doubles have almost doubled during the same time range. Why? It seems like size/shape of ballparks would have something to do with it as well as relative running speed of batter to outfielder's arm. The next plot is a little more obvious: homeruns have increased 5x over the century to more than 1 per game in recent times while strikeouts have also increased. Recent times, when steroids were at their highest level of use, batters were swinging for the fences more than ever before, accentuating the trend. It also indicates the evolution of the game on a longer time scale: while fielding and base running were the primary tenet of focus earlier in baseball's lifetime, an increasing focus on the dual between the pitcher and the batter drives both player's kpi upward. Hits per game, batting average, runs per game and walks per game are all fairly flat over time.

The Stars in the Sky

2011-05-22T10:39:00.000-07:00

I've been working on this project for awhile: what are the brightest stars in the sky, how big are they, how heavy, how hot and also how bright. Similar to the Herzburg Russel diagram, but possibly more useful since it is only the stars that are most visible. Click here or on the image for the interactive version. A nice result of plotting the data like this is that you can see lines of equal density running as diagonals pointing NE on the plot, which are shown as "same color". If you travel NW you go from low density stars to higher density stars which become hotter (and blue-er). The difference between a cool red and hot blue star is something any amateur astronomer can see in the night sky, even in San Francisco :)

When is it Time for Pie?

2011-05-15T17:57:00.000-07:00

Pie charts get alot of bad press, but they are effective for pictorially describing trends in data. Pie charts imply fractions of whole. Other ways to show fractions of a whole are through bar charts or treemaps. So when do you choose a pie over these other forms? As shown in the first diagram, pie charts are good ways to show fractions such as a half and a quarter (or deviation from halves, quarters). They are also good for showing equality (all equal slices).

But when you start to get more than 3 or 4 pieces of data, showing pies becomes more difficult and area charts increase in utility.

Its likely that a treemap would do even better in this situation and surely as the number of data points increases. Area charts become even more useful when you want to chart fractions over time. So pie charts, I believe are only useful in the cases described. And they are never useful when not talking about fractions of a whole.

Contest

2011-05-10T07:43:00.000-07:00

Listed below are the eight brightest non-star objects in the night sky. The bars represent how far they are away in light years base 10. So while the Orion Nebula is a little over 1000 light years away, the Andromeda Galaxy is pushing 2 million and the Pinwheel galaxy is well over 10 million. Using the base 10 of a list of quantities allows one to compare several length scales at once and show in a proportional way how they are orders of magnitude different. But still, looking at this, we are probably not as impressed as we should be. What other ways are there to view this data set?

Lists

2011-05-08T10:19:00.000-07:00

We generate data not only in the form of text (from blogs or newspapers or comment or status updates) or activities (choices such as purchases or clicks or downloads) but also through more complex content such as image uploads (as well as video, music). Images are probably the way we share the most data. Many images are a "bad data set" because they are not tagged well with metadata (location, time, context, tags), but a small percentage of images are.

What is useful to do with an image's metadata? Is it useful enough to combine images together in meaningful ways using their metadata? Including below two example: all the Messier objects (non star items in the night sky almost visible with the human eye and a collection of gems). Are these lists useful? Do they tell us something? Are lists efficient ways to visualize data?

The Not Obvious

2011-04-10T09:10:00.000-07:00

In one sense, data visualization is in its infancy: we have come to a critical point where our ability to consume the amount of data available requires new solutions. From another perspective data visualization has been around since the dawn of civilization: humans have created pictures to tell stories and communicate in ways that written and spoken language could not. Beginning possible with the cave drawings in Lascaux, France and continuing with all the map makers in ancient civilizations, people have turned to pictures to share complex ideas ways that are easiest to understand and remember. Although Galileo and Leeuwenhoek where scientists at heart and inventors by duty, they were also data visualizers by default, using their sketches of worlds small and large to make available for popular consumption worlds never seen before by human eyes. I leave below a few more data visualizers, ignored in the traditional sense, but nonetheless consistent with the very essence of communicating new ideas through pretty pictures.

Eadweard Muybridge (1830 - 1904)

Ernst Haeckel (1834 - 1919)

John James Audubon (1785 - 1851)

Weatherspark

2011-04-09T20:37:00.000-07:00

I tiring to hear people argue wether "This year was not as rainy as last year" etc without data to back it up. A simple visualization tellling how much and when we had rain in the last few years solves the problem The following below from weatherspark, shows the amount of rain per week over the last three years. While not perfect, and without the months labelled, it still does the job.

The World's Populations

2011-04-04T08:58:00.000-07:00

The next step in looking at all the world's people at once.

Letter Transitions

2011-03-14T09:30:00.001-07:00

From a purely statistical perspective, for a given language, there are patterns in how we choose each letter depending on what letter we chose last. For example, we can measure all the pairs of letters in a huge amount of text, and find that ("t", "h") and ("h", "e") occur very often, primarily because "the" is one of the most common words, but also because "those", "then", "head" and "heat" are also fairly popular. Shown below is a measurement of all the letter transitions for a large amount of text (a political philosophy paper of one of my old professors containing about 200,000 words). The verticals are the first letter and the horizontals are the second letter. From blue to yellow to red shows an increasing probability of occurring (see most red for "th" and "he and most blue for "ya" and "zz"). The blank regions (white spots) are for letter transitions that were not observed in this document ("jb", "gz").

This visualization gives a snapshot to how we use letters to write words and communicate. How specific is it for "academic political philosophy"? how does this picture change for newspaper sports writing or biotech research or second grade book reports? How different are these visualizations across language (for those that use the same alphabet)? Who has done this before?

Visualizing Inequality is Hard

2011-03-04T11:29:00.000-08:00

Alot of data that is strongly skewed, like 80% of data is produced by 20% of the producers. Although its hard to show this visually. Does this do a good job? I get some insight, but still I feel like I'm missing something.

Noticing Changes In Metrics

2011-02-21T11:48:00.001-08:00

The following chart shows a hypothetical scenario of some metrics over time. The y-axis is the average measure (say fb pokes per user per day) and the x-axis is days. Because the average population is 10,000 users per day and suppose the total variance for each one of these distributions is 10, so sigma/sqrt(N) = 0.1 wide fluctuations for each of these measures. On day 20, after three weeks of constant data, we lower the average value by one, two, three and four average fluctuations (in blue, red, yellow and green respectively). At what point, if ever, does someone who is looking at these graphs notice that the change occurred?

Visualization of the Egyptian Revolution

2011-02-21T10:35:00.000-08:00

Kovas Boguta's visualizations hint at a new type of journalism based on large amounts of unstructured public data. He has shed light on how the volume of trending topics on Twitter relate to real time news events and real time social structures. His practice of dissecting and disseminating the massive amounts of information shared on the internet every day is invaluable. However, in his latest work regarding the Egyptian revolution, his visualization that made it to the New York Times falls short.

People that speak the same language are obviously more likely to be connected to each other and people that speak two languages will likely be positioned in between the two groups, connected to people in both languages. Also, users that are more 'influential', wether by means of page rank or centrality, etc. will be more likely to be found in the middle of this group of people. Otherswise, I think this graphic shows a tangled mess of Twitter users.

Visualizing the structure and characteristics of social networks on twitter is a great start to undertanding the origins of political events, but visualizations should strive to provide more specific insight from the data that it represents. Similar to Kovas's previous work, I think it is more important to understand how information was shared in this network. Where did popular messages, hashtags start and how did they get retweeted, propagated through this network and how did they react to or create real events? As data visualization becomes more central in journalism as well as in education, it should be held to the same set of standards: to inform and enable, not simply to entertain.

I Hate the Name 'Deep Analytics'...

2011-02-19T06:06:00.000-08:00

but I like this article's list for working on business analytics.

1) Knowing which questions matter most, and where to find the answers.
2) Learning how to quickly recognize and respond to information.
3) Using the correct statistical techniques to come up with answers.
4) Dissemination by visualization
5) Automation (so resources can be based on new research and development)
6) A business culture based on facts.

Carbon Dioxide Emission Per Capita

2011-02-06T11:39:00.001-08:00

Seeing plots like this are scary. The rate at which we are burning fossil fuels seems to have accelerated ever since consumption began during the industrial revolution. Have we done anything to slow this rate of fosil fuel use? This infographic seems to provide an answer, but the answer is very hard for me to see. Is there an easier way to say the same thing? Wikipedia has a nice link detailing the carbon dioxide emissions per capita per country all over the world since 1990 that anyone can use to make an infographic. The obvious observation from this data is that many (all?) of the countries have increased their emissions over time. But my question is how is this rate changing? Are any countries doing any better in lowering their emissions. The following plot shows a point for a given country with its percent change of carbon dioxide emissions per country in the last 17 years compared to emissions in the last 10. From this perspective, it looks like overall, countries have "decreased their rate of increase". For the countries that produce little enough, they were actually close enough to the axis that they were able to cross is and start decreasing the rate at which they produce.

Visualizing the Inner Planets

2011-02-05T17:35:00.001-08:00

here

Trends in Waste Production

2011-01-25T16:19:00.000-08:00

I've become curious about whether our ability to reuse and recycle has helped lessen the waste we produce and exactly how fast these trends are changing. Although data for the United States was hard to find, I found a great data set from geocommons of the historical record of waste production for all European countries. What is the difference that has made some European countries produce less garbage and some produce more? I don't really see any connection between countries that are in each group. Also, I'm curious what percent of each recycle? Some information can be collected here. If data like this is not readily available for all countries, states and cities, how will we have any transparency on wether we are actually improving?

The Colors of Words

2011-01-23T08:26:00.000-08:00

Cymbolism is an amazing website that allows users to vote on what color they most associate with a given word (war, peace, happy, sad). It collects these votes over time and provides visualizations on the popularity of a color choice for a given word. Its an interesting concept to look at these visualizations and try to interpret what they mean. Some are obvious: "sun" is mostly yellow and "sad" is mostly grey.

But, it is more interesting to look for patterns across words. For example, how do color choices for negative words compare to those for positive words? Are the color choices for opposite words complementary? To have a start, I collected the all time color choices for six pairs of opposites with sufficient data collected on each word (shown below). With a brief look, I can make the following statements: positive words are brighter with more equal selection of colors (rainbow) while negative words are darker, grayer and often times red. Red, followed by yellow and then green, are the most popular color used to associate a word (most communicative colors). Finally, the similar color choices between love and hate (love has slightly more red than hate) shows some truth to the idea that there is a "fine line" between the two. What other observations can we make from this?

Ellie's First Year

2011-01-22T08:29:00.001-08:00

Visualizing Relationships in A List

2010-12-28T10:03:00.000-08:00

Data often comes in the form of a relationship between pairs of items in a list (i, j) such as how many people that play baseball also play soccer, etc. I've explored trying to visualize this in a color coded matrix form with some, but limited success. Another approach discussed in Flowing Data tries to visualize these relationships in a circular form (circos). Here is a hypothetical example of circos: what colors of hair people of a certain hair color are attracted to :

I find this approach rather unusable. It is busy and I can't quite understand the area around the circle or what the thickness of each band is or why it changes. Similarly, it can't be done easily for more than 5 or 6 items in a list. Another approach is a linear form of circos as the following figure showing the transition from childhood religion to adult (current) religion relates:

Even in this case, however, it tells me more than I want to know. What I really care about is what is the breakdown of childhood religions (this tells me this, but not well), and for each one, how likely is it for a person to transition to another. In addition, it would be most interesting to know the most popular transitions (evangelical -> none and catholic -> evangelical) as well as the least popular transitions (none -> catholic). With complex data sets, the last thing one wants to do is make an even more complicated picture. Capturing and communicating the "essence" of a data set in the most simple way remains the goal of visualization.

What if We Made Things As Simple As Possible?

2010-12-25T09:39:00.000-08:00

Here.

Geographical History of Religions

2010-12-12T20:11:00.000-08:00

Future Research Ideas

2010-11-27T21:54:00.000-08:00

I don't know where else to put this.

Average Volatility Per Month

2010-10-08T03:49:00.000-07:00

Inspired by Visualizing Economic's blog post about October volatility, I set off to find my own way to view this data. I took the same data set, between 1950 and 2010, and I computed the average volatility (difference between high price and low price and then divided by the average price) for each day of the year. For most of the year, the average size of the fluctuation is rather constant, possibly dropping slightly between the beginning of the year and summer months. But as fall approaches, volatility hits a peak, likely around early November, before descending down to its lowest point at the end of the year and resetting. I'm curious what the origin of this trend is. Does it have to do with fall harvest or the election season or christmas shopping?

The Physics of Confetti Falling

2010-09-05T05:51:00.001-07:00

I wanted to see what the relationship is between the width of spreading of confetti (0.5 cm circular paper cutouts) and the height you drop it at. The experiment was crude but the results were sensible. It looks like the max piece of confetti scales as about half the height (if you drop a piece of confetti from a mile high in windless conditions, it can translate as far as half a mile in distance). Meanwhile the expected trajectory is much smaller, about 1/6.

Food Consumption in America

2010-08-30T07:29:00.000-07:00

This link is really a great resource for food consumption by type per American over time to spot trends in how our diet is changing as food prices vary and new trading partners get established. You can see some of the fastests growing edibles: Limes, Bell Peppers, Brocoli, Romaine Lettuce, Cucumbers, Onions and some of the recent droppers: Oranges, Graipfruits, Potatoes, Head Lettuce. The image above shows the amount of pounds of meat per year the average American eats broken down by type. Look at chicken surge!

Why Stoplights Are Red

2010-08-30T06:08:00.000-07:00

Since my daughter is into pressing buttons, especially ones that are connected to a computer screen, I built a little Processing program that would change the color of the background whenever a keyboard button was pushed. Being the loving father I am, I then thought it would be a cool experiment to measure how long each color on the screen stayed on for, maybe to see if there were colors she liked more than others. So I built this program here that would change screen colors when a button was pressed and record how long each color would stay on the screen. I then took this test myself, but changed the game to see how fast I could recognize each color. The task was that when a color was shown, I will say the color out loud so I actually have needed to mentally process the color I saw before pressing the button again. The graph attached shows the average reaction time (0.7s) to each color including black and white. White was by far the fastest, while purple was the slowest. An interesting conclusion when thinking about traffic lights (the human eye responds faster to red than green) and street signs (none are purple)!

Large Amounts of Money

2010-08-27T08:12:00.000-07:00

Visualized beautifully here.

Visualizing Food

2010-07-24T17:21:00.000-07:00

Colors in Culutre

2010-07-07T07:46:00.003-07:00

.
Geesh, there must be a better way to visualize this?

Removing Some Important Links

2010-06-29T11:25:00.000-07:00

Byte Mining
Color Lovers
Learn R
Visualization Class
Daniel McLaren
Revolution Computing'
Strange Maps
Wall Stats
Poly Monkey
Neoformix
Many Eyes
Well Formed Data
Web Science
Tom Carden
Swivel
Juice Analytics

Zero Intelligence

I dont get this visualization

2010-06-20T00:19:00.000-07:00

I think they did a bad job.

Military Spending: U.S. vs. The World

2010-06-15T08:03:00.000-07:00

Heights, Weights and Positions of NBA Players

2010-06-06T22:04:00.001-07:00

Yay!. Any suggestions on how to improve it?

Words to Communicate Visualizations

2010-06-01T20:21:00.000-07:00

Skyscrapers

2010-04-22T21:34:00.000-07:00

Everytime I go outside in San Francisco, I look at the skycrapers that cluster north and south of Market St. I've come to learn alot of details about the buildings: I know many by name, their address, their height, number of stories and year built. I've gained this view in my mind of how they grew out from the ground, year after year, how they would compare one to another if we lined them up in a row, how cool it would be to fly from one to another with the click of the switch. This visualization is the result of this line of thought. How can I change this to improve the experience of a user who is learning about San Francisco's skyscrapers?

What Would You Do?

2010-04-09T08:42:00.001-07:00

How would you visualize the average temperature for three different cities differently?

The Quality of A Twitter Audience

2010-04-08T08:45:00.000-07:00

For the most engaged users, Twitter is a popularity contest. The larger a user's audience, the more their voice will be heard and the more influence they potentially have. However, it is possible that two users who have the same size audience may have very different levels of influence in the Twitterverse since the quality of that audience may be very different. Quality of an audience is likely proportional to how large their subsequent audience is (secondary audience) as well as inversely proportional to how many other users that audience is listening to (following). A measure of the ratio of followers to following, which can be termed "bandwidth", may be a useful general measure. In terms of the social network that twitter followers create, the bandwidth of a user can describe the likelihood of information flowing through that user as a node on that graph. The number of followers and followings of a user's audience, along with that user's audience's distribution of bandwidths may help to quantify the quality of the audience.

For the following figure, I choose two users MissouriSandT and lattesc who both have nearly 1000 followers. The following figure shows the subsequent audience size and number of users they are following for each member in the primary audience, MissouriSandT is in blue and lattesc is in red.

.

The differences are somewhat obvious: the blue does not follow as many users and is followed by more users than the red. In other words, given that these two audiences are the same size, a user would rather have the blue because it is a higher quality audience and has a higher bandwidth. The average number of users MissouriSandT's audience follows is 1544 and the average number of users that follow MissouriSandT's audience is 3609. The average bandwidth of this audience is 3.65. lattesc's audience on the other hand follows 3206 users on average, has 2727 users following them and has an average bandwidth per audience value of only 0.61. The following figure shows a different perspective in visualizing this difference in audience quality by plotting bandwidth as a function of number of users they are following for both audiences.

Although it is difficult to identify precisely, there are clear differences of the types of users that are likely to have higher quality audiences than others. Fixing audience size, it appears that users with high quality audiences are more involved in a community, likely to discuss ideas about technology and publish tweets that are likely more original and more likely to be retweeted. Users with low quality audiences, on the other hand, are more likely to be news sources and serve possibly as a "dead end" source of information for users that use Twitter in a less proactive way. To get a better idea of what I am talking about here are other users with about 1000 followers and have a high quality audience: cheray, wei_yang, dandobi, johnmartinez and users with low quality audiences: samesexokla, anylondon, my_smartphones, news_in. I'd like to hear your own interpretations of what users in these two groups have in common. Surely, going forward, incorporating twitter age, tweet frequency and retweet history will either reinforce or improve upon the audience quality metric discussed here.

Where Are the Best Basketball Players?

2010-03-29T21:14:00.000-07:00

Watching the NCAA final eight games last weekend with some friends, we started talking about why was it that there were so many middle America states in the tournament. Do states like Kentucky, Michigan, Kansas and Illinois have a stronger "basketball culture" and do they actually produce more talented basketball players?

Using this wikipedia page, containing the current NBA rosters for 2010, I ventured to find out. I parsed the html for each player in the league and extracted the school that they went to. I then mapped each school to its respective state and used Google's Visualization API to create the following map of NBA basketball players per state. North Carolina is number 1, California is number 2 and much of the midwest makes a solid showing. Surely, the thing to do now would be to normalize by either a) the number of universities in the state or b) the population of the state to really get the probability of producing an NBA player. Next time.

Your browser does not support iframes.

Work from Mozilla

2010-03-29T15:32:00.000-07:00

I really like this visualization that shows the general proportions of one item compared to another in addition to the comparisons of the subtypes for each type, all pretty in one color properly labeled. Nice work! More work here and here.

What Is Data Visualization?

2010-03-23T07:40:00.001-07:00

I am occasionally asked what are the visualizations that I am most proud of. It is an awkward question to say the least, with no clear answer for me and I think it really misses the point for someone who takes the discipline seriously.

Data visualization, I'm convinced, is more art than anything else. In this case, the medium is the data itself, and, if the data is rich enough (not difficult to find these days), it can be sculpted, mixed , patterned and placed in an endless number of ways. The artist's job is to navigate the possibilities that the data offers and find the one(s) that work: that communicate a message, that are beautiful and that , if you are really good, impress and inspire the audience. Like other artforms , data visualization has obvious techniques: color, shape, area, position and simple interactions and manipulations. Like other artforms there are styles and genres: network graphs, timelines, heatmaps, pie charts and word clouds. But I'm sure that the best visualizations in the times to come will succeed by conjuring new ways to look at data. Data visualization is fundamentally about being creative with the information and about the discovery process for both the artist and their audience.

Could you ask Beethoven what was his favorite song or Picasso which of his paintings he liked the best? For artists, their favorite work is likely to be the one they just finished or even more so, the one they are currently working on. Or, if they are feeling sentimental, it might be their first piece they ever completed, that moment in their life when they "got it" and knew that things would never be quite the same ever again. For me that moment was when I made my first "mixing field" at Los Alamos that indicated where , in turbulent flow, the areas where most and least of the mixing occurred.

On top of being beautiful, images like this communicated something very insightful about how fluid flow actually occurs that no one has quite understood in the same way before. If you want to learn more about the visualization, you can (try to) read the paper here or watch a movie of a related metric called the Okubo Weiss criterion. This work taught me that compelling graphics can translate the insight shared by only a few scientists who have access to the data to much larger audiences. I was determined to do more.

Surely, I'm getting carried away here when I compare the activity of exploring data to classical music or modern art. But for a topic that pervades so many aspects of our lives from news and education to entertainment and business development, the understanding of what a person who does data visualization actually does is lacking. If the length or clarity of the wikipedia article has any merit on indicating the public's perception of a topic, then making analogies between established forms of art and this rather new field of data visualization is helpful. And in the long run, I don't think its a huge stretch to say that, in the data rich world that we are entering, data visualization has the potential to enrich society as much as any artform has accomplished before.

Ellie's First 22 Days (Roughly Speaking)

2010-02-23T18:44:00.001-08:00

The black is sleep and the white is awake. Not much day / night differentiation yet!

County Map of Unemployment (Thanks to Flowing Data!)

2010-02-21T18:34:00.000-08:00

Another awesome tutorial, found here, using a csv file that contained unemployment data and an svg file that we write a python script to edit using Beautiful Soup (who knew an svg was a xml file!)

NBA Game Performance of Top 50 Scorers

2010-02-20T21:37:00.001-08:00

An absolutely wonderful tutorial available here.

Things Learned:

Its easy to read in a csv file (when you specify that it is separated by commas)

nameYouWantToCallIt <- read.csv("www.fileFromTheWeb.com", sep=",")

You can sort that file by some column label

file <- file[order(file$ColumnToSortBy),]

Turn that file into a matrix:

file <- data.matrix(file)

Do some "heatmap" operation on that matrix!

file <- heatmap(file, Rowv=NA, Colv=NA,
+ col = cm.colors(256), scale="column", margins=c(5,10))

Bad Religion Visualization

2010-02-20T18:39:00.001-08:00

Visualizing A Football Game

2010-02-07T10:55:00.001-08:00

What a great idea, would love to see this in the newspaper. Happy Superbowl Sunday!

Visualizing the World's Population

2010-02-02T21:10:00.001-08:00

A one stop shop for viewing the population of all the countries in the world.

The Universality of Song Length?

2010-02-01T22:04:00.000-08:00

This article has been moved HERE.

Each of the libraries themselves displayed similar distributions, depending on what genre's of music they listened to. In the next figure, I looked at the song length probabilities of different genres of music (as labeled in the itunes file, not always an appropriate label, but a good approximation).

The blue (shortest) are punk rock songs, the red are hip hop and the yellow are techno. Its interested that the hip hop songs, which generally speaking have the largest audience (the distribution for songs labelled "pop" is also very similar) centers itself directly on the 4 minute mark. The techno and punk rock songs deviate slightly (in opposite directions) from this mark, and the smaller, more niche following that these genres attract seems like logical result. Thats all for now, I have other interesting observations that I'll share in the near future. Please feel free to suggest your own things to take a look at.

Visualizing the Lives of Presidents

2010-01-23T09:42:00.000-08:00

I've been thinking for awhile about a visualization of all of the United States presidents that could quickly teach someone the most important aspects of each president's life and how the events of one president related other presidents and fit into the larger context of American history. After much thought and debate, this is the design I've come up with. I've broken each of our last dozen presidents into a horizontal with time on the x axis (I got lazy updating the data here. I've included major life events: birth, school, time in congress, president, etc. This helps visualize the rapid ascent of Obama's presidency compared to Reagan's or the massive amount of time Eisenhower spent in the military or Ford spent in congress. Would love to hear suggestions on what you think or ways to improve this visualization.

Beach Boys Songs

2010-01-22T21:37:00.000-08:00

Where the White People Are

2010-01-06T12:10:00.000-08:00

Best Color Combinations

2009-11-21T11:16:00.001-08:00

When presenting data in a colored form, which color scheme is best? I seem to be a fan of the upper right one (variations on a color (w/opacity)) or the bottom right one (variations on gray with a single standout color).

Movie Narrative Charts

2009-11-19T22:54:00.001-08:00

An amazing feat, especially The Lord of the Rings!

Online Dating Advice

2009-10-13T21:53:00.001-07:00

Found here

The Grunge Slowdown

2009-09-28T09:25:00.001-07:00

Beats per minute for the most popular songs of the year. Most popular songs hover above the 80 bpm (lively, dancey) level. Most striking is the drop off in the 1990s. Can this be attributed to the emergence of grunge and the lack of disco / Michael Jacksonish music the preceded? I would love to see the bpm for the top song or top 10 songs per week to see a better average.

Atlantic Hurricane Tracks

2009-09-27T20:01:00.000-07:00

How Much Colder is This Year?

2009-08-29T09:59:00.000-07:00

This data is for San Francisco average temperatures using this source from weatherunderground. Blue is this year while red is last year. Especially in the last 3 months (since June), last year's weather is likely to be warmer than this year's weather 75% of the time so far. Click on the image for better resolution.

Maxim's Top 100 Women

2009-07-21T11:45:00.000-07:00

2009-06-28T22:16:00.001-07:00

2009-06-20T21:55:00.001-07:00

2009-06-20T13:49:00.000-07:00

Social Network Analysis in R

2009-06-09T19:41:00.001-07:00

R contains several packages relevant for social network analysis: igraph is a generic network analysis package; sna performs sociometric analysis of networks; network manipulates and displays network objects; ergm implements exponential random graph models for networks; latentnet has functions for network latent position and cluster models; degreenet provides tools for statistical modeling of network degree distributions; and networksis provides tools for simulating bipartite networks with fixed marginals. Most of these packages are part of the statnet suite, obtainable through the statnet meta-package.

Processing Test

2009-06-05T14:15:00.000-07:00

Yeah! This is how you can embed an applet in blogger!

APPLET name="VioletPiece2" WIDTH="400" HEIGHT="500" archive="http://www.thevioletpiece.com/music/VioletPiece2.jar" standby="Loading Processing software…" codebase="http://www.thevioletpiece.com/music/" code="VioletPiece2" mayscript="true" scriptable="true" image="http://pappmaskin.no/opensource/processing/sketch_081001a/loading.gif" boxmessage="Loading Processing software…" boxcolor="#FFFFFF">/APPLET>

Visualizing the Possible Chess Moves

2009-06-04T11:51:00.001-07:00

Absolutely wonderful here

Meat Map

2009-05-17T16:50:00.000-07:00

Lincescum is the Best?

2009-05-17T14:21:00.001-07:00

Wolfram Alpha is Here!

2009-05-16T23:09:00.000-07:00

Thank You Stamen

2009-05-10T00:50:00.000-07:00

The average of late show images from David Letterman, Jay Leno and Conan.

Many planes take off from the airport.

Sorting by Geometry

2009-05-09T20:49:00.001-07:00

Discussion here

Twitter Work

2009-05-05T20:52:00.000-07:00

The ways you can search, sort and view Twitter data from their open API seem endless. The image is a plot of Tweets about "cloudy" (gray) and "sunny" (blue?) for geo location across the United States (can you make out the outline?). More to come, stay tuned.

Political Contribution Visualizations

2009-05-05T20:47:00.000-07:00

A cool site that allows you to view party contributes by region and per person in the 2008 election.

Drawing Networks is Silly

2009-05-03T16:51:00.000-07:00

I'm fairly convinced that trying to draw networks that are bigger than a few dozen are silly and pointless (unless you want people to say: "ooh look at how tangled it is...that's complex!")

More Pie Anyone?

2009-04-24T16:32:00.000-07:00

New front page to the VP, now with more pie charts. This lib will be useful for the new colors of the sky / calendar viz and network viewer I'm working on.

Visualizing Time

2009-04-23T13:57:00.000-07:00

I've been working with how to view dates / timelines of a list of actions or events (such as spore creations from the Spore API). The idea would be to read in a list of dates, display them and coordinate other info (rating, size, type) with the color and size of the visual return. Here is where I've gotten so far:

Each disk is a day, a colored disk indicates that a creature was made on that day, mousing over the disk shows the creature. It would be easy to relate color to creation type (creature, spaceship...) and maybe size proportional to rating or complexity. Can also easily plug this into to Twitter tweets per user, Flickr uploads, etc. A good start, but now a break.

The Top Tweeters

2009-04-22T15:37:00.000-07:00

Thanks twitterholic.

Otl Aicher

2009-04-17T13:48:00.000-07:00

is great.

Another view of the world's eyes

2009-04-17T13:40:00.001-07:00

(un)photographed Spain from senseablecity on Vimeo.

K Means Klustering

2009-04-09T10:06:00.000-07:00

Is a way to separate data into groups, given that each data point is represented as a vector.
-For each data point, compute its membership in clusters by choosing the nearest centroid.
-For each centroid, recompute its location according to the numbers.

Cabspotting

2009-04-08T15:40:00.000-07:00

Another piece of work by Stamen.

Eric Rodenbeck (Stamen)

2009-04-08T15:24:00.001-07:00

Financial Metrics

2009-04-08T14:08:00.001-07:00

Some nice metrics to help one think about the current state of the economy.

Legal Drinking Age per Country

2009-04-01T14:55:00.001-07:00

Life Expectancy in the U.S.

2009-03-29T17:20:00.001-07:00

Market Capitalization of Selected Banks

2009-03-22T12:37:00.000-07:00

Though I'm not sure yet how the data was obtained.

Most Popular Engish Words

2009-03-12T13:52:00.000-07:00

here