Sep 24, 2013

Top Movies – A Zoomable Timeline of Movies

I’m proud today to release my first app in almost two years. It’s called “Top Movies”, and it’s a fun and intuitive way to explore 100 years of great films. This project has been a chance for me to try some ideas in interface design, experiment with new technologies, and get back to making things. I hope you find it fun and interesting. You can check it out here.

The Zoomable Timeline

While Top Movies is intended to stand alone, it’s also intended to be a showcase of a interface framework I call the “zoomable timeline”. There are many ways to search content (e.g. iTunes Search, Google, IMDB), and there is plenty of coverage of the latest and greatest (Top 100 lists, iTunes Editorial, blogs), but there aren’t many ways to browse content. To just lay every single app or movie or album of even paltry success in front of you and let you navigate through them without drowning.

"Above all else show the data." – Edward R. Tufte

That’s what I’m trying to do with the zoomable timeline – show the data. I’m not hiding the top content amongst 100 pages you need to navigate between but putting it right in front of you. And when you want more, you can directly manipulate your way through the data space.

Most Movies Are Neither Good Nor Successful

"Anytime there is a lot of content – most of it is going to be bad." – Aaron Sorkin, writer of The Social Network and creator of the West Wing.

There are over 900,000 apps out there, and most of them are bad. And I’m not trying to bag on apps here – most movies are bad, as are most songs, most books, most art, and so forth. According to Metacritic, the quality of movies is normally distributed, with the mean being “mixed”. And according to the Box Office, the success of movies is exponentially distributed, with the median being not very successful.

image

image

If you restrict your attention to only movies with modest success (i.e. above the median), only 5 to 10 movies per year come out that receive “universal acclaim”. That’s less than one a month. And only about 40 per year come out that receive “generally favorable” reviews. That’s less than one a week. If you watch more than one movie per week, which I have on average for the entirety of my life, you eventually need to look back through time.

Looking Back Through Time Is Difficult

You simply cannot do it on iTunes and only barely on Netflix. A site like Rotten Tomatoes has ways to go back through time but only with an absurd amount of unpleasant navigation. I’m not sure anyone has an evenly modestly large corpus that you can directly manipulate your way through.

There are also issues of signal vs. noise. If you’re simply making a list, you need to make a strict tradeoff between quality and quantity. If you set the bar high, you end up with a short list. If you set the bar low, you end up with a long list. You can work hard to make it the best list you can (what I call reaching the “efficient frontier”), but you’re still making a strict tradeoff. For instance, no matter how you look at it, the IMDB Top 250 has no fewer and no more than 250 movies.

Pinch Gestures – Intuitive Traversing of Orders of Magnitude

The pinch gesture is an amazing and I believe underrated invention. A pinch is an intuitive way to traverse an order of magnitude. I think that the original Apple Maps is nothing short of stupendous. All of a sudden the entire world is at your fingertips. You can zoom in as far as your own driveway, and then smoothly transition outwards to your neighborhood, city, country, continent, and finally world. Go from 30 ft to 30,000 feet, and all very intuitively.

I wanted to do the same thing for content – let you intuitively traverse a huge corpus – and because there are vast differences in quality and success, I employed the pinch gesture. Combining a pinch and a timeline allowed me get away from the aforementioned strict tradeoff between quality and quantity. You now control that tradeoff. Zoom out as far as the decade and see just 11 movies or zoom in as far as the day and see 36,000.

Top Movies – A Zoomable Timeline of Movies

Top movies is about browsing and exploring. You can go back week-by-week and see what’s new and interesting, or you can go back year-by-year and see what was popular throughout your life. You can pop up to the decade, go back a bit, and then drop down to the year to see what was happening in the 1960s. Or you can go down to the day level and see more movies than you could ever watch. I hope you enjoy exploring 100 years of great films.

Download Top Movies for the iPad or iPhone.

Sep 11, 2012

"Life Expectancy" Doesn’t Measure How Long You’re Expected to Live

The US Government estimated its population had a life expectancy of 78.5 years in 2009. If you type “life expectancy” into Google, it will spit back the World Bank estimate of 78.2 years in 2010. You’ve likely read numbers close to these in textbooks and articles.

But what do these numbers actually mean? You might guess from the first that someone born in the United States in 2009 could be expected to live about 78.5 years. This is not the case! It actually measures how long someone would be expected to live if every year of their life was spent in 2009. In other words, there is no accounting for progress that decreases mortality rates. And that’s on purpose. It is what is known as a “period life expectancy”.

Period life expectancies are used to track the general health of a population. With them you can easily compare one country to another. You can also monitor general population health over time. But the number you want if you’d like to know how long people will actually live is known as a “cohort life expectancy”. It measures how long someone born in a particular year (a cohort) can be expected to live. It is also not in the US Government yearly mortality report for 2009. The reason is that we won’t know it until everyone born in 2009 is dead! That will hopefully take a long long time. So, while we accurately know the period life expectancy for 2009, we do not yet know the cohort life expectancy for 1909. There are estimates of course, including one of about 83.9 years in 2009 from the Social Security Administration. I would like to offer, however, my own estimate of 86.4 years in 2009 and give the reasoning behind it. In particular the method of calculation reveals why life expectancy is likely still in the 80’s and not in the 120’s or 200’s like some would hope.

Life expectancy is calculated from something called a life table, which is just a big list of the probabilities of dying for someone of a certain age in a certain year. Below is plotted the one for 1940 and one for 2000. The period life expectancy for 1940 and 2000 would be calculated from exactly this data.

The cohort life expectancy for 1940 would only use a single value from this table, namely the one for age 0. The value for age 1 would come from the 1941 life table, and the value for age 60 would come from the 2000 life table. By the time 2000 came around, the chances of dying at age 60 had fallen from 2.2% to 1.0%, a decrease of over 50%! If we plot the 1940 life table against the known cohort data, you can see that they’re quite different, and the the difference grows every year.

To complete the picture we need to estimate what age-specific death rates someone born in 1940 will face in the coming years. As I’ve written before, the mortality rate for any age decays exponentially, the rate of progress is relatively stable for any age, and we’ve made a lot more progress for younger ages than older ages. I’ve plotted the average yearly progress for each age below.

To estimate the cohort mortality rate we can take the period rate and discount it by the age-specific progress over how ever many years it will take someone to reach that age. As an example, the average progress for a 60 year old is 1.25% per year, which means that a 60 year old next year will be about 1.25% less likely to die than a 60 year old this year. Compounding this over 60 years yields an estimated decline of 53%, taking the 1940 value of 2.2% to an estimated 1.05%. Pretty close to the 1.0% value actually seen in 2000. The plots below shows the 1940 period mortality data, the 1940 cohort mortality data that’s available, and the 1940 cohort mortality data estimated using this method. The first truncates at the end of the known cohort values, and the second includes all ages up to 100. The estimates are pretty darn close.

One interesting aside is that the period and estimated cohort mortality curves reunite in old ages. While more distant ages have more time for progress to compound, they have less progress per year than younger ages. In other words, mortality might decrease at 3.3% per year for a 5 year old, but you only have 5 years of compounded progress until you reach that age. At this point the probability of dying will be (1 - 3.3%)^5 = 85% of what it was at your birth, down 15%. Mortality for a 60 year old decreases at a more modest 1.3% per year, but over 60 years it can fall quite a bit. In this case (1 - 1.3%)^60 = 47% or about 53% of what it was in the year of your birth. Eventually, however, the yearly rate of progress falls so far that the compounded rate of progress starts falling. This happens at about 72 years old, and it falls continuously from then on. The net effect of this is what is called the rectangularization of the life curve, which produces a lot more people living to old age, but no one living dramatically longer than before.

Applying this method to the 2009 data produces the below curve. It starts more rectangular than the 1940 curve and gets even more rectangular.

It is from this data, using a method very much like the one that the US Government uses, that I calculated the 2009 cohort life expectancy of 86.4 years, some 9.9% longer than the official 2009 estimate above.

Whether you believe this number or not, it is actually intended to measure how long someone born in 2009 is expected to live. Almost every discussion I have seen on this topic, including the Scientific American article linked above, mistakenly interpret period life expectancy as cohort life expectancy. If life expectancy numbers have ever seemed low to you, it is because they are.

Aug 20, 2012

I Don’t See Any Evidence That We’ll Live Hundreds of Years

I’ve been told that I could live for hundreds if not thousands of years. That technology will stop and even reverse the progress of aging. And I’ve been told this by some pretty smart people, like famed inventor Ray Kurzweil and accomplished scientist Aubrey DeGray. There’s even an entire TED Theme called "Might You Live A Good Deal Longer?". And I must say, I’m tempted to believe them. Technology has come a long way since I was born, and it looks like there are some amazing breakthroughs on the horizon.

But I feel like, if this is really true, we should see clear statistical evidence of it. Kurzweil’s predictions of the future center around technology progressing exponentially, which makes it possible to see these trends coming. And yet he produces no such graph in his TED Talk, just a table with four values. I also didn’t hear anything about it in school, the relevant Wikipedia articles make no conclusive claims, and I have yet to see a good argument float up on Reddit or Hacker News. So I decided to go take a look. In particular, I went looking at how the data used to calculate life expectancy has been changing over time to see where the trend is going. This underlying data are called “life tables”, which are just a big lists of your chances in a particular year of dying at a particular age. The US Government has been collecting these in usable form since 1933, and they make it publicly available through the National Center for Health Statistics. Researches who study this sort of thing have processed it for computer analysis, and I examined it in Stata.

I don’t see any evidence that we could live hundreds of years.

The simple explanation is that there are two competing forces: progress (technology, knowledge, etc.) and aging. In the next year, progress will decrease the chances of someone your age dying, and aging will increase it. To truly stop aging, these two forces must be equal. In other words, progress must decrease your mortality rate by an amount that exactly balances out aging, so that your chances of dying do not increase with age. To defeat aging, the force of progress must be greater, so that your chances of dying go down every year.

Unfortunately neither of these appear to be the case now or in the foreseeable future. Average progress across all ages is indeed exponential, and it has a growth rate of about 2%. This means that a year from now you will be about 2% less likely to die than someone who is a year older than you right now. Unfortunately the progress of aging is also exponential, and its growth rate is about 8%. This means that someone a year older than you is about 8% more likely to die than you are. So while progress is exponential, aging is as well, and it’s growth rate is significantly higher.

The reality is actually worse than this makes it sound, because while the average progress across all ages is 2%, these gains are not evenly distributed. Most of them go to people younger than yourself. For instance, progress for a child in its first year is about 5% per year, whereas progress for an elderly person in their 100th year is about 0.1% per year. This means that the life curve isn’t so much flattening out or even shifting out but rectangularizing. More and more of the population will live healthily to an old age, but they’re not going to start living hundreds of years. At least not in our lifetimes.

I’ll show some data to back up these claims below.

The Force of Progress

The good news is that the chance of someone your age dying is as low as it’s ever been, and it’s falling exponentially! A 60 year-old has less than half the chance of dying as they would have had 75 years ago, down from 2.3% to about 0.9%. Our biggest gains are actually for newborns, which have historically had very high mortality rates. Just a few hundred years ago a quarter of people buried their child in its first year! Your chances of burying your child in its first year are a tenth of what they were 75 years ago, down from just over 6% to about 0.6%.

 

To know where these curves will go in the future, we take a look at the rate of change in the past. To do this we calculate the percentage change between each year, averaged across all ages. While it does vary a bit over time, it’s actually remarkably stable, which means smooth exponential decay. Each year the average chance of dying across all ages is about 2% lower than it was the year before. Just as importantly, while mortality rates have fallen dramatically, the percentage rate at which they are falling hasn’t changed much in the last 75 years. If anything, we’re making less progress each year as time goes on.

While 2% per year might sound small, it’s not. A 2% decay per year leads a value to fall in half in about 34 years, and then to fall in half in the next 34 years after that, and so forth. But as I mentioned before this progress is not the same for all ages, with the chance of a 1-year-old dying falling in half roughly every 17 years, and the chance of a 70-year-old phone falling in half roughly every 53 years. Most of the gains we’ve made are in the first twenty years of life, with progress for older ages appearing to slow almost to a halt.

The Force of Aging

We’ve known for a long time that aging progresses exponentially. This was discovered by famed actuary Benjamin Gompertz in 1825. He claimed that your chance of dying doubles every 8 years. This implies an average rate of about 9% per year. This is very, very fast, with doubles every 9 years.

As Gompertz himself noted, this progression very accurately describes the mortality curve between the ages of about 30 and 80. The mortality rate actually falls every year for the first ten years as children escape from (or succumb to) the dangers of entering the world. The rate then spikes up through the teenage years before smoothing out. Late in life mortality slows its increase. The curve must inevitably be an S-curve instead of a true exponential because the mortality rate cannot increase above 100%. In fact, there is evidence to suggest that the mortality rate flattens out at very old ages somewhere between 40% and 50%. Still though, at these rates someone cannot live for every long.

No Evidence That We’ll Live Hundreds of Years

Putting it all together, I just don’t see any evidence here that we could live for hundreds of years. As I mentioned above, to truly stop aging the advance of progress must as quick or quicker than than that of aging. What we see is just the opposite – progress, while fantastic, advances significantly slower than aging. Furthermore, progress does not seem to be accelerating and on pace to overcome aging in the future. What we see is just the opposite – average advances have been slowing over time.

It does look like we will live longer than any of those who came before us, but it doesn’t look like we’ll live for hundreds of years.

Mar 11, 2012

Facebook’s Logout Page: Definitely Not Self-Referential Design

I was pretty confused by the announcement that Facebook would advertise on their logout page. Who sees the logout page? Why would you even logout?

Then I read this article on Pando Daily and face palmed, as I assume the author Greg Kumparak did. It turns out… get this… that some people don’t have their own personal computer! Huh.

The decision to advertise on the logout page strikes me as a wonderful example of not self-referential design. Self-referential design is when you craft the product based on your own particular needs and preferences rather than those of the users whom you’re ultimately targeting. It’s a really common pitfall that people naturally fall into.

There are some benefits to this approach. It guarantees that you can use (“dog food”) your own product, which helps you stay honest about its capabilities and shortcomings. It’s also easy, as you always happen to have your target user (yourself) on hand.

But self-referential design is in many ways really bad. For one, you are always going to be one of the most advanced users of your product, intimately familiar with its behavior, and totally comfortable handling extreme complexity. Your users are not so capable. You’re also going to have huge troubles resolving debates with other members of your team, because if your self-referential preference is different from their self-referential preference, there’s no way to come to resolution. Your intended users are the much-needed common ground.

You’re also, relevantly here, going to miss potentially giant insights. You are not, it turns out, like everyone else. You make software. This makes you very different.

I own a personal computer. Everyone in my family owns a personal computer. All of my friends own personal computers. But apparently some set of Facebook users do not own personal computers and thus go to the library or other public facilities to use them. And of course, when they do, they use Facebook. And when they’re done, they logout and leave the browser on the logout page. And if that page shows ads, then all those who pass will see them. Genius.

So bravo Facebook designers and developers on understanding a way that your users are not like you, and then profiting from it.

Jan 31, 2012

Personal Analytics with Sensor Data

Some things you do leave a digital trail that can be analyzed for the purposes of personal analytics. Mint.com is a great example. Analytics on your email or your computer usage patterns are others.

A huge number of important questions cannot be answered, however, without the addition of new sensors to your life. While they have more of a “life is a game” than a “live your life better” pitch, GreenGoose is working on attaching sensors to everything. There has also been some buzz recently around Twine and the Jawbone Up. Each gives the promise of tracking more data, and once this data is in digital form, it can be actioned upon and analyzed.

One direction of this that I keep thinking about is the close monitoring of your body and biochemistry. I haven’t the foggiest idea how many calories I eat. Or how much fat. Or protein. Or sugar. I know I shouldn’t eat too much sugar, and I know I still eat sugar, but I don’t know if I’m being super conservative in my intake or rash and imprudent. Tracking my consumption would make doing the right thing way easier.

This tracking dramatically narrows what are usually really wide feedback loops. Right now if I consistently eat too many sweets, I might feel a little ill, but I won’t really know something is wrong until I get diabetes in five years. If I had a continuous glucose monitor, however, I bet I could see my blood sugar levels spike on a daily basis. This would tighten the feedback loop from years to minutes and allow a change of behavior to occur before it’s too late.

Similarly, it’s difficult to see the link between working out and getting stronger, or between dieting and losing weight. These behaviors cause a series of imperceptible changes that add up over the course of months or years.

These loops and oh so many others are in dire need of dramatic tightening. My health and oh so many other things I do in my life are in dire need of better monitoring.

Navigate
« To the past Page 1 of 3
About


Subscribe via RSS