The temptation, I suppose, is to compensate for my post-post deficit by rattling off a list of “personal updates” that account for the intervening 117 days. (Blogging aside: Lists generally lend themselves to blog posts. A blog itself is basically a poorly organized list. Slapping in some sublists creates the illusion of structure.) I will resist this urge, and opting to save these pearls for the impending swarm of summer blogging. (Second blogging aside: overly long posts are a blight on the casual blogger.)

Instead, I simply note that on April 28th my doctoral fate was all but sealed with the successful presentation of my long awaited thesis proposal. This strongly suggests that my thesis defense will take place in the fall, and that the up-and-coming fall semester will be my last at Brown. An extremely timely exit, since a delay of even a few months would permit me to make the dubious claim that “my tenure as a student at Brown spanned 3 decades”. At that point, I assume I’d be a lock for graduate student orator at my graduation ceremony next May, but my preference is still to acquire a PhD in time for my 10 year high school reunion (slated for Thanksgiving weekend).

With the uncertainty of my post-Brown existence drifting closer by the day, the need for a more regular blogging regiment is at an all-time high. Expect more updates shortly.

Apple does not, in fact, own the New York Times. For the entire day, however, the Times website has featured an overly prominent Apple advertisement. Much like an earlier Apple ad that appeared on several and tech sites, this ad is comprised of two interacting panels. The side panel feature John Hodgman and Justin Long doing the whole Mac vs. PC thing, and the top panel features some stationary text. Near the end of the both ad, however, Apple blows your mind by having the characters from the side panel interact with the text in the top panel. This reveals that they are actually part of the same ad (in the first two-panel Apple ad, the panels weren’t even touching, so this revelation was a genuine surprise).

The problem big problem the New York Times ad is that the top panel text looks just like a full page headline. It’s in a serif font and it’s a quote from the Wall Street Journal. It reads just like actual news. Being Apple/Internet obsessed, I was already familiar with this style of Apple ad, I’m worried how the average person perceives the ad? If someone doesn’t bother to turn on the ad’s sound, and doesn’t spend 20+ seconds watching the entire ad, how are they even supposed to know that the quote/headline is part of the ad? When someone doesn’t figure this out, it’s basically like Apple paid the New York Times to put up a headline about how great their operating system is.

Would the New York Times ever do this sort of thing in print? I doubt it. Would they let Fidelity, for example, pay them to put up the headline “Fidelity’s Magellan Fund outperforms the S&P 500″? I seriously hope not. If they are letting Apple put up a headline on their website, it means one of two things. Either the New York Times is seriously desperate for cash (in which case I hope they made a fortune off this ad), or they view their website as being significantly less important than the print edition. I read the New York Times everyday, but like many people I know, I almost never read it in print. Come to think of it, I don’t even know where it’s sold in Providence.

According to Wikipedia, nytimes.com gets about 13 million unique visitors per month. The paper’s average daily circulation is about 1.2 million. For many people, it’s the website that matters most. The New York Times should acknowledge this by selecting its web advertisements more carefully. In the meantime, I hope I’m not the only blogger that encourages the Times’ to do so.

According to Dave (and his team of experts) the last 70 years may soon be viewed as an abnormality in the world of political prediction. Today we rely heavily on polling data, but back before 1930, betting on wall street was all the rage. During the 1916 election of Woodrow Wilson over $160 million (in today’s dollars) was wagered in Wall Street’s outdoor curb exchange (this later became the American Stock Exchange).

With money at stake, newspapers could predict elections simply by reporting how the masses were betting. This same idea is gaining prominence today. Dublin-based InTrade, for example, hosted close to $25 million dollars in bets for the US 2004 election (their founder, who also started TradeSports, recently engaged in q & a over at freakonomics.com).

InTrade (and TradeSports) is just one an example of a prediction market, a financial exchange in which the final value a position is tied to the occurrence of some real-world event. Another example is Predictify (also the focus of a recent freakonomics post), a new website where you don’t even need to wager to make money (although it’s too soon to tell if it actually works).

Since Dave didn’t have space, I thought I could supplement his article by briefly explaining how a prediction market works. For example, suppose you want to predict which candidate will win an election, Ms. X or Mr. Y.

A simple approach is to allow individuals to buy a share of the winnings if a particular candidate wins. In this case, you can choose to spend d dollars on Ms. X, and if Ms. X wins, you’ll get a share of the total amount bet on either candidate. If T dollars were bet in total, M of those dollars were bet on Ms. X, you’ll get d*T/M dollars.

The nice thing about this is that before the election, M/T can be interpreted as the probability Ms. X wins. Why is this? Well suppose M and T are both big numbers, and M/T is .15. If the election is about to occur, and you think Ms. X has a 20 percent chance of winning, the amount you expect to gain by betting 1 dollar on Ms. X is .2*[your payoff if Ms. X wins] = .2*T/(M + 1).

We assumed (M/T) = .15, and both M and T are large, so T/(M + 1) is very close to 1/(.15). This means that by betting a dollar on Ms. X, you expect to gain at least .2/.15 > 1 dollars. In other words, you expect to make money by betting on Ms. X. More generally, when M/T is less than Ms. X’s true probability of winning, an informed investor has the opportunity to make money by betting on Ms. X. Similarly if M/T is too large, an informed investor can bet on Mr. Y.

This all sounds great, but there’s a problem. Before the election, M and T change over time. If you notice one week before the election that M/T is way too small, when can invest a dollar in Ms. X, but your final payout is tied to the final values of M and T once the election takes. Unless the election is about to take place, you have no incentive to invest.

To fix this, prediction markets don’t simply act as betting pools, they create shares for a given event, and allow investors to buy the shares at a price depending on M/T. In our election example, a prediction market can initially sell each candidates shares for $0.50. After a while, when M and N dollars have been spent on Ms. X and Mr. Y, respectively, and T = M + N dollars have been spent in total, a Ms. X share is priced at M/T, and a Ms. Y share at N/T. Once the election takes place, if Ms. X wins, a given share of her stock originally bought for d dollars is worth d*T/M, and a share of Mr. Y stock is worthless.

Now let’s check that this works. As before, assume Ms. X has a .2 chance of winning. When the election is about to occur, the expected return on a Ms. Y share is .2*(M/T)*(T/M), so if M/T is less than .2, you should still buy her stock (and similarly if it is greater than .2, you should buy Mr. Y stock). Now observe that, unlike above, you should still buy even if the election is a week away.

If you expect M/T to eventually approach .2, but shares are selling for $0.15, you can still expect to profit. Even if you are wrong and M/T drops further, you’ll make even more money if Ms. X wins. Finally, if M/T starts to increase beyond .2, you can make money by selling your shares for a profit. In conclusion, the market works! Capitalists rejoice!

In the end, there’s actually much more flexibility regarding how a prediction market can operate (and other problems to worry about). If all you want to predict is the outcome of an event, as oppose to each outcome’s probability, money isn’t even required. All investors really need is a good incentive to choose correctly. Back in the days of the curb exchange, non-monetary bets were more than acceptable. Here in Providence the classic “roll a peanut up a steep hill with a wooden toothpick”-wager seems most appropriate.

Anyway, according to the Times, Princeton has lowered their average for humanities PhDs to 6.4 years (down from 7.5 in 2003). This statistic makes me think the 8.2 year average was also limited to humanities. If not, I dare say I’m wildly ahead of schedule, as is my entire department, and every CS PhD I’ve ever met (I’m guessing computer scientists take 5.5 years on average). Personally I’m shooting for 5 years, although I did start working with my advisor before I was enrolled as a graduate student.

I’m not sure what these humanities folks do for 8+ years, but I’m guessing it isn’t work towards their PhD full time. Science PhDs generally get paid enough to be full time students and come out of school debt free, humanities not so much. To reduce the time it takes to get a PhD, Princeton guarantees all PhDs at least 5 years of funding (this includes tuition, health insurance, and about $30,000 a year to live on). Brown has recently made the same guarantee. This decision doesn’t affect us science types much, but it sure makes getting a degree in an English or History more appealing. Sadly most universities don’t offer a similar guarantee, but this could be the beginning of a trend.

Another major factor in the length of a PhD program is what students plan to do with their degree. In the sciences, PhDs often get jobs in industry. This is generally much less competitive than going into academia and as an added bonus, the pay is better. In contrast, humanities students often vie for a small number of academic jobs, which puts additional pressure on them to have a truly groundbreaking thesis (good luck with that). Finally humanities types read a lot of books. That’ll hold anyone back.

There’s plenty of stuff I could write about this argument, but since it’s late, I’ll defer to folks who’ve already obtained their PhD. Philosopher David Chalmers wrote a very accessible article about why we should and shouldn’t worry about being merely simulated humans. You can also watch him being interviewed at Blogginghead.tv. If your up for discussion, a number of physics and computer science types have had plenty to say about the article here.

Among the commenters is Peter Shor (famous for his work in quantum computing), who points out that if we live in a simulation, all bets are off in terms of our discovering a Theory of Everything. After all the simulation could contain a bug. Scott Aaronson (slightly less famous for his work in quantum computing) also questions the value of the Simulation argument, since it fails to make any actual prediction about how our universe (simulated or not) should behave. In other words, it’s not really science. Personally, I’m not going to lose sleep over the whole thing either, but I am a little less dismissive.

I think the Simulation argument highlights the much more scientific understanding of consciousness computer science may provide us with. Also, as we do develop our own increasingly complex simulations of physical phenomena, we may become aware of their shortcomings. Perhaps a complex computer simulation necessarily leaves its imprint on the phenomena being simulated. If this were the case, we could at least develop an understanding of the implications of living in a simulated universe. I imagine it would look kind of pixilated.

As I mentioned yesterday, comments on Ezra Klein’s blog fill in some of these details. My old friend Matthew Yglesias also has the right idea, and Eugene Volokh spells out the mean/median discrepancy in more detail. Despite all of this discussion, a number of important considerations have gone unmentioned, hence this more comprehensive post.

First the facts. The New York Times cites a federal survey stating men have a median number of 7 heterosexual partners, while women have only 4. It also cites a British survey in which men have 12.7 partners and women have 6.5. There is simply no way 12.7 is a median from a large survey (even 6.5 is very unlikely) so we should assume these numbers are means (Jordan Ellenberg of Slate backs me up here.)

The Times also explains that the average number of heterosexual partners men and women have should be equal. Why is this? Suppose a population has M men and W women, and within that population N male-female pairs have had sex. Men have an average of N/M partners, while women have an average of N/W. In the real world, M and W are very close (the Times didn’t mention this assumption), so the averages are nearly identical.

As the article points out, this proof of equality doesn’t account for one gender having a lot of sex outside of the population being surveyed (for example, US men might have lots of sex overseas). Still, are most men having half of their sex overseas? Of course not. So what gives?

One possible culprit, as many have already pointed out, is “the whore effect”. In the female population, there will be a very small percentage of women (prostitutes) who have had a very large number of sexual partners. These outliers increase the female population’s mean, but not its median. Think about it, you survey 100 women, 5 report 0, 5 report 1, 10 report 2, 20 report 3, 30 report 4, the rest 5 (or more). The median is 4, and if any one of these women were replaced with a prostitute, the median is still 4. The average, on the other hand, can increase by a great deal. If the prostitute had slept with 300 people, the average would increase by 3.

So if a population contains female prostitutes, but not male prostitutes, having heterosexual sex, they can bring up the average of the female population without changing the mean. Still, what about the British survey which gave means? It turns out prostitutes influence the mean as well. They do so in two ways.

First (and less significant), a prostitute being surveyed likely does not know exactly how many people they have slept with. They probably stopped keeping count. As a result, they would generally give a conservative estimate, artificially lowering the female mean. Second (and this one is more interesting), any researcher taking a survey is likely to throw away a small percentage (say 1 percent) of outliers. They do this with good reason.

Suppose you are surveying a thousand women, and they all give you numbers like 5, 10, 20, and then all of a sudden one says “1000″. This number is unlikely to be exactly correct, but it significantly effects the sample mean. Even worse, it hugely effects the variance, which you’ll probably use to calculate a confidence interval (the mean plus or minus some range). So if for some reason 1 in 1000 respondents decides to mess with you, they can greatly impact the quality of your results. Rather than screen for liars, it makes sense to throw out the biggest outliers. Unfortunately, this masks the existence of legitimate outliers, in this case prostitutes.

So the whore effect is real, even when estimating the mean, but how big is it? Happily (or sadly?) it cannot be that large. Suppose 1 in 1000 women are prostitutes (a rather high estimate) which on average have slept with 1000 men each (again, a high estimate). The exclusion of these prostitutes would lower the female average by 1. Similarly, if men discounted the sex they had with prostitutes, the male median would unlikely change by more than 1. To see why, assume 1 in 10 men have slept with prostitutes, and figure at least half of these have slept with 6 or more non-prostitutes. Including female prostitutes, 50 percent of men slept with 7 or more women, so we can assume 55 percent have slept with at least 6. When prostitutes are excluded, at most 5 percent of male totals dip below 6, so the median remains at least 6.

So we are forced to conclude that the impact of female prostitutes is small, and that men or women (or both) are not being truthful when surveyed. Still, I doubt the liars are solely to blame. As commenter Kathy G writes on Ezra’s post, both parties in a sexual pairing may not agree about what happened. For example, if a survey doesn’t explicitly call sex consensual intercourse, all bets are off. There could easily be a tendency to skew one’s totals by defining sex more or less liberally.

At least traditionally, I’m sure men are more likely to receive oral sex than women, so ambiguous sexual encounters would probably tend to bolster the male totals. Also there can be ambiguity over consent, which would once again lead to an increase numbers for males. Finally there’s the issue of gratification. If an encounter leaves one party less satisfied than the other perhaps they’d be less inclined to remember it as sex. Sadly, it’s hard to estimate the impact of each of these effects.

So there you have it, a concise 10 paragraphs of supplimental commentary. The conclusion: there may be some whores out there, but men and women still need to get their stories straight. The funny thing is, had this article been better written, I would have found it a lot less interesting. Sadly, all the discussion it prompted among bloggers may discourage writers from trying to incorporate mathematical observations in future articles.

Finally, I should mention that in the course of writing this I came across an actual paper on the subject (although I haven’t read it).

With all that out of the way, I decided to turn my attention to Newsweek’s cover story, Facebook. I dare say that Newsweek seems a little late to the table with regard to the Facebook craze (for example, there was this lengthy New Yorker article over a year ago). Still, as Newsweek points out, Facebook is growing at 3 percent a week, so they’ll no doubt be a lot more Facebook news in our future.

Media attention aside, Facebook came a long way this last year. For one, it opened itself up to non-students (and apparently much of its current growth comes from people over 25). It also added newsfeeds, allowing users to quickly (and constantly) check up on each other. Most recently, it has provided a platform for web developers to write their own apps for use on Facebook. As the web becomes filled with people using Facebook-dependent applications, Facebook’s future becomes that much more secure.

Any social networking website relies on users designating their relationship with other users. In Facebook, friend relationships form the links in a social network. The network is used to spread information as well as navigate the website. When discussing the utility of Facebook, I think people tend to focus on its ability to share information among friends. As the internet grows however, there’s bound to be lots of new and easy ways for people to share content with each (for example, consider last week’s post on internet-based computing)

The real staying power of Facebook lies in the value of having online social networks at all. These networks allow us to take our real life identity and put it online, and once we all have online identities, we can do a lot more than just keep tabs on each other.

Suppose I have a website where I want to allow people to vote (for example Wikipedia could allow people to vote on whether certain articles are accurate). Voting only works if one person cannot pose as thousands of people, and vote thousands of times (in computer science land, this is called a sybil attack). Given that one person can fairly easily get their hands on many computers around the globe, posing as multiple people has traditionally been easy. Social networks offer a solution.

Though it’s certainly possible to start multiple Facebook accounts, it’s quite hard to create tens or hundreds of accounts. If you create all these accounts, they’ll need different friends, different profiles, different photos, etc. To fake all these things, you’d need to start making all of your phony accounts friends with each other, but this discrepancy would be reflected in the structure of the social network. As a result, an active user on Facebook (or perhaps some future, more refined version of Facebook) can be considered nearly as legitimate as you meet face to face in the real world. As a result identities on Facebook (or other social networking websites) are destined to become a building block of many future online applications.

I’m sure the folks at Facebook know all this and are busy adding buckets new features. In terms of future growth, it seems like they should focus on features that make for the most useful social network. That’s why, I’m surprised they haven’t added other types of relationships between users. If your boss, roommate, and grandmother are all on Facebook, it would be useful if you weren’t simply “friends” with all of them.

Also, Facebook should probably allow certain types of relationships to fade over time. If everyone you’ve ever met is your friend on Facebook, knowing that two people are friends becomes a lot less valuable. At the very least you should be able to “reaffirm” a friendship (although I suppose there’s poking and messaging people for that). Similarly some kind of “trusted” friend relationship would be useful for a lot of yet to be developed Facebook applications.

All I’m saying is that if Facebook adds these things, I’ll definitely consider signing up.

Currently nytimes.com only allows readers to comment on op-eds, not news stories. If an informed reader such as myself wants to discuss a news story, we have to go off and find some blog. This doesn’t make sense. The New York Times should want their website to be a forum for public discourse, and I think today’s homepage is a step in that direction.

If the New York Times does allow for more commenting, they should probably have some form of comment moderation a al digg and slashdot. As of right now the website has received over 500 comments on the bridge collapse in Minneapolis, but I doubt most readers made it past the first ten. Comments that receive a higher rating, or more replies, should be pushed to the top.