Sunday, January 3, 2010

I Invented Google

One reason people have given me for resisting the free sharing of their ideas has to due with regret. It seems to be an almost universal experience to feel resentment when you have an idea and then later see someone take the same idea and make a lot of money or get a lot of credit from it. In most cases you know that the idea was developed completely independently and that the person who succeeded did so solely on their own merit, but if you've broadcast the idea beforehand then the thought that you might have given them your idea and then seen them succeed with it seems too much to bear. For this reason, people still want to hoard their ideas.

I thought I'd share an example from my own life of giving away an idea that ended up being the foundation for a hugely successful company. I hope it may help give a sense of perspective on this.



Back in the mid-90s, I came up with the basic idea that Larry Page and Sergey Brin later developed into PageRank. They used it as the basis of their search algorithm and founded the company Google as a result. I even mentioned the concept in a published paper. I don't know whether either of them ever read that paper, but they might have because we worked in the same field at the time, Information Retrieval.

If anyone should feel resentment I should, shouldn't I? That could have been me running a billion dollar company.

What nonsense. I have never felt anything but happy for their success. They deserve everything they have. In the unlikely event that they even read that paper, in the even more unlikely event that it unconsciously inspired them to invent the Page Rank algorithm, then I am nothing but proud to have given a tiny amount of inspiration to a technology that has improved peoples' lives to such a degree. But I didn't start up Google the company. I didn't make the millions of decisions that made it become the company that it is or that caused it to impact peoples' lives the way it has. I didn't even write a search engine for web pages.

Now, my statement that I came up with the basic idea behind PageRank is a bit overstated, so I need to give some context to show how little importance there is in having previously thought of an idea that leads to someone else's success.

During the 90s I worked on a research project that looked for ways to write artificial intelligence programs that worked with the law. One of these programs was a search engine for court decisions. Among the things that lawyers need when working on a case is something called a "leading case". This is a legal decision that is the first to decide a new principle in law. In trying to adjust the search results to bring the leading cases to the top, I realized that I could do it by looking at how often the case was cited by other cases. Anyone working in the same area of law was going to use the same leading cases so it would be cited more often than any others. By adjusting the search results to take into account how often other cases cited that one, the leading cases popped to the top of the results.

There are three big leaps from that idea that Larry and Sergey made that I didn't. The first is to apply this not just to academic papers (which is the equivalent to legal text that they started with) but to all web pages. Because I was thinking about the problem in terms of leading cases, I don't know that it would ever have occured to me to apply it to the web. What is a "leading" web page?

The second big leap was to realize that the importance of the web page/academic paper/legal case that cites another should be used in calculating the importance of the cited one. Since judges decisions need to be treated as equally important (at least within a certain level of the court) it is doubtful that would have benefited the search results in my program. In any case, using the citation score of a citing case did not occur to me (although using the matching score of the citing case to a particular search result did).

The last big leap they made was to start up a search engine company for the web using that algorithm as the foundation. I would never have done that. In fact, I thought it was a really bad idea not just because of all the competition already out there but also because I thought crawling the web for a search engine wouldn't scale as the web grew in size. I had a friend working on the OpenText search engine at the time, and I had many conversations with him about why I thought it was the wrong approach for indexing the web. I'll write another post sometime about what I thought would work and what I did about it. I'll post it just for historical purposes, though, because it turns out that thanks to MapReduce, BigTable, and the Google File System I was dead wrong.

In any case, the point I am making here is that you should get over the feeling that sharing your idea will make you feel like a fool later on if someone else is successful with it. As I said in a previous post the idea is the least important part in someone becoming successful. If I can realize the silliness of the notion that I invented something as big as Google, you should be able to do it to.

No comments: