Friday, March 23, 2012

The price of knowledge

At least one commenter has spotted the problem about the application of Hayek's paper and Wikipedia. Hayek's central argument is that organisation is best made by people who are familiar with local circumstances, who know directly of the relevant change in circumstances and have the resources immediately available to meet them. No central planner can have this knowledge of circumstance. Thus organisation is best performed – and is being performed in many cases – by a decentralised price system.

Now if there were an analogy between price-based decentralisation and Wikipedia's decentralisation, there would have to be a way for consumers to signal demand for knowledge by paying for it, and a way for the 'miners' of that knowledge to be compensated by digging deep into the earth of information, trying to find veins of wisdom running through the slag of trivia, or by refining the crude ore containing knowledge thinly spread, into the pure metal of subtle and profound wisdom.

There is no such mechanism in Wikipedia. It is paid for by an annual fundraiser which appeals for donations to collect 'the sum of human knowledge', without any mechanism for donors to sponsor chosen articles containing parts of that sum. Even if such targeting were possible, there is no way of directing donations to individual 'knowledge miners'. All editors of Wikipedia are unpaid volunteers*. This fact has already been noted by Harvard researcher Andreea D. Gorbatai in 2011, who questioned whether collective production such as in Wikipedia creates social utility.

Now there is a sort of reward system on Wikipedia whereby editors can achieve non-financial status similar to kindergarten 'stars'. But follow-ups on Wikipedia to Gorbatai's study concluded that this reward system was perverse, with greater recognition being given to editors producing large numbers of low importance articles than to editors producing small numbers of high importance articles.  An article in the Wikipedia signpost concluded -
- It is interesting to compare the most prominent author of high importance articles at low production rates, Garrondo, with the most prominent author of low importance articles at high production rates, Ucucha. Garrondo has written one FA, Parkinson’s disease in 2011. Ucucha has produced 14 FAs on rare, Latin-named, mammal species. Garrondo has a lousy strategy for climbing up the WBFAN. However, when we look at the impact of the two editors' articles for the readers, there is little question. Because the single Parkinson’s disease article has 180 times the views as Ucucha’s average article, Garrondo achieved 13 times the total contribution to reader-viewed FA content. The problem is all our systems of rewards, all our tracking systems, all our unconscious assumptions, talk page remarks, and so on simply talk about number of stars…instead of the importance of them. We are incentivising the high production of low importance articles and discouraging the opposite. Yet the latter strategy is the more efficient way to serve the readers. [My emphasis]
This provoked fury from Wikipedia's established editors. The discussion is here - beware the heated and often incoherent ranting.

There is another, more subtle, question here. The 'market mechanism' assumes that what consumers want is what they actually need (or rather, it makes no distinction but wanting and needing). There is an older tradition, dating from at least Plato, that the ordinary mass of human beings don't really know what they need, or what is really good for them. The most recent proponent of this view was Lord Reith of the BBC, who believed that broadcasting should be 'an authoritarian system with a conscience', carrying to the greatest number of people everything that is 'best' in every department of human knowledge, endeavour and achievement, and to avoid whatever is 'harmful'. He was criticised for not giving the public what it wanted but he replied that few people know what they want, and fewer still know what they need. I discuss that in an old post here.

*Except for paid editors, of course, who are employed by public relations agencies or rich individuals to embellish articles about their clients, or themselves. But that's another problem.

14 comments:

William M. Connolley said...

Similarly, there is no such mechanism elsewhere. There is none in Britannica, for example. The difference is only that you could see that there possibly could be on wiki. As a contributor, I accept that this is just the way of it. I suspect that the GW article is read a lot.

Anthony said...

The mechanism elsewhere is the actual marketplace. It doesn't exist at Britannica per se, but Britannica is a part of it. As is Wikipedia, though Wikipedia is a part of the marketplace which threatens to destroy the marketplace.

Edward Ockham said...

>> there is no such mechanism elsewhere. There is none in Britannica, for example.

Two points on that. First, I deliberately mentioned the more traditional, Platonic principle that the 'masses' don't really know what they need to know, so that must be left to experts. The Britannica articles on philosophy are written by subject matter experts. Paul Vincent Spade, who is a giant in his field, has written much of the material on medieval philosophy. I have made a comparison between his work and the corresponding articles and, well, there just isn't any comparison. The Wikipedia stuff is mostly a horrible mess.

Second, there is an indirect price mechanism at work. In the universities there is a highly competive system that awards high status to those who pass the 'peer review' process (let's assume this process works, for the moment). Making the further assumption that humans are driven to achieve status (reasonable enough), this resolves the problem of 'supply'. On the demand side, ask about how universities finance themselves. In the purest system, as exists in the US and also in the earliest universities, which originated in the medieval period by the way, people (or their parents) pay for education. The highest demand is for those universities who can attract high-status academics. So there is a price mechanism at work. Britannica is simply leveraging off that system by using high-status academics to write flagship articles.

Wikipedia leverages off this too. For example, the best articles on the history of philosophy and theology are those directly sourced from 100 year old Britannica and Catholic Encyclopedia articles. The problem with this, as I noted in another post, is that in 100 years time there may be no articles left. If Wikipedia manages to destroy the existing, albeit imperfect, market for supply of knowledge, then Wikipedia will have managed to destroy knowledge entirely. This is where Anthony is coming from, I suspect.

On the economics of encyclopedias, by the way, it works by employing high-status academics for the flagship articles, and much lower-paid employees to compile the smaller articles on highly granular subjects, usually by plagiarising existing older material. Wikipedia works much better for this kind of article, precisely because they are easier to write. So Wikipedia is weakest, and strongest, precisely in the areas you would expect from a traditional analysis of how knowledge is compiled.

>> Wikipedia is a part of the marketplace which threatens to destroy the marketplace.

I think this enigmatic and apparently paradoxical statement needs further explanation.

Anthony said...

"First, I deliberately mentioned the more traditional, Platonic principle that the 'masses' don't really know what they need to know, so that must be left to experts."

Yes, but this is wrong.

Anthony said...

"If Wikipedia manages to destroy the existing, albeit imperfect, market for supply of knowledge, then Wikipedia will have managed to destroy knowledge entirely. This is where Anthony is coming from, I suspect."

I'm talking about the GFDL, and CC-BY-SA, and the anti-copyright movement in general. The marketplace for knowledge is a marketplace. Intellectual property serves as the property. Price serves as the price mechanism (though increasingly through the indirect mechanism of advertising).

In Wikipedia, copyright is replaced with copyleft, and price is replaced with free (and ad-free to boot).

In a free market such an altruistic enterprise would probably not survive. But Wikipedia has an important set of advantages over other market participants. Not only do their revenues come in tax free, but donors are rewarded with a tax deduction.

William M. Connolley said...

What I meant was, there is no per-article marketplace. Britannica has no means to reward those who write the most popular articles. I thought this was what your whole post was about?

If you're talking about a whole-encyclopaedia market place, then yes there is one and Britannica dropping the print version whilst wiki manages to fund raise enough for its needs gives you a hint of the results.

Edward Ockham said...

Agree with you on the copyright thing. As you know I talked to Stallman, who originally held that ownership rights only apply to material objects, and not to a 'form' which can be replicated onto many different materials, in the way that a ring can make the same impression on different bits of wax. He felt that the mere replicability prohibits ownership – prohibits in the ethical sense, I mean. When I pressed him on this, he was not so sure, because he could see there is no incentive for an artist to produce work otherwise. His position is now a bit softer, and he only applies the principle to artifacts which have to be modified in order to produce further value, such as source code. If the producer of the code keeps it proprietary, it is difficult for engineers to modify it to produce better code, he argues.

>> Wikipedia is a part of the marketplace which threatens to destroy the marketplace.

I found your statement enigmatic because I was reading the 'which' as referring back to 'marketplace'. Does it actually refer back to 'Wikipedia'?

>>Yes, but this is wrong.

I.e. the Platonic principle that experts know what is best. Plato has the idea, a very famous idea, that most people live as though in a cave, impressed by defective images of reality. Even when they perceive reality, they are blinded by it as owls are blinded by sunlight, and so fail to recognise it. This is a powerful idea that has impressed men for thousands of years. Its very durability and power suggest that it should not be dismissed lightly.

Edward Ockham said...

>> Britannica has no means to reward those who write the most popular articles.

My post was simply about supply and demand. Demand may be high, but supply may be high as well, and so cost will be low. The problem is where there is demand, but limited supply, in which case the cost will be high. Britannica has a way of meeting the demand for high cost articles by employing experts. Wikipedia has no way of meeting this demand, which explains why certain high-cost articles are poor on Wikipedia. 'Philosophy' e.g.

Anthony said...

"Britannica has no means to reward those who write the most popular articles."

Are you saying that Britannica pays the same amount for every article that is written?

I doubt it.

"If you're talking about a whole-encyclopaedia market place, then yes there is one and Britannica dropping the print version whilst wiki manages to fund raise enough for its needs gives you a hint of the results."

Well, Britannica dropping the print version is just a vote against the medium, not the content. But I think there is a futher war being waged against the content, and it's what I hinted at with my comment about Wikipedia threatening to destroy the marketplace. The anti-copyright movement is strong, even stronger than the anti-profit movement, which has significant overlap. I believe Hayek himself was anti-copyright, though obviously not anti-profit. Jimmy Wales seems to lean that way sometimes as well, though it's hard to read Jimmy Wales - he is so often seemingly self-contradictory.

Anthony said...

>> I found your statement enigmatic because I was reading the 'which' as referring back to 'marketplace'. Does it actually refer back to 'Wikipedia'?

Ah, yes, sorry about the ambiguity. :)

Anthony said...

>> I.e. the Platonic principle that experts know what is best. Plato has the idea, a very famous idea, that most people live as though in a cave, impressed by defective images of reality. Even when they perceive reality, they are blinded by it as owls are blinded by sunlight, and so fail to recognise it. This is a powerful idea that has impressed men for thousands of years. Its very durability and power suggest that it should not be dismissed lightly.

I haven't dismissed it lightly. I've dismissed it after much careful thought and consideration. I just didn't think this was the place to go into the details as to why he was wrong.

Anthony said...

I've had some interesting conversations, on the Creative Commons mailing list, about Stallman's view of copyright, and how he is primarily anti-copyright with regard to functional as opposed to artistic works. On the latter he is more nuanced.

Interestingly, I somewhat agree with him as far as copyright and software go. My belief is that software, as a primarily functional work, should be primarily governed by patent law, not copyright law.

Stallman, obviously, disagrees about the patent law part. :)

Edward Ockham said...

>>I've dismissed it after much careful thought and consideration.

I would be interested in your thoughts. (You can have a guest post, if you like).

Anthony said...

That sounds too much like work, and with no reward.

There's nothing unique or original in my thoughts on the matter anyway.

If you'd like to discuss things privately, you know how to contact me.