Wikia Search May Have Trouble Achieving Critical Mass

from the notability dept

Mathew Ingram notes that Jimmy Wales's company, Wikia, has unveiled a new version of its search engine. The basic premise of the search engine, allowing users to edit search results the way they can edit Wikipedia pages, is clever. But I think Wales is going to have difficult making the project successful. The fundamental problem, I think, is a matter of raw mathematics: there are far, far more potential web searches than there are pages in Wikipedia. Last month I critiqued the business model of Biographicon, a site that's attempting to create a Wikipedia-style page for everyone. I argued that they're likely to have trouble making it work because any given page is unlikely to have the critical mass of contributors necessary to make the wiki model work. I think Wikia's search engine is likely to suffer from an even more serious case of the same problem. Wikipedia achieves this critical mass by limiting itself to subjects that are "notable." But a search engine can't have those kinds of limits. People want a search engine to have good responses even for (maybe especially for) obscure searches. And by definition, it won't be possible to get a bunch of people to contribute to the page for an obscure search term.

Closely related is the problem of bias. Wikipedia strives to take a neutral point of view, presenting all viewpoints fairly and accurately without passing judgment on which one is correct. This often leads to pages being longer than they would otherwise be, but they tend to be reasonable representations of what various people think on the subject at hand. This approach won't really work with a search engine because people expect the most important search results to be at the top, and deciding which results are the most important is an intrinsically subjective decision. If Wikia's search engine ever became popular, it could be beset by edit wars that would make the infamous Danzig/Gdansk edit war look tame. Companies pay search engine optimization firms thousands of dollars to improve their Google ranks, a successful Wikia search would likely succumb to the same kinds of pressure, and the site appears to lack Wikipedia's well-defined procedures for resolving disputes.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: search, user generated, wikia, wikis


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    mobiGeek, 30 Apr 2008 @ 12:52pm

    Not so obscure

    Though I agree with the SEO problem (payment for "improved"/skewed search rankings), I don't buy the "obscure" point.

    I ran a large search engine for a few years and the number of "unique" search queries was ...er... unique. We had some test queries to test our indexes and result speeds, and even those arbitrarily weird queries eventually were not unique (20 or so pseudo-randomly chosen characters).

    At one point in my life (read: when I was younger, kid-free, etc.) I could see me spending time tweaking search results on "linux queries" and "developer queries", all in the hopes of improving the results for those communities. Done "right", this type of search engine could replace the need for manually created FAQs.

    link to this | view in chronology ]

  • identicon
    Joel Coehoorn, 30 Apr 2008 @ 1:15pm

    Hybrid

    A common problem with search engines is ambiguous terms. For example, if I search for 'Python' I might be looking for a snake or I might be looking for a programming language. A user-edited search could help solve this.

    I envision a hybrid system that has a Google-like engine under the hood. But if your search includes certain *notable* keywords or keyword combinations it could leverage it's users to first create a place where you tell the engine exactly what you mean by that term (if there's a conflict) and then rate pages that match user-selected results for your variety of that keyword as much more relevant.

    link to this | view in chronology ]

    • identicon
      Sean, 30 Apr 2008 @ 5:04pm

      Re: Hybrid

      I was going to say about the same thing I would love something like google but allow users to edit results to some degree. This way users can mark a page as spam or a repeat of the same article giving more speciffic information and removing most of the repeated content. Also having the ability to link results so if searching for "side effects of coffee" you could have to the side a suppliment with "bennifits of coffee".

      For people that add irrelevant pages and others can vote it down or report it and for results with conflicts someone can review it and decide.

      link to this | view in chronology ]

  • identicon
    Hua Fang, 30 Apr 2008 @ 2:22pm

    Coding the concept, not words, you will be there.

    This is about the point that I have brought along before, even with James's programmer (Jer). A word or words are not carrying specific meaning when they are standing alone. To get the full meaning of unknown texts, so called “unstructured contents", the most fundamental unit to measure one or more concepts must be created. I call it "Codon", "-LCP-" in short form. Then, any unknown contents will be searchable, at least in such theory named "Codonology".

    Anyway, I am trying to use current Wikia as the platform to start the Codonology project. Hopefully, the dream may come true in terms of true "Concept Search Tool", and reasoning tool as well.

    Hua Fang

    link to this | view in chronology ]

  • identicon
    Joel Coehoorn, 22 May 2008 @ 1:25pm

    SEO

    I've been thinking about this, and SEO is big enough business, SEO scamming enough of a problem, that I think they'll have too difficult of a time keeping it from being gamed.

    link to this | view in chronology ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.