Wikipedia:Version 1.0 Editorial Team/Wiki Sort
Project Overview edit
The WikiSort Project is a subproject of the Version 1.0 Editorial Team that aims to select articles for the proposed Wikipedia 1.0 by a collaborative process. From prior discussion, it has been determined that the most likely collaborative process would be some sort of numeric rating system. The primary goal of any rating system is the automatic selection of articles for releases of Wikipedia, 1.0 and beyond. Various systems have been proposed, one of which is detailed below.
Project Philosophy edit
Of the four subprojects of the Editorial Team, WikiSort is the only project that is scalable. A subset of editors manually validating articles will quickly reach a state where the rate of new material rapidly outpaces the rate of validation. However, by utilizing the same wiki principles that have allowed our beloved encyclopedia to grow, WikiSort will be able to succeed. Equally important to scalability is the fact that more editors will undoubtedly select articles from a greater range of subjects than a subset of editors. That is, selection bias decreases as the number of editors increases.
Proposed Rating System edit
Introduction edit
- The proposed rating system will endow each article with multiple, categorized ratings. The two primary ratings will be a scope rating and a comprehensiveness rating. The scope rating is a measure of the importance of a topic. This is analgous to Project Core Topics's goal of including articles in order of decreasing importance. The comprehensiveness rating is a measure of how well an article fulfills its subject matter and/or the overall quality of the article. This is analogous to Project Featured Articles First's goal of starting with quality articles and then sorting based on subject matter.
Benefits of Rating Systems edit
Automatic Article Selection edit
- The purpose of all rating systems is to automatically select articles for releases of Wikipedia. The software could select all articles above a given threshold for final vandalism/copyvio checking, which would relieve the burden of verification almost entirely.
Stub Elimination edit
- The software could also mark all articles below a given threshold as stubs, thereby eliminating a large source of 'work' that is currently done manually. This would also allow the large group of people currently dedicated to improving stub categories to instead focus on improving regular categories.
Advantages of a Categorized Rating System edit
Better Algorithms edit
- A categorized rating system is inherently better than a singular system for several reasons. The primary advantage of a categorized rating system is that much better selection algorithms can be developed. For example, there are quite a few articles on wikipedia that, while short, contain pretty much all there is to say about their respective topics. Such articles should probably be included in Wikipedia 1.0, but would not make the cut under a single rating system.
Expert Tag Elimination edit
- Whereas any rating system can eliminate stubs, a categorized rating system can also replace expert tags. If an article has both a low scope rating (that is, it is highly specialized) and it is of poor quality (or has NPOV/factual issues), the software can safely assume the article is in need of an expert.
Focused Editing edit
- A categorized rating system would greatly assist in the development of software that could identify one's "edit signature". That is, aided by this proposed system, software could be developed that would analyze a user's contributions, determine what kind of articles a user likes to edit, and suggest a list of articles that the user would potentially be interested in editing. Of course, such software will undoubtedly be imperfect, so Your mileage may vary. It is also worth noting that such software would greatly benefit the identification of vandals and sock puppets.
Disadvantages edit
- The only major disadvantage of a categorized rating system is that it will require (potentially major) software modifications. A rudimentary rating system will probably not constitute a major effort. Actually, a basic rating mechanism already exists (but is disabled) within the MediaWiki software. However, some of the more advanced algorithms will obviously require more significant changes. Said algorithms may also rely on the statistical analysis of data, which would dictate a phased implementation. The amount of effort put into the software will be directly tied to its benefits, but the main idea is that a phased installation will not require that work to be done all at once.
Vandalism edit
Any rating system (as all articles) is subject to vandalism. In order to limit the effects of vandalism, a short rating history can be kept and averaged. If a new rating deviates a given amount from the average, it can safely be considered vandalism and ignored. If a new version receives a worse rating, this will reflect poorly on the editor, and allow any "filtered" view to show the version with the better rating. Administrators may also reserve the right to override a rating they deem faulty.
Various advanced methods of vandalism detection may also be used. For example, in the event that an editor substantially improves an article in one edit, the software could detect this (through various means) and correctly allow the editor to augment the article's rating. Moreover, once a rating system is in use, the software may be able to analyze the data and develop various other rating methods that we could never predict beforehand. In short, vandalism will be a virtually non-existent problem.
Validation edit
- A rating system will ensure the right articles get chosen, but it does not ensure those articles will be free from vandalism. Due to the dynamic nature of Wikipedia, a fundamental problem of Wikipedia 1.0 is figuring out how to release a verified, vandalism free article when vandals can interject their handywork in the middle of the process. There exists two major ways to ensure this.
Individual Verification edit
- The first method is to have registered users verify each revision of an article as vandalism free. When the time comes to release 1.0, the software will choose the most recent verified version. The problem with this approach is that it will either result in articles that have undergone many revisions without verification, or else will require an unmaintainable effort.
Last Minute Fork edit
- Another method would fork the 'pedia at the last minute and have editors skim for obvious vandalism and copyright violation. This method would require less day to day effort, but would take longer to verify after the fork. Perhaps the best solution would be to verify revisions of articles, fork at the last minute, and simply present to editors articles whose most recent revision isn't already verified. This would speed up fork time to the range of hours or days, while limiting the need to verify each and every revision of an article.
The Third Option edit
- The last option would be to throw verification completely out the window! In all seriousness, the entire concept of Wikipedia 1.0 is that readers will be informed of the quality of articles. With a rating system, the software could simply insert a disclaimer at the top of articles informing the reader of their reliability. This method has the advantage of not excluding what may potentially be valuable encyclopedia content. Purists may cringe at the thought of stray vandalism permanently influencing the public's perception of Wikipedia, but this could very well be an option.
Other Benefits edit
Editor Rating edit
- A rating system could potentially be used to rate not only articles, but editors. The software could compare an article's rating before and after an edit, and change the editor's rating accordingly. Alternatively, users could rate other editors directly (perhaps on categorized criteria). The ideal solution would most likely incorporate a variety of methods. An editor rating would be very useful for a variety of things.
Notability edit
- As it stands, wikipedians who are well known are often given extra consideration in discussions. A rating system could assist that by providing a numerical value derived from an editor's contributions.
- Despite being looked down upon, a candidate's edit count is often used as criteria for adminship. However, a candidate's rating history would be far more representative of adminship criteria. If perfected, it may even be possible for the software to numerically determine a candidate's eligibility.
Expert Rating edit
- An editor could select areas where he or she is knowledgeable. Then other editors could rate that editor's level of expertise after reveiwing edit histories, etc. It may even be possible for the software to determine the expert rating. The software could compare an article's rating before and after an edit (assuming an improvement), and using its category determine that the editor is knowledgeable in that subject. Alternatively, if the (proposed) Wikiversity ever decides to include testing, those scores could be used to determine an expert rating.
Rating Articles edit
- An editor's rating could also be useful when rating articles. Editors with a higher rating could have their ratings weighted when contributing to an article (particularly if an editor is deemed an expert).
Implementation edit
While it may be a bit early to detail the implementation specifics, let it be clear that this rating system is designed to be as transparent as possible. For the collaborative advantages of WikiSort to appear, it must be very easy to use.
Some possibilities (few of these are mutually exclusive):
- MediaWiki
- Rubric (the engine behind de.lirio.us)
- Mozilla Firefox extension(s)
How You Can Help edit
The best way to help us right now is to visit the talk page and tell us what you think! We are always looking for new ideas, so have at it. If you like the concept, or would like to actively participate, feel free to add you name to one of the lists below.
Members edit
- Joel Kibitzer
- Titoxd(?!?) Kibitzer
- David Gerard Interested spectator and Wikipedia 1.0 obsessive