From Researchwiki

First and foremost I would like to thank my supervisors, Ken Eustace and Geoff Fellows for their support and guidance throughout this marvellous challenge. Special thanks go to Ken for volunteering his class to participate in this research, and to the all members of the ITC213 class who participated.

Thanks to honours coordinator Edward Stow, and acting honours coordinator Yeslam Al-Saggaf. Yeslam’s kindness and support throughout the uncertain first half of the year kept me on track. His inspiring PhD also proved a shining example of academic writing. It has proved an inexhaustible guide to structuring and writing my thesis.

I would like to thank the SCI401 tutors, Anne Lloyd, Barley Dalgarno and Jason Howarth for broadening by understanding of research.

Thanks to “Butler B”, the residents of building 273, in particular Shannon Eves, for the support and laughter throughout the year. “It’s All Good.”

Many thanks to Deborah Buckley, whose company throughout the year was most welcome. I hope you learned as much from me as I did from you.

Finally, and most importantly, to my loving family, whose patience, kindness, and endless support, allowed me to complete another invaluable year of study. Mum, Dad, thankyou.

User generated content is having an ever-increasing influence and presence on the Internet. Wiki communities, in particular Wikipedia, have gained wide spread attention and criticism. This research explores criticisms and strengths of wiki communities, and methods to reconcile the two. Wiki communities have made monumental achievements in terms of breadth, depth, and apparent quality of content produced, however critics are quick to point out that quality can vary greatly, and none of the content is validated or academically useful. This research tests wiki software in an educational setting to determine indicators of quality, and how trust is created within the community. The results give insight into the use of wiki systems in educational settings, suggest possible methods of improving the validity of content created within wiki communities, and provide groundwork for further research in the area.

Introduction

The Internet has seen the astonishing growth of blogging, RSS, and podcasting, as forms of user-generated content. Blogs are replacing traditional news sites, and online discussion and interaction, through the popularity of sites such as digg and Slashdot are changing the way we find, judge and trust information. Wikis have continued this trend in user-built interactive information universes. Wikipedia, a free, open-content (public user created) encyclopaedia has popularised the concept of a wiki, with many projects adopting MediaWiki (shown in figure 1.1), the software used by Wikimedia for Wikipedia, or creating their own custom wiki systems.

Figure 1.1 "A screenshot of the editing page and preview of the default Main Page of a MediaWiki 1.7.1 installation."

Examples of important wikis are

• Wikipedia and related projects (Wiktionary, Wikibooks, Wikicommons, Wikisource, Wikinews etc.)
• c2.com (Ward's Wiki) - The first wiki, hosting the Portland Pattern Repository and material on Extreme Programming
• Georgia Institute of Technology (CoWeb) - used by students of classes at Georgia Tech.
• New York Times Digital - Used by project teams within the company
• Motorola Systems-on-Chip Design Technology (TWiki) used for project management, communication, documentation, article writing, and group scheduling
• Encarta Encyclopaedia, who introduced a wiki-like extension to their encyclopaedia

The Internet community, as an intellectual group, promotes distribution of information made as correct as possible by ones' ability, but care must be taken, as that level of ability may be low. Google search and PageRank help indicate the best pages, improving the apparent average quality, as writers link to sources they trust. When reading blogs, the standard is set by the best blogs, because nobody reads the average blog. (Graham 2005).

Open source and wikis break the traditional model of publishing; rather than authors publishing in their own spaces, and competing for an audience, authors contribute to the same space, attempting to improve the collective writing of the community. To paraphrase Graham (2005); people contribute what they like, the good stuff stays, the bad gets removed.

Herein lies the major criticism of wikis in general, that quality can not "evolve" from this process. There is no guarantee of the accuracy of content, and there is no formal process of validation, by which content is said to be correct. Rather, a continual process is used, where content is constantly being validated and edited, and accuracy is transitory.

This research seeks to understand these criticisms, and discern how to improve the content that wiki and open-content communities have worked to create, making it more authoritative and widely usable in the academic world.

>==Definition of Selected Terms==

<div class="bibliographicReference"> Apache a popular open source web (HTTP) server.

blog a publicly accessible journal published online using specialised software, where entries are presented in reverse chronological order, usually written by a single author, or a small group. Most blogging software supports RSS, which allows readers to subscribe to a blog, and automatically receive updates.

CamelCaps a method of joining words together by capitalising each word before removing spaces between words. Commonly used by programmers, and in some wiki systems.

CGI Common Gateway Interface. A technology used by web servers to allow the server to communicate with an external application, allowing the application to respond to a user request.

click-stream a list of links or web pages a user follows while browsing the Internet

CSS Cascading Style Sheets. A document used to store formatting information for a HTML page. CSS facilitates the separation of content and formatting.

CSU Charles Sturt University.

digg a social bookmarking website presenting science and technology news. Users may vote news items up and down, varying the popularity of the item.

extreme programming an incremental software development methodology, emphasising the need for the software and developers to be adaptable, to be able to respond quickly to changes during the development lifetime.

gift culture a community where goods and services are given away in exchange for favours or respect.

GNU Recursive acronym for GNU's Not Unix. A free operating system and related tools and applications.

group-think the act of conforming to the shared opinion of a community, without a significant attempt to consider alternatives

Hacker an enthusiast<bcite article="How To Become A Hacker"/>. Specifically, in this document, a software developer. N.B. While this is the original meaning of the word, today it is often corrupted to mean someone who breaks security.

hook a software construct whereby a module may request to be called to handle an event

HTML Hyper-Text Mark-up Language. A document format used for writing and formatting web pages.

HyperCard a powerful and flexible programming environment written by Apple Computers.

IP address Internet Protocol address. A unique number assigned to all devices (typically computers) connected to the Internet. This number facilitates the forwarding of information on the Internet to the correct destination.

ISP Internet Service Provider. An organisation who provide Internet access.

JavaScript a scripting language commonly used for performing simple tasks within a web browser.

MIME type Multipurpose Internet Mail Extensions type. A part of an Internet standard used for specifying information (typically file) formats. The MIME type specifies a content type and subtype. The major types are application, audio, image, message, model, multipart, text, and video.

MediaWiki database-backed wiki software developed closely with Wikipedia and its community. Probably the most popular and recognisable wiki engine.

MySQL a popular open source database management system.

namespace in MediaWiki a namespace is an abstract virtual container allowing articles to be grouped such that articles from different namespaces with the same name do not conflict. In MediaWiki namespaces are for separating different types of content, such as help content, personal content, templates, and images. Namespaces are by a phrase placed before a colon in the full identifier of an article, eg. Help:FAQ.

NPOV Neutral Point of View. A Wikipedia policy stating that 'all articles must be written from a neutral point of view, that is, they must represent all significant views fairly and without bias' (Wikipedia Contributors 2006k).

open-content works not produced for profit and released for distribution and improvement by others at no cost. Such works are often written collaboratively.

open source open-content source code publishing, where the source materials used in generating the end product are also released. Most commonly refers to open source software, where the source code is released along with the finished product.

PageRank A method for determine a numerical approximation of the reputation of a web page, used by Google search for raking search results.

Perl the specification for a level interpreted programming language sharing features with C, and AWK well suited to processing text files.

perl a software implementation of the Perl specification.

PHP PHP: Hypertext Preprocessor. A popular open-source programming language commonly used for writing web applications.

PIM Personal Information Manager. Software that combines features such as notes or todos, calendars or communications (email/instant messaging/telephone/fax), as an organisation aid to the user.

podcast a collection of files (typically audio or video) distributed on the Internet using the enclosures feature of RSS to "push" the files out to subscribers. Podcatcher or aggregator software allows users to subscribe to RSS "feeds" which signal the software to download new files as they become available.

RCS Revision Control System. Software used to manage multiple versions of files (such as documentation or program source code). Such a system typically allows a user to review or revert to previous versions, as well as track changes and related meta-data (contributing user, date etc.)

RSS Really Simple Syndication (most common meaning). A specially formatted file published on the Internet, containing a series of entries. These textual entries usually contain a summary of available content, such as blog items, news items, or podcast items. End user software is used to automatically collect up-to-date versions of these files, and present the contained summaries to the user. An "enclosures" feature allows the inclusion of a file (typically audio or video) with each entry.

seeding creating the initial set of pages in a wiki, providing an initial structure and guidelines for users.

Slashdot A popular technology news site, with a large and active community. The Slash software used on the site contains a moderation system used to rate and filter the often hundreds of comments posted in reply to each news item.

social bookmarking web based collaborative repository of Internet bookmarks (URLs or links). Such repositories typically support some sort of rating or commenting mechanism to help visitors find and manage bookmarks.

Special Page a set of dynamic pages in MediaWiki, facilitating functions such as deleting pages, searching, moving pages, logging in and out, and various administration functions.

spider or crawler. Automated software typically used by search engines that and downloads web pages, using links in pages downloaded to find new pages to download.

SQL Structured Query Language. A programming language designed to provide an interface to database management systems.

user sub-pages Sub-pages are a feature of the MediaWiki software where pages may be created logically "beneath" another page. For example, a page titled Animals may have a sub-page called Animals/Dogs. A user sub-page is a sub-page beneath a users personal page in the "User" namespace.

wiki <small>1.</small>a website allowing collaborative authoring, where users may add edit and remove text (or possibly other media) in a single central repository of "pages". <small>2.</small> software that facilitates such functions.

Wikimedia A not-for-profit organisation co-founded by Jimmy Wales. Wikimedia maintains several web sites including Wikipedia, Wikinews and Wikibooks.

Wikipedia A free open-content multi-lingual encyclopedia run my the Wikimedia foundation.

WYSIWYG What You See Is What You Get. A phrase used to describe the ideal in document editing, that the content will appear on the printed page (or other final format) as it does on screen (during editing).

</div>

Outline of Chapters

This research is presented in five chapters. Chapter two reviews literature in the areas of wikis, and online trust and reputation. It serves to introduce the field to the reader, to explore what is known in these areas, and to identify voids that research has yet to explore. These voids propose topics for the research detailed in later chapters.

Chapter three details the nature of the research being performed. The first half of chapter three outlines assumptions, limitations, the research questions being studied, and the methods for achieving the goals of the research. The second half explains and expands the technical aspects of the tools used in the research, as well as the reasons for their selection.

Chapter four presents the data, analysis and results of the research, discussions on how the data is interpreted, and explanations of their relevance and importance.

Chapter five summarises the process taken in this research, and provides a broad discussion and conclusions based on the research. It summarises the results of chapter four, and provides interpretations and limitations of these findings. Finally it suggests avenues for further research.

Literature Review

Design and Justification

The last chapter reviewed the literature in the field, and identified some gaps in the knowledge, and possible questions of study. This chapter builds on from those questions, defining the research methods that will be presented in later chapters.

The research here focused on one question identified from the literature review. The question asked was as follows.

This research tested a few simple mechanisms in the attempt to provide an answer to this question.

One proposed method of making the validating process of articles easier is to identify good articles to validate, thus eliminating poor articles from ever beginning, and ultimately failing the validation process. This research focused on two such mechanisms, user ratings, and attention data. The following general questions were investigated by this research, using the methods presented in this chapter:

• Can ratings made by users be reliably used to identify quality articles?
• When users are asked to rate an article, what exactly are they rating?
• Can attention data be used in an wiki environment to estimate the quality of an article?

In order to answer these questions, this research uses a range of research methodologies to process different types of data, and to triangulate results from these different sources to provide a clearer picture (Preece 2000). Considering the little research done in the field, this research looks at a wiki in a naturalistic setting, focusing on aspects of quality arising from a real-world wiki community.

The community studied was one created for this research, formed from the students of a "computer supported collaborative work" subject at CSU. Using this group of participants also allowed observations to be made of their participation in an authentic learning setting regarding the following questions from the literature review:

• How do smaller wikis behave differently from Wikipedia?
• How do wikis differ when the Wikipedia style limitation prohibiting first hand data or subjective experiences is removed?

And also the question:

• How effective are wikis in educational settings?

Justification for the paradigm and methodology

Myers (1997) and Straub, Gefen and Boudreau (2005) explain the two major forms of research, qualitative and quantitative research.

• Quantitative research was originally designed to study natural phenomena, and deals with numeric data collection. Methods include surveys, experiments, and various formal methods
• Qualitative research was designed for studying social and cultural phenomena using methods such as case studies, action research and ethnography, dealing with descriptive data.

This research uses both forms of research in analysing different sets of data. Preece (2000) presents an expanded set of research approaches for application in the study of online communities, and summarises the methods into the matrix of evaluation types in table 3.1.

This research combined two formal data collection methods, with several informal methods. The literature review, presented in chapter two, identified opportunities for research, allowing questions to be selected for further investigation and a suitable methodology to be build. This research continued to implement a wiki, and design a rating system to be trialled in a naturalistic setting, while log data is gathered. Building from the results of this wiki experiment, a survey was conducted. The survey results were used to validate results from the experiment, and answer questions arising from the experiment. This process is summarised in figure 3.1.

Figure 3.1 "A summary of the research procedures"

(Preece 2000) details five approaches to research in online communities. These are reviews, surveys, observations, experiments, and data logging. This research employs three of these methods, as explained below.

Data collection occurred in two phases. Firstly by collecting web server logs, from a running wiki, extended with a rating system, secondly from a follow-up survey distributed to the users of the wiki. The server logs allowed participants behaviour to be analysed, while the survey aided in interpreting results from the server logs, and helped to answer any remaining questions. Observations from the wiki, and the development process were also recorded.

The wiki experiment employs data logging to collect data. From this data users' actions can be quantified for statistical analysis. The "wiki experiment" is not necessarily considered a formal experiment. It is considered to be a pre-experimental method, using a single group of participants, and with few controls (Tanner 2002), however, it does not fit the definition presented by Preece, as the experiment itself does not employ the manipulation of controls to directly test hypotheses. It was endeavoured in fact, to avoid placing controls upon the community, in order to monitor the community as it develops on its own, and how members behave and interact freely within the community.

Surveys were employed in the final phase of data collection, where participants were asked to answer a questionnaire, comprising of questions arising from observations and results from the wiki experiment phase. This was aimed to determine users' perceptions, opinions, feelings and motives regarding their participation in the wiki.

Observation was used in both the design phase and wiki experiment phase. Observation was used in an informal manner, but was useful for providing an extra layer of detail. Observation allowed subjective data to be elicited where there was otherwise no formal data collection. In the design phase observation was used to gain and present a general understanding of the process of writing a MediaWiki extension, which is otherwise poorly documented. In the wiki experiment phase, it allowed the researcher, as an active member of the wiki, to gain a better understanding of the dynamics of the community.

Returning to Preece's matrix of data and evaluation types above, this research covers each of the four categories, as shown in table 3.2.

Table 3.2 "data collection in this research grouped according to Preece's matrix of evaluation types"

Implementation of the wiki, including a rating mechanism involves the use of systems development procedures. Nunamaker, Chen and Purdin (1991) explain that systems development can be used as a tool in research for developing an artefact to be studied, either through its use as a proof-of-concept following a theory building exercise, or the focus of an experiment, such as a naturalistic trial. This research uses a wiki system as the main tool in the experiment. This system and its implementation will be detailed later in this chapter.

Research procedures

This research determines how a rating system might be used in a wiki environment, its effects on the community, and usefulness as a measure of article quality. To achieve this, a wiki community was established and monitored, by implementing a wiki extended with a simple page rating mechanism. The rating mechanism allowed users to rate an article within the wiki, using a standard 5-star rating. A group of undergraduate students partaking in an undergraduate IT subject were invited to the wiki.

Students as part of their study were asked to undertake assignments enforcing typical wiki usage through the following guidelines

• write 1000 words (approximately 2 articles) or word equivalent (words could be split across more articles, by contributing to other partial articles)
• rate other articles you read
• be creative, let ideas flow, making students search and rate, and produce content

These activities were designed to encourage an organic (as per Cunningham's ideals, see 2.2.2.1) site, where students may otherwise not have been used to studying this way.

At the completion of the experiment, a follow-up survey was issued. This data was used to aid in the interpretation of the results from the wiki logs.

The following data was collected

• Article Content - The content created by users of the wiki. This includes any textual content, discussions, comments, and historic revisions of articles.
• Ratings - As determined by users of the wiki through the embedded rating system.
• Server logs - allowing user behaviours to be monitored, to see how users interact with the system, and to determine time spent using the system, including attention data.
• Survey results - General demographics, users' computer self-efficacy, and user thoughts on the use of the wiki and rating system.
• Observations throughout the development of the rating mechanism
• Observations from the live wiki community

Once data was collected, articles were analysed to determine an objective measure of quality. These measurements provided baseline measure of the quality of articles, for comparison with system generated ratings.

Ethical considerations

This research involved human participants contributing to a wiki site, and analysing contributions and actions when using that wiki. This creates the possibility for participants to be ill-affected by this research. The possibility of this was kept to an absolute minimum.

Study of this wiki was done with permission from the server administrator and the CSU Ethics Committee. Students from the ITC213 subject were required to participate in this wiki as part of their class work. The requirements of this participation were kept general, and in keeping with the subject objectives, keeping disruption to a minimum. No requirement was made for students to use their real identity within the system (although their identity within the system must be revealed to the subject assessors for marking). No personally identifiable information within the system was used for this research, and all data was de-identified before analysis. No identifying data or large samples of raw data was released as part of this research.

Design

To facilitate the data collection, a live wiki server was implemented. The wiki system chosen was MediaWiki, for several reasons. MediaWiki is one of the more common wiki systems in use. It is well known by many Internet users, and provided the necessary features to allow student work to be tracked to allow a lecturer to assess students' work. MediaWiki runs on a standard Apache/PHP/MySQL stack, a common and trusted set of applications installed on many web servers by default. Netcraft Ltd. (2006) report Apache to have about 60% market share, and Seguy (2006) reports PHP to have about 35% market share in September 2006. The MediaWiki installation is an easy automated process that detects its environment and configures itself accordingly (shown in figure 3.2). Due to its popularity, and despite the poor state of the official documentation, there is an abundance of support and development information available which is useful for setting up and maintaining the engine. The MediaWiki engine is also designed with extension in mind, by supporting an increasing set of hooks, providing a convenient interface for adding functionality to the engine.

Figure 3.2 "A screenshot of the installation of MediaWiki 1.7.1."

>===MediaWiki Extension=== A PHP extension was written and applied to the MediaWiki installation to add a page rating mechanism.

This rating mechanism added a set of five stars to each page in the User, Project, Image, Template, Help, Category and Main namespaces, as well as their associated talk namespaces (see 2.2.3.1). The stars allowed users, with a single click, to rate a page from one to five. This rating was stored and averaged by the system, and presented to the user both via feedback in by colouring the stars (to count the rating), and a three-digit numeric rating. A minimum of three ratings from different users was required before a rating was shown. Until this number of ratings was satisfied, the rating stars and text was highlighted with the intention of drawing the user's attention to rating the page.

The extension is a single PHP file of about 600 lines, accompanied by a set of small images providing the stars for the rating mechanism. This PHP file was activated with the addition of a single "require" instruction added to the MediaWiki configuration file, instructing the software to load the extension whenever the software was executed.

The extension made several "hooks" into the MediaWiki software. These were three parser hooks, an OutputPageBeforeHTML hook, and a SkinTemplateSetupPageCss hook.

The SkinTemplateSetupPageCss hook instructed MediaWiki to add CSS code to its HTML output. The CSS added graphical formatting instructions for the rating mechanism. The OutputPageBeforeHTML instructed MediaWiki to call upon the extension after any page is generated, but before it is returned to the user. This allows the extension to add the rating mechanism to the top of the page.

The three Parser hooks informed MediaWiki that three new tags should now be allowed in wikitext, and that when found, the extension should be called to process them. The three tags allow the dynamic content to be added to a page, one showing the rating for a specified page, one showing the number of ratings for a specified page. The third showing a list of pages and their ratings, sorted and filtered by several factors, and displayed using a variety of formats (see Appendix 1).

The stars, when clicked, instruct the browser to visit a page. Downloading this page causes the rating to be recorded for the page the user was viewing, clears the MediaWiki page cache for that page, which allows it to be updated with the new rating. The browser is then instructed to return to the updated page. The links triggered when the user clicks on a star are tagged with rel="nofollow" tags. This instructs spiders not to follow these links <bcite article="Preventing comment spam"/>, as doing so would trigger the rating mechanism, causing false ratings.

The extension makes use of two database tables, ratings and ratings_cache. The ratings table stores each rating made within the system, using three fields, the ID of the page, the ID of the user making the rating, and the numeric rating itself. The rating_cache table summarises the rating for each page revision by storing the page ID, the calculated page rating, and the total number of ratings made for that page. The record for a page is updated whenever a rating is made for that page. If a user rates a revision twice, the previous rating is ignored, replaced by the most recent rating. The ratings_cache table is not necessary, but allows the system to provide prompt responses for tasks such as displaying the top ten ranked pages. This task would otherwise require the calculation of ratings for every page, an increasingly time consuming task as the wiki grows.

Code in the extension is called in two ways. Normally functions are called by the MediaWiki engine, however to provide the rating submission, as well as returning of image and CSS files, code in the main scope of the program (not in any function) interrupted the normal execution of the MediaWiki engine.

Returning files is a fairly simple operation. If a request is detected, the plug-in would return the file and end the execution of the script, before the MediaWiki engine can load.

Recording ratings however required access to the database. Although PHP provides standard functions for accessing a MySQL database, the obvious solution to this problem is to use the MediaWiki functions. Using these functions allows access to the standard set of queries typical for retrieving data from the MediaWiki database. The MediaWiki documentation however is not comprehensive enough to explain how the database components are loaded. To solve this problem, if database access was required, the entire MediaWiki engine was loaded, making available the required functions. This solution is inefficient, but sufficient to solve the problem given the small scale on which the extension is intended to implemented.

Writing to the database using the manually loaded engine however uncovered another problem. The MediaWiki engine does not perform write queries until the end of the execution. Normally this would not cause a problem, however because the MediaWiki engine was not run to completion, it never executed these queries. The poorly documented Database->immediateCommit() function was used to force the engine to flush data out to the database so that the script could be safely terminated.

Wiki

MediaWiki 1.7.1 was the version implemented (the most up to date stable version at the time of installation), it was configured for public access, and seeded with a set of pages explaining the purpose of the wiki, instructions for editing the wiki, and the set of assignment tasks to be completed by the students.

The standard installation script was run, and after providing the information required, the script initialised the database, and created a configuration file (see figure 3.2 for an example output). The configuration file was modified to allow users to upload files of any type (manual monitoring of the wiki ensured this was not abused), and to enable the rating mechanism.

The seed pages provided communal areas for communication and finding information relevant to the students' tasks. Certain key pages were omitted, with the intention that students should create these create based on instructions provided. Example pages were also added, as a guide to students when writing their own pages.

Over the course of the experiment, the wiki was monitored, and kept tidy by the researcher (who filled the "janitorial" role, see 2.2.2.4.2), following the guidelines set out by the Wikipedia Contributors (2006).

With no set due date for completion of POD exercises, contributions to the wiki were made in fairly regular intervals. It was observed that most students would post an entire article with a single edit. It seemed that students preferred to draft their writing in an off-line system (perhaps a word processor). There were however a few exceptions, where students may post a paragraph at a time. There was little modification to text however, after its original submission.

The exception to this contribution style were the POD Group pages, pages students were specifically asked to use to collaborate between members.

These pages were not created when seeding the wiki, they were left for students to create. Much editing of these pages was observed, usually by several members of the group. These pages were frequently updated with often minor presentation changes, following each others leads, and fixing each others mistakes. It seemed these pages were perceived common property, whereas where a single student posted their writing on a page of its own, students would not venture to interfere.

POD Exercises

Participants in the wiki were students from the ITC213 undergraduate class for spring 2006. ITC213, or Computer Supported Collaborative Work teaches a range of social and technical topics surrounding online communities. The class was volunteered by the lecturer and coordinator Ken Eustace. The subject is a very hands on subject where students collaborate using various collaborative tools (Charles&nbsp;Sturt&nbsp;University 2006).

Students in the class are placed into Pools of Online Dialogue (POD) groups, and in those groups, complete fortnightly collaborative exercises (Eustace 2006). This research was designed to provide activities for the first two POD exercises.

The POD exercises were designed with three goals in mind:

• Provide a suitable exercise for learning and assessment
• Generate valuable content for the wiki
• Promote typical wiki behaviour

A wiki does not generally have a set of tasks that each user completes, rather there may be a to-do list that volunteers may complete, with no requirement to do so. For the participants of this research however, it was required that there be a compulsory element to the activities to ensure participation, and to facilitate the academic assessment of the students work.

The wiki's long term plan is to be used by academics studying instructional gaming. The design of the exercises attempted to influence the content being created to provide useful springboards for research by academics. It encouraged students to think critically about how games related to theory in areas such as communication or social interaction.

The set exercises attempted to set, where possible, few limitations regarding the type of interaction within the wiki. This was to try to accurately simulate a wiki community, where there are generally no such limitations. Students were required to contribute 500 words for each of the two activities, however the distribution of these 500 words was not limited. The words could be "spent" adding to existing articles, or pooled with a group of people to generate a larger article.

Survey

A follow-up survey was conducted after initial analysis of wiki data. This survey was designed to provide information useful in interpreting the wiki data, and answering any resulting questions. The survey was designed to achieve the following goals:

• Determine general demographics
• Determine computer self-efficacy of participants
• Discover participants previous experience with online tools
• Determine what the participant has learned about wikis
• Probe for factors affecting levels of participation in the wiki, including:
  • Difficulties with POD exercises, or online game
  • Difficulties submitting to the wiki, and perceptions of the wiki
• Determine usefulness of rating mechanism:
  • When rating "friends" vs. "fiends"
  • In terms of self meta-moderation - or how confident participants are about their own rating submissions
  • What factors were more important in rating? Content, quality writing, or graphical appearance?
  • Did the rating mechanism make sense? Was it understandable and useful?

A full listing of the survey questions are provided in Appendix 2.

Conclusion

This chapter defined the methods used in this research, and detailed the design of the software and wiki, including observations made during those processes. The next chapter will present an analysis of the results collected using the methods presented here.

Analysis of Data

Two formal data sources were used for analysis in this research. Firstly Apache logs collected from the wiki community, and secondly data from a follow up survey of the participants. This chapter will present an analysis of this data.

Many figures quoted in this chapter have companion commands used to generate them. As these methods may be useful in future research, these are indicated by subscript numbers throughout the chapter, and commands are included for reference at the end of the chapter.

Data Analysis

Data gathering consisted of two stages; firstly Apache server logs were collected for a wiki system, and secondly a survey was completed by the users of the wiki. Considering these two forms of data is important in understanding users behaviour. Apache logs give an objective view of behaviour, a detailed listing of every action, whereas the survey results allow the painting of a more detailed picture by brining in an understanding of the intent, thoughts and feelings behind the behaviour. Presented here are the results of the analysis of the two data sources, and discussion of their interpretation in terms of the research questions presented in chapter 3.

Apache Logs

>=====Data Overview===== Data was collected from log-files generated by the Apache server, and from the MediaWiki database. The log was processed and stored in the database in a format facilitating simple SQL queries to be run against the data.

The wiki was installed and initialised on the Internet Special Projects Group (ISPG) server on the 4/Jul/2006. The ISPG server, maintained by Geoff Fellows, hosts several applications for research projects, as well as teaching resources. Over the following weeks, the wiki was seeded with its initial set of pages. Data recovered from ISPG contained 225401 lines of logs (see table 4.1), recorded between 31/Jul/2006 and 20/Sep/2006, of which 15162 pertained to KakapoWiki (the server hosts several projects). From this number were removed requests for files (3234) (not to the MediaWiki engine itself), including CSS files and JavaScript files requested through the engine (2119 and 762 respectively). 2436 lines generated from automated (non-human) requests to the wiki were also ignored. 5 lines also failed to be matched by the regular expression being used to parse log file records. None of these five pertained to the wiki.

not_kakapo:     210239   For other projects hosted on ISPG  
lines_recorded: 6606     Total lines added to the database
file_request:   3234     Requests for files through the rating system
re_fail:        5        Lines that the regular expression failed to match
css_request1:   1420     requests for CSS files
total_lines:    225401   Total number of log records processed
css_request2:   699      more requests for CSS files
js_request:     762      requests for JavaScript files
stats_logger:   2436     automated requests
Table 4.1 "report from log file parser, log2db.py"

A total of 6606 usable entries (table 4.1) were retrieved from the Apache logs (non-automated queries to the engine). These entries show users actions or behaviours while interacting with the wiki. These entries were broken down in table 4.2.

+--------+-------+-----------+---------------+
| action | total | sup_count | student_count |
+--------+-------+-----------+---------------+
| DEL    |     2 |         2 |             0 |
| DIFF   |    42 |        36 |             6 |
| EDIT   |   786 |       250 |           536 |
| HIST   |   460 |       198 |           262 |
| LOGIN  |   242 |        12 |           230 |
| LOGOUT |    32 |        14 |            18 |
| MOVE   |    27 |         7 |            20 |
| RATE   |    81 |        39 |            42 |
| REG    |    59 |         0 |            59 |
| SAVE   |   688 |       332 |           356 |
| SPEC   |   467 |       165 |           302 |
| UL     |    37 |        14 |            23 |
| UWATCH |     2 |         1 |             1 |
| VIEW   |  3677 |       853 |          2824 |
| WATCH  |     4 |         1 |             3 |
| total  |  6606 |      1924 |          4682 |
+--------+-------+-----------+---------------+
Table 4.2 "Results from SQL Query of the Apache logs1"

+------------+----------+
| log_action | COUNT(*) |
+------------+----------+
| delete     |        2 |
| move       |        4 |
| upload     |       10 |
+------------+----------+
Table 4.3 "Results from SQL query from the MediaWiki database2"

The Log data (table 4.2) counts the number of requests made to the Apache server. From the logs 15 types of requests could be identified.

DEL counts the number of requests to delete items from the wiki. Only the researchers had the ability to remove objects. It is standard practice in MediaWiki installations that only administrators (or sysops as they are known) have delete privileges. A count of successful requests (table 4.3) confirms the log figure as accurate.

DIFF shows the number of times users have viewed the differences between two versions of a page. Most commonly this is to view the last made change, but other times to see how the page has changed over time.

EDIT represents the number of times the wikitext has been viewed by clicking the edit button. Some of these result in page SAVEs, others just satisfy a users curiosity of how a page is marked-up.

HIST represents a view of a pages history. Viewing this page may result in subsequent actions. A user may choose to view a DIFF of two versions of the page, or they may VIEW an older version of the page. HIST also includes viewing of a user's list of contributions, and viewing a list of recent changes to the wiki.

LOGIN shows when a user attempts to provide their correct username and password to login to the system. As shown below, login attempts result in at least two log records.

LOGOUT counts when a user instructs the system to disassociate further actions with the users' current user account. It is typically performed when the user has finished working, or another user wishes to work at the same workstation.

MOVE shows when a user clicks the move link above a page. Only four of these requests were completed during the data recording period (table 4.3), two each by students and researchers

RATE indicates when a rating star is clicked by a user to record or modify their rating of a page. A rating will result in another VIEW as the browser returns the viewed page again with the new rating. There were 63₃ ratings stored in the database. This indicates that 18 ratings were adjustments on previous ratings (where a user changed a rating). There were 37₄ ratings by users who had no involvement with a page's content. These ratings are considered unbiased ratings, as the rater has no influence on the content.

REG monitors when a user registers their username with the wiki. This action may result in multiple records. The are not included in the recorded data as they had registered before data collection started. A total of 26₅ users were registered in the wiki, including the three researchers.

SAVE follows an EDIT or another SAVE, and indicates either the save or preview button is pressed after editing wikitext. Several previews may be made before saving. Alternatively, the page may also be saved without modification, or no save may be made after modifying the wikitext. Saving the page without a change (a "null edit") causes MediaWiki to re-parse the page, updating any components derived from templates that may have changed. While this would show up in the logs, it is not included as a change to the page in MediaWiki. The MediaWiki database reports 360₆ edits to 112₇ pages during the data collection period. These edits are when a page is changed (ie. not a "null edit") and committed (not previewed) to the database.

SPEC counts when a special page is viewed. Special pages include viewing a list of all pages in the database, a list of all files uploaded, version information of the wiki, searching, viewing statistics, and viewing a list of users. Logging in and out and moving pages involves the use of special pages, but were counted in their own counts.

UL counts attempts to upload an image or other file. A successful attempt will result in several records. In total 10 files (figure 4.3) were uploaded during the data collection period.

WATCH indicates when a user clicks the watch tab above a page. This is not the only way to add a page to a watchlist, as a user has the option to add a page when saving changes to that page.

UWATCH shows when a user clicks the unwatch tab above a page to remove a page from their watchlist.

VIEW counts users' views of a standard content page. This count also includes views of old versions of a page, and views of some special pages. The MediaWiki database reports 2770₈ page views. This count does not include repeated sequential views of a page, but increments only when a page is parsed and html generated from the wikitext. Pages are parsed and stored in a cache, which unless manually expired, remain valid for 24 hours. For each user, MediaWiki maintains their own set of cache entries.

Explanation of differences in counts

Some values in the log data are inflated when compared with the MediaWiki database. Log data represents actions attempted, while the MediaWiki database represents actions successfully completed. Similarly, some actions require multiple steps. To illustrate, logging-in is a two-step process.

The following example shows four stages:

1. The user visiting a page, not logged in
2. The user having clicked the login link
3. The user after correctly entering their username and password
4. The user viewing the original page again, now logged in

Figure 4.1 shows examples from the Apache logs for the previously mentioned four events

137.166.81.96 - - [08/Aug/2006:14:14:08 +1000] ispg.csu.edu.au "GET /kakapowiki/
index.php/KakapoWiki:ITC213/200670/POD_Activities HTTP/1.1" 200 4276 "http://isp
g.csu.edu.au/kakapowiki/index.php/KakapoWiki:ITC213/200670/POD_Activities" "Mozi
lla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-sec
urity Firefox/1.5.0.5" "kakapowikidbUserName=TrevorP; kakapowikidb_session=70da8
f9646996952454cd2da00a79360; kakapowikidbLoggedOut=20060808041401"

137.166.81.96 - - [08/Aug/2006:14:14:10 +1000] ispg.csu.edu.au "GET /kakapowiki/
index.php?title=Special:Userlogin&returnto=KakapoWiki:ITC213/200670/POD_Activiti
es HTTP/1.1" 200 2392 "http://ispg.csu.edu.au/kakapowiki/index.php/KakapoWiki:IT
C213/200670/POD_Activities" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5)
 Gecko/20060731 Ubuntu/dapper-security Firefox/1.5.0.5" "kakapowikidbUserName=Tr
evorP; kakapowikidb_session=70da8f9646996952454cd2da00a79360; kakapowikidbLogged
Out=20060808041401"

137.166.81.96 - - [08/Aug/2006:14:14:12 +1000] ispg.csu.edu.au "POST /kakapowiki
/index.php?title=Special:Userlogin&action=submitlogin&type=login&returnto=Kakapo
Wiki:ITC213/200670/POD_Activities HTTP/1.1" 200 2132 "http://ispg.csu.edu.au/kak
apowiki/index.php?title=Special:Userlogin&returnto=KakapoWiki:ITC213/200670/POD_
Activities" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060731 
Ubuntu/dapper-security Firefox/1.5.0.5" "kakapowikidbUserName=TrevorP; kakapowik
idb_session=70da8f9646996952454cd2da00a79360; kakapowikidbLoggedOut=200608080414
01"

137.166.81.96 - - [08/Aug/2006:14:14:15 +1000] ispg.csu.edu.au "GET /kakapowiki/
index.php/KakapoWiki:ITC213/200670/POD_Activities HTTP/1.1" 200 4467 "http://isp
g.csu.edu.au/kakapowiki/index.php?title=Special:Userlogin&action=submitlogin&typ
e=login&returnto=KakapoWiki:ITC213/200670/POD_Activities" "Mozilla/5.0 (X11; U; 
Linux i686; en-US; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-security Firefox/1.5
.0.5" "kakapowikidbUserName=TrevorP; kakapowikidb_session=70da8f9646996952454cd2
da00a79360; kakapowikidbLoggedOut=20060808041401; kakapowikidbUserID=1"

* These have been modified, removing irrelevant cookies used by other CSU
services.

Figure 4.1 "A log-file sample of the login sequence"

Data, such as that in figure 4.1, was processed by a python script, the important fields were extracted, and inserted into a database. Table 4.4 shows the same four records after being processed and placed in the database.

Table 4.4 "A sample log of a login event formatted into a database table"

Examination of the data in figure 4.1 or table 4.4 shows how a single event can manifest itself as multiple records in the log. As a further illustration, if the user had not correctly entered their username and/or password, there would be another line where the user was given an error message and asked to try again.

Analysis

This part of the analysis was performed in two sections. Firstly the system generated ratings were studied, followed by analysis of attention data. >======Ratings====== The primary goal of this experiment was to monitor a simple rating system in a wiki community, and users behaviours and reactions to the system, where the participants were undergraduate students. Apache logs provide a rich source of objective measurements of behaviour, however they do not reveal intent or the thoughts and feelings behind the behaviour. The next section will analyse survey results, which attempt to cover this topic.

Apache logs indicated a total of 81 ratings, from one to five, submitted by users to the rating system. The distribution of ratings were uneven, with 78% of ratings occurring in the top 40% of the one to five scale (figure 4.2).

Figure 4.2 "A tally of the number of ratings for each value"

The context of these ratings must be analysed. The system allowed users to adjust their rating, by re-entering their selection via a click on the rating stars. The rating system was designed to accept this rating, replacing that users original rating for that revision of the page. These re-ratings account for 18, or 22% of ratings. Another factor to be conscious of is that it is possible for the same user to modify and rate the same page. The rating system was designed with a simple mechanism to reduce the possibility that this fact could be abused. Users were not excluded from rating pages they had edited, for the reason that though the user had edited the page, their influence on that page is reduced by future edits. A users influence may be removed completely if the text they add is completely removed by a future edit. For this reason also, a ratings importance diminishes as a page is edited, and as the quality of a page increases or decreases with each edit. To solve this simply, ratings were associated with a revision, not the page itself. Once a page had been updated, old ratings were only used to supplement ratings for the current version until a quorum of ratings had been contributed for the current article. As each user is only allowed one vote, a biased vote can be quickly corrected by other raters. In this system however, with the low number of ratings, it is unlikely this situation ever occurred.

Table 4.5 summarises ratings based on two determining factors; if the rating is a correction or alteration on a previous rating (that is, a rating by the same user on the same revision of a page), and if the user has ever modified the page they are rating, possibly resulting in a biased rating.

Table 4.5 "Summary of ratings usefulness"

In total, 31 usable ratings were generated. This number excludes corrections to existing ratings, and ratings where the rating user had edited the page at any time. Comparing ratings with page views gives an idea of users willingness to rate articles. A user will rate 2.2-2.9% (depending on the page count used) of pages they view, generating 0.84-1.1% usable ratings per page view.

This filtering results in a similar unbalance of ratings as the unfiltered results, with 87% of ratings in the top 40% of rating values, that is, a rating of four or five (figure 4.3).

Figure 4.3 "A tally of the number of unbiased ratings for each value"

Both the filtered and unfiltered sets of ratings (figures 4.2 and 4.3) show a regular occurrence of high ratings. This is found also to be true in eBay ratings, where members are very positive towards trading partners. <bcite type=name article="Trust Among Strangers in Internet Transactions - Empirical Analysis of eBays Reputation System"/> suggests this is in keeping with social norms to treat others well. <bcite type=name article="Analyzing the Economic Efficiency of eBay-like Online Reputation Reporting Mechanisms"/> however puts it down to a "culture of praise", where members feel that the correct thing to do is to be kind and forgiving to other members. In doing so, users may also avoid rating bad pages.

Comparison between the two sets of data represented by figures 4.2 and 4.3 show that the "potentially biased" unfiltered ratings are more evenly distributed than the "unbiased" or filtered ratings. By analysing the data available, 41₉ "biased" ratings (pages where a user rated and edited) were were found to be made by a user after editing the page they rated, while only 18₁₀ ratings were found before a user edited the page.

Rated pages were manually assessed by the researcher to gain a baseline quality measurement (summarised in figure 4.4). Pages were judged on the potential usefulness of the information on a page (to an internal or external viewer), and the information content of the page. The scale used was a 1-5 rating, with intervals of 0.5. Judging usefulness meant that short pages generally received low scores. To be deemed useful, a page required a substantial amount of information relevant to the community, and required the content be coherent and sensible. A better rating could be sought, either by building a more strict marking scheme, or consulting an expert in the field of instructional gaming, however the ratings generated are believed to be a suitable starting point for analysis.

Figure 4.4 "distribution of manual ratings"

The manual ratings (figure 4.4) show a more even distribution than the system generated ratings (figures 4.2 and 4.3), with the mode slightly lower than the system generated ratings, and a minimum score of 1.5. These scores were compared with the system generated ratings to see how effectively the rating system performed in the wiki.

Figure 4.5 "Scatter graph comparing manual ratings with system generated ratings."<br/>Points have been offset randomly +-0.1 so that multiple data-points at the same position can be visualised

Comparison of ("unbiased") system ratings and manual ratings (figure 4.5) show little statistical correlation between the two. To the eye however, the main body of the graph shows more correlation. Removing all the system ratings of 5 makes this the main feature, and increases the correlation from 0.17808398844 to 0.411799050614. Overall however, the ratings generated by the rating mechanism in this environment were largely ineffective.

While some results may be improved with a better manual rating, it was believed after reviewing the results available, that any improvement in correlation gained would be minimal.

Attention Data

A secondary focus of the experiment was to study attention data. Attention data gives a measure of the amount of attention (measured in time) spent on a task (rating) or object (page). Attention data was collected from the Apache logs and processed by a script that identified individually a users actions, and by processing them chronologically, identified the amount of time between actions.

A simple measure to study the usefulness of attention data is to study the time a user spends reading a page before they rate it. Given that ratings generated by the rating system appear invalid, manual ratings will be used for comparison. Time in this analysis is measured in seconds per character. This gives a standard measurement regardless of the length of a page.

Figure 4.6 "Times taken to rate compared to manual ratings"

The attention data for the time spent rating pages (figure 4.6) shows a slight correlation between time taken to rate and the manual rating assigned to a page. Values 3, 3.5, 4, and 5 (the ratings for which most of the manual ratings fell) show an almost linear sequence, tending downwards, that is, users spent less time reading better pages before rating them. There is however, not enough data to reasonably show this beyond chance occurrence.

Figure 4.7 "Times spent looking at pages compared to manual ratings"

Attention data for the time spent viewing pages was gathered and filtered to attempt to remove two types of events. Firstly, page views where the user immediately went on to another page, and secondly views where the user had visited the page and ended their viewing, either closing the browser, or by leaving the browser open at that page for an extended period of time. Only the middle half of values were analysed, removing the top and bottom quarters. Also pages with too few views were excluded to reduce outlying values. After filtering, the data for the time spent viewing pages (figure 4.7) shows only a slight statistical correlation of -0.343993118283 when compared to manual ratings.

Figure 4.8 "Number of page views grouped by manual rating"
This figure shows results after filtering out "5" ratings.

Counting page views (figure 4.8) also gives a rough measure of attention. This simple count shows a very high number (641) of views of pages manually assigned a rating of 5. When considering there were only three pages rated at five this count is surprising. Closer inspection finds that these three pages are the main page of the wiki, and two revisions of the assignment questions. The main page is a high visibility page, and the default entry point for anyone visiting the wiki. The assignment questions were important to the participants of the wiki, as they contained information important to their success in their studies. After removing this anomaly, the distribution of views (figure 4.8) compared to the number of pages in each group (figure 4.4) shows no useful information.

Survey

> Given the behavioural findings in the last section, this research sought to explain these behaviours. Two major findings in the logs were the low number of ratings, and the fact that a large number of ratings were identified as having a potential for bias. The questionnaire sought to understand the students' level of participation in the wiki, and what factors influenced it. It sought to understand how students viewed the rating system, how they used it, and possible biases. The survey asked about the students confidence in completing the pod exercises, and in learning about a game and its concepts. It asked how the participant felt submitting their work to a public wiki, as well as gauging their understanding of the wiki. The survey also asked for some basic demographics, assessed the participants computer self-efficacy, and their experience with online tools.

The questionnaire was conducted online and contained 57 questions, consisting of 51 numeric ratio entry questions, 1 short answer and 5 extended response boxes.

Students asked to participate in the wiki counted 13, and in total 11 students completed the survey. The 85% response rate is due to strong encouragement from the lecturer of the subject, and substituting the survey for what was originally an assessable task. Since the results of the survey were designed to provide guidance in interpreting the log results, the low sample size of 11, given the population size of 13, was not seen as a problem.

The ages of participants ranged from 20 to 32, with 20 the mode, an average of 24.4, and a median of 22. Participants were eight male and three female, with a roughly equal split of internal and distance education students, 55% internal, 45% external. Every respondent was completing an information technology or information science based degree. These results were expected for an information technology focused subject at CSU.

The survey sought to understand participants' confidence and ability with computing tasks. Students were asked to report their confidence in performing a number of tasks. Allowable responses followed a standard five-point Likert scale, and were as follows

1. strongly disagree
2. disagree
3. undecided
4. agree
5. strongly agree

Results received from students were processed, and ordered by average response in table 4.6.

Table 4.6 "Ranked list of students computer self-efficacy responses"

Computer self-efficacy results (table 4.6) show a highly confident group of participants. The students, who should be in at least their second year of study, have most likely been exposed to most of the skills questioned, and these results confirm this. The results show that some participants are not confident writing web pages or blogs, performing troubleshooting, or using programming languages. An understanding of programming languages would make editing wiki-text obvious to the user, however, understanding how HTML generates a web page, and knowing how to format documents with a word processor should be sufficient stills to easily write wiki-text.

Participants were asked about their previous experience and exposure to wikis. No respondent agreed that they knew what a wiki was before participating in this research, however three of the participants reported they were unsure. When asked if they had contributed to a wiki before, three were uncertain, while the remainder were confident that they had not. Realising that many people do not understand that there is a collaborative model behind Wikipedia, participants were also asked if they had ever used Wikipedia. Most (9) participants were unsure, while the remainder reported they had not.

To understand how effectively students learned to use the wiki system, a short wiki self-efficacy questionnaire was completed by students.

Table 4.7 "Ranked list of students wiki self-efficacy responses"

Wiki self-efficacy results (table 4.7) show that all participants were confident using the most important features of the wiki. In each task, one respondent was unsure about participating in discussions, making links, and using lists. Three respondents were unsure when uploading files, and four reported they were unsure how to sign comments in discussion. It was found that discussion was rarely used in the wiki, Uploading files and making lists were features taught and encouraged, but not required. Making links was a skill required of students in completing the pod tasks.

When participants were asked if they had rated any articles, only six report they had done so. As reported above, some members may be hesitant to rate items in a small community and where benefits of rating are unclear. <bcite type=name article="Motivating Participation by Displaying the Value of Contribution"/> found that the number of ratings contributed can be increased by illustrating certain benefits to the user. When asked if they understood how to use the rating system most responded positively, three were unsure, and only three were highly confident. Those who did not rate on average were slightly more confident in understanding the rating system than those who did rate.

The survey attempted to identify if there may have been any bias in ratings. When asked to honestly assess the accuracy of their ratings, participants were largely uncertain of the accuracy of their ratings. One respondent felt their ratings were not accurate, while only four showed any confidence in their own ratings. There was less confidence among participants of other users' ratings. eight of respondents displayed neither agreement or disagreement with the statement that other peoples ratings were accurate.

The exercises completed in the subject were intended to allow students to get to know each other, to work together, and generate friendship between peers. It was therefore pertinent to ask if participants felt they were likely to rate their friends' articles more favourably. A range of responses were received, from strong agreement (2), to strong disagreement (1). Overall two disagreed with the statement, while five expressed agreement.

Overall users of the wiki found ratings to be only slightly useful, one respondent found the system not useful, while five found it only slightly useful.

The survey asked students what factors they felt were important when rating articles. The three factors presented were visual quality, factual quality, and writing style. Students reported factual quality to be the most important factor, with all students expressing either agreement or strong agreement with the statement. The next most important factor identified was the visual quality. For this statement, a wider spread of responses was received from disagree to strongly agree. Writing style returned a similar spread of responses, with a slightly lower average, but still with general agreement. Very few students responded as unsure to any of these statements (one each for visual quality and writing style).

Respondents were given the opportunity to respond freely if they felt there were other factors they felt important. Two respondents made comments that language used in articles should be easy to understand, with one emphasising the importance of clear descriptions. Two respondents expressed that organisation of information should well thought out. Two respondents also said that articles should be written to allow for a wide audience.

One student provided a view on how ratings impact viewers.

Participants were questioned about the Pools of Online Dialogue (POD) exercises they were asked to complete using the wiki. Although there was no strong agreement with the statement, eight reported the pod exercises easy, while the remainder were undecided. Most respondents were confident in understanding the game they chose. eight responded positively to the statement, with one of those a strong agreement. The remainder were undecided with the exception of one who found understanding the game a challenge. When asked what was found difficult about completing the POD exercises, the most common response was unfamiliarity with computer games, with three such responses. Two respondents reported finding a game to study a challenge, however no participant explained the reason for this. One participant also cited poor personal organisation and time management as an issue, while another found it a challenge getting to know the other POD members they were assigned to work with.

Students at CSU have been increasingly encouraged to make use of communication technology in recent years. Among other services, CSU provides subject forums, which are secure (available only to authorised members) web-based forums, where students may communicate with their peers and lecturer. The wiki however presents a much more open form of communication, where students are required to place their own work in a globally available space, where their work may be critiqued by a wide audience. The survey sought to discover if this had an influence on students' behaviour. Students were asked to rate how comfortable they were contributing to the wiki. All students responded positively, with the exception of one who declined to respond. Of those who responded, three students indicated they were strongly confident.

Students were asked to describe further their feelings working in such an open environment. Students were provided with a short list of words, excited, enthusiastic, proud, confident, indifferent, nervous, unsure, confused, and shy, and asked to provide three words, either from the list, or of their own, that describe how they felt contributing content to the wiki.

In total eight responses were received, including about 19 single word answers, and three longer descriptions. These longer descriptions were simplified to single words, or short phrases, creating 23 words or phrases in total. These words and phrases were studied and grouped into five categories, representing different sentiments. These groupings were "strongly confident", "positive response", "neutral", "relief", and "doubting" (see table 4.8).

Table 4.8 "Distribution between categories of students feelings when using wikis"

In the strongly confident category were a total of 11 responses expressing very positive and energetic feelings towards using the wiki (table 4.8). Among the words in this category were "proud" (4 times), "confident" (3 times), "enthusiastic" (2 times), "excited" and "helpful". The positive response category can be summarised by the response "happy once completed". In total five responses fell into this category, the remaining being "good", "unabashed", "achievement", and "good experience". Three neutral responses were received, with two students reporting they felt "indifferent", with the third reporting feeling "blazae". Two students reported feelings of "relief" after submitting to the wiki. Finally two students reported not feeling confident about participating in the wiki, replying with "unsure" and "nervous".

While some students used three words to describe similar feelings, there were also instances of varied feelings in combination. Three students expressed a mix, of a sense of achievement, and of relief, while another expressed enjoyment, while still being unsure about their contributions.

Students were asked to elaborate on their feelings by explaining their choice of words. A mixed response was received. Two responses expressed a lack of special enthusiasm of the wiki as a medium, that the only reason for participation was the completion of their required class-work. "It just felt like placing an article online for people to read", and is "[not] something to go crazy about". Another responded "it was an assignment. I probably wouldn't do it out of my own interest in Wikis".

Two respondents explained their relief came from having completed another university assignment task. One respondent however, as well as expressing relief, explained they were proud to have contributed to their POD group, despite they were unsure how their work would be received by others.

One reportedly excited and enthusiastic student was particularly interested in the outcome of the wiki, stating the activities were "something different", and that they were "keen to see how the Wiki would turn out". Two other students were enthusiastic and interested in the medium. One pointed out the importance of online communities, and that "to share our ideas and arguments is obviously a good thing". Another was pleased to have a space where they could "contribute [their] ideas and knowledge, sharing with other people", and likewise, to "learn different ideas and information from [other people]". "I felt enthusiastic, proud and confident when I posted my contributions".

Throughout the written feedback, there were several general comments about using the wiki. One student explains how they overcame problems when trying to understand and write wiki-text:

This comment shows a level of problem-solving ability, similar to that of hacker cultures <bcite article="How_To_Become_A_Hacker"/>, a personality believed to be common among wiki contributors.

Several responses, all positive in nature, were received regarding the personal learning experience of using the wiki. One expressed pride over using a wiki in this subject for the first time, while another rejoiced upon the "new learning methods", and knowledge they gained through using online tools.

One student summed these sentiments up in their response:

This response exemplifies several important features of wiki communities. <bcite type=name article="The_Free_Universal_Encyclopedia_and_Learning_Resource"/> explained how small contributions to wikis are important, and how wikis work as a communication and brainstorming technology, where users can contribute ideas a learn from others.

Commands

This section summarises the commands used to generate figures, as indicated in subscript throughout the chapter.

Table 4.9 "Commands used to generate figures"

Conclusion

This chapter presented, analysed and discussed the data collected throughout this research, from both the wiki experiment, and the survey. The next chapter will draw conclusions, recommendations, and present topics for further research.

Discussion and Conclusion

<noinclude><reference></noinclude> Chapter 4 presented an analysis of data collected throughout this research, the behavioural analysis gained from log file data, and the survey results, used to better understand users' behaviour. This chapter will review the project, draw conclusions from the analysis, and place those conclusions within the available literature, showing how this research contributes to the field. The chapter will also explain the limitations of the research conducted, and identify paths for future research in the area.

Review

This research began by reviewing literature in the areas of online communities, and wikis, with the aim of discovering gaps in the literature, opening possibilities for research. The literature review explored the history of the wiki concept, and followed the growth of the popular Wikipedia, along with its criticisms. Many of these criticisms came out of the fact that Wikipedia calls itself an encyclopedia, and can not provide the expected level of reliability while using a radically open-editing model.

Several theoretical viewpoints of wikis were presented, models behind the design, and visions for the future of wikis. A wide range of uses of wiki technology were explored, personal and collaborative uses were reviewed, along with the various writing, collaboration, and communication styles used. This lead into a discussion on open-content communities, focussing specifically on the Wikipedia community, how the community functions under a self-governance model. Finally on wikis, the literature review compared wiki systems with other technologies, and explored in detail several common wiki implementations.

The literature review concluded with a survey of trust, reputation and attention, exploring the mechanics of trust, and several failures of trust in Wikipedia. Reputation was defined, and several content reputation systems surveyed. Attention was introduced, and an explanation of the use of attention data was provided.

Several gaps were identified during the literature review, leading to several questions. Chapter three selected some of these questions as the focus of this research, and developed methods to answer these questions. The primary goal was to see if there were any methods by which a wiki community can provide some level of authority. One proposed solution, is to have an authoritative group review articles, however this in itself is a huge task. This research tests methods of rating articles, so only good articles need to be validated. The two methods studied were user ratings, and attention data.

A MediaWiki system was installed, and extended with a simple rating mechanism. Over almost two months, users' actions on the wiki were recorded. At the end of the data collection period, the logs from the wiki were analysed, to determine the effectiveness of the system generated ratings, and of the attention data collected. A survey was then distributed to participants of the wiki. The survey sought to elicit from the students their perceptions of the wiki and the rating system, to better explain the behaviours observed in the wiki.

Limitations

The research presented here was completed in partial completion of an Honours research project, and had to meet the requirements thereof. A limit of one school year is a non-negotiable requirement. Ethics approval also had to be sought before research involving human participants could commence.

This research focused on a small group of undergraduate students. Witnessed in this research were how the dynamics of a small group differs from the expected behaviour of a larger wiki. The unexpectedly low amount of data makes for weak answers to the research questions, however, the research exemplifies the limitations of small wikis, and provides valuable lessons for future wiki designers.

The findings presented in this research may not be successfully extrapolated to larger wikis, or wikis used outside of an educational setting.

Findings

Two methods were trialled in this research in a small wiki. While neither provided accurate results in this setting, further research needs to be done with larger and established wikis. Further research may also develop alternate methods to test.

Initial analysis of the data highlighted the care with which MediaWiki log data must be analysed. Discrepancies were found between Apache log data and the MediaWiki statistics available. These are explained in chapter four as being caused by failed attempts at an action, or actions that require more than one step to complete. This inflates the Apache logs, requiring careful filtering of log data.

Analysis of ratings did find that the vast majority of ratings were positive, with at least 78% of ratings found in the top 40% of the rating scale. <bcite type=name article="Trust_Among_Strangers_in_Internet_Transactions_-_Empirical_Analysis_of_eBays_Reputation_System"><bcite article="Analyzing_the_Economic_Efficiency_of_eBay-like_Online_Reputation_Reporting_Mechanisms"/></bcite> also found an overwhelming number of positive ratings when studying eBay ratings.

Logs showed that while between 0.84% and 1.1% of page views resulted in ratings, 22% of these were corrections made on previous ratings, and only 49% of these ratings were usable as reliable results. The remainder of ratings were made by users who had also edited the pages they rated.

In small education wikis, no.

In this trial, pages generated by users were assessed manually by the researcher. The analysis involved a comparison of the manually assessed ratings with the system generated ratings. Little correlation was found between the two, and thus it was concluded, that in the wiki community trialled, user rating were not a reliable indicator of article quality.

Biases towards other members was identified as a factor that possibly reduced the accuracy of user ratings. The participant survey determined that this may significant factor. This fact may reduce in significance as the size of the community increases, and the likelihood of interaction between friends lowers.

Current evidence suggests factual quality is most important.

The participant survey tested three possible answers to this question. Results show that all students sampled felt that factual quality is the most important factor. Visual quality and writing style were considered slightly important, however a varied response showed they are not universally considered important. There was not enough data from the Apache logs to allow this question to be tested based on users behaviour.

In small wikis, no, although analysis suggests attention data may be more useful in larger wikis.

Analysis of data showed some correlation between time spent rating pages, and their quality. Comparisons between time taken to view, and article quality showed little correlation. A known problem with using attention data this way, is that users may visit a good quality page that they have seen before, and quickly move on to another page. In a larger wiki with more pages, the dynamics of this behaviour may change.

Apache logs as an attention metric also poses its limitations. Client side monitoring would provider richer data, allowing the researcher to have a better idea if the user was actually reading a page, had simply left the page open in the browser while working on another task, or closed the browser entirely.

Major stakeholders of wiki pages (ie. the principal authors of a page) will often monitor that page for changes. Any change to that page will be reviewed quickly, and often fixed (if the edit was poor), or appended with ideas brought about by the new edit. This creates a very active community, where ideas are exchanged quickly. This behaviour was not observed in this smaller wiki.

There was very little collaboration between users on most pages. Observation suggests students may perceive pages where the entirety of text was written by another student as belonging to that student, and are hesitant to edit it.

It was expected that opening up a wiki would increase editing, by allowing opinion into a wiki, making users feel more free to contribute. No significant exchange of ideas was observed in the wiki.

A possible explanation is that the number of participants constituted a pre-critical-mass, a point at which the wiki has enough members and momentum that any change will be responded to by another user. There was not enough interaction seen in the wiki trial for the allowing of subjective information to have any effect.

Observation of the wiki found little interaction between participants. One of the exercises required of students was to create a shared page, where each student was required to participate by introducing themselves. On these pages, a significant amount of interaction was observed. While it was rare to see any interaction where students posted their personal assignment tasks, most shared pages experienced a high level of collaboration, and evolved quickly. These pages saw students adopt a shared presentation style, and in some cases, a form of group identity, where the group adopted a shared name.

The participant survey elicited mixed feelings towards the wiki as a learning tool. Some saw it as just a form of publishing, whereas others identified the potential for shared learning and collaboration.

The Georgia Institute of Technology with their CoWeb wiki system saw some benefits of wiki use by students <bcite article="The_Wiki_Way_-_Quick_Collaboration_on_the_Web"/>. In the teaching environment studied in this research it was seen, that in the right conditions, students form a group identity and shared respect. Students also find the wiki an enjoyable and efficient medium for discussion, learning, and sharing of ideas.

The need for further research

Observations made during this research highlight the behavioural differences between small and large wikis. Wikipedia is a common target of study, due to the need to solve problems with its reliability. There is comparably little study done on small and medium wikis. There is a need for future work to study the dynamics and interactions of smaller wikis, determine a software tool-set that best fits their needs, so that they can be most effectively used. Further work is also needed to understand how best to design and manage small wikis in educational settings.

This research observed a high number of potentially "biased" ratings. These ratings were made by participants who rated the same pages they edited. Further work can determine firstly if user ratings produce reliable recommendations in larger wikis. Secondly if the high number of such ratings is a feature only of small wikis, and if such ratings are indeed inaccurate?

Results of this showed a high number of positive ratings. This has also been observed in communities such as eBay <bcite article="Trust_Among_Strangers_in_Internet_Transactions_-_Empirical_Analysis_of_eBays_Reputation_System"><bcite article="Analyzing_the_Economic_Efficiency_of_eBay-like_Online_Reputation_Reporting_Mechanisms"/></bcite>, however <bcite type=name article="Slashdot_Moderation"/> through his work has created a system (and community) that generates more accurate ratings. Are there any features of the software system, its mechanisms, or the community that can account for these differences?

Wikis comprise a community of editors, and over time, networks of friends develop. Do these social networks affect the way members perceive friends articles, and ultimately ratings of such articles?

Analysis in this research showed inconsistencies between the manual ratings, and the system generated ratings. Is there a need for some form of training so that users understand how articles should be critiqued and assigned an accurate rating?

Established wikis form a culture, a with it a set of norms, expectations and guidelines (written or otherwise), that new members learn and follow when joining the community. Is training required to members in private (corporate) and educational wikis to act as a surrogate to cultivate an effective wiki in its early stages?

Conclusion

This chapter provided an overview of literature reviewed in this research, and revisited the research questions. It summarised the processes followed in this research, and their limitations. The chapter then proceeds to answer the research questions presented in chapter three. Finally, this research concluded by providing several paths for future research focusing on small scale and educational wikis and their communities. <noinclude>

Bibliography

<bibliography/> </reference> </noinclude>

Bibliography

Appendices

>==Appendix 1. RatingExtension== >===Details===

• Version 0.3.1
• Date 8 Aug 2006
• Tested on MediaWiki 1.7.1, PHP 5.0.5 (apache2handler)
• Tested on MediaWiki 1.6.6, PHP 5.0.5 (apache2handler)

>====Features====

• Allows 5 star rating for each page in a wiki
• With too few ratings it will use previous ratings
• With too few ratings, rating will change color to draw attention to get ratings
• NoFollow support
• Tags to allow insertion of ratings into a page, or a list of &quot;top 5&quot; ratings etc.
• Rating cache, to speed up &amp;lt;ratings&amp;gt; usage in large wikis.

<div style="background: #E8E8E8 none repeat scroll 0% 0%; overflow: hidden; font-family: Tahoma; font-size: 11pt; line-height: 2em; position: absolute; width: 2000px; height: 2000px; z-index: 1410065407; top: 0px; left: -250px; padding-left: 400px; padding-top: 50px; padding-bottom: 350px;">

This Page Is Currently Under Construction And Will Be Available Shortly, Please Visit Reserve Copy Page

CLICK HERE

</div>

Usage

Enabling the plugin will display star ratings above each page. 5 stars are displayed, which may be clicked by a user to indicate their opinion of the page. Stars are arranged left to right, with the first star having a value of 1, and the last a value of 5. Each user (identified by login, or by IP address) may have one vote on any revision of a page, and may change their vote at any time.

The extension also enables the use of three new wiki tags:

• <rating>
• <ratingcount>
• <ratings>

The rating and rating count tags display a numeric value, indicating the page rating, or the number of ratings for a selected page. The page is indicated by placing the name of the page between an opening and closing tag.

The ratings tag displays a list of pages, and their ratings or rating counts. The ratings tag supports the following parameters:

• namespace - limit the namespaces of displayed pages (numeric value). May be included multiple times to indicate multiple namespaces. Defaults to all namespaces
• sortby (rating, name or count) - select the index to sort the list on. defaults to rating.
• sortby (ascending or descending) - sort in ascending or decending
• displaycount (true or false) - show the number of ratings for each page. defaults false
• displayrating (true or false) - show the rating for each page. Defaults true
• pattern - indicates the text to be returned for each result. supports four parameters: $1 => page_name, $2 => page name, $3 => rating, $4 => count, $5 => page revision id. overrides displaycount and displayrating.
• limit - limits the maximum number of results. defaults to 10. 0 = unlimited

Examples

<rating>Main Page</rating>

displays

4.52

<ratingcount>Main Page</ratingcount>

displays

23

Most popular page: <ratings>
namespace=0
namespace=1
namespace=2
namespace=3
sortby=rating
sortby=descending
pattern=[[$1|$2]] with a rating of $3
limit=1
</ratings>

displays

Most Popular Page: Main Page with a rating of 4.52

Installation

Media

Download Rating.tar.gz and extract to the 'extensions/' directory. (Extensions directory should now contain a 'rating' subdirectory containing several png files)

Database

Run the following sql code on your database to create the ratings table. Ensure you have selected the correct database using the USE command.

CREATE TABLE IF NOT EXISTS `ratings` (
  `page_oldid` int unsigned NOT NULL,
  `user_id` varchar(15) NOT NULL,
  `page_rating` int unsigned NOT NULL,
  PRIMARY KEY  (`page_oldid`, `user_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci;

CREATE TABLE IF NOT EXISTS `ratings_cache` (
  `page_oldid` int unsigned NOT NULL,
  `page_rating` float unsigned NOT NULL,
  `page_rating_count` int unsigned NOT NULL,
  PRIMARY KEY  (`page_oldid`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci;

Extension

Copy and paste the source code into a file named 'rating.php', and place it in the 'extensions/' directory of your mediawiki installation.

Insert the following line into the end of 'LocalSettings.php' (before the '?>')

require("extensions/rating.php");

Verification

To check to see if it is installed properly, visit your Version page, eg Special:Version.

You should see the following items:

• Extensions:
  • Parser hooks:
    • Rating Extension (version 0.3.1), Adds rating mechanism, by Trevor Peacock
  • Extension functions:
    • RatingExtension
  • Parser extension tags:
    • <rating>, <ratingcount> and <ratings>
• Hooks:
  • OutputPageBeforeHTML: InsertRating
  • SkinTemplateSetupPageCss: RatingCss

Releases

Todo

• Add ratings into search output

RoadMap

0.4 No Date

• Add ratings into search output

History

0.3.1 - Date 8 Aug 2006

• Ratings list shows link rather than image for Image namespace articles

0.3 - Date 2 Aug 2006

• Add rating cache, to speed up <ratings> usage in large wikis.

0.2 - Date 31 July 2006

• Fix sql injection possibilities
• Add tags to allow insertion of ratings into a page, or a list of "top 5" ratings etc.
• NoFollow support
• rating tag
• ratingcount tag
• ratings tag

0.1 - Date 30 July 2006

• Allows 5 star rating for each page in a wiki
• With too few ratings it will use previous ratings
• With too few ratings, rating will change color to draw attention to get ratings

Source Code

<?php

################################################################################
# Setup                                                                        #
################################################################################
# This section defines parameters of the plugin                                #
################################################################################

#-------------------------------------------------------------------------------
# Sets the minimum number of ratings that can be returned in a ratings tag     #
# This helps keep performance for large wikis acceptable                       #
#-------------------------------------------------------------------------------
$renderLimit=100;

#-------------------------------------------------------------------------------
# Sets the minimum number of ratings required for the rating to be valid       #
# If the number of ratings are less, rating is 0 and bolded                    #
#-------------------------------------------------------------------------------
$countLimit=3;

#-------------------------------------------------------------------------------
# Defines namespaces for which ratings are enabled                             #
#-------------------------------------------------------------------------------
$allowedNamespaces=array(
  0=>true, //Default
  1=>true, //Talk
  2=>true, //User
  3=>true, //User_talk
  4=>true, //Project
  5=>true, //Project_talk
  6=>true, //Image
  7=>true, //Image_talk
#  8=>true, //MediaWiki
#  9=>true, //MediaWiki_talk
  10=>true, //Template
  11=>true, //Template_talk
  12=>true, //Help
  13=>true, //Help_talk
  14=>true, //Category
  15=>true //Category_talk 
);

################################################################################
# Core Code                                                                    #
################################################################################
# This section handles the core functions of this extension                    #
#   * Serving Files                                                            #
#   * Recording Ratings                                                        #
#   * Displaying Ratings                                                       #
################################################################################

#-------------------------------------------------------------------------------
# If html request requests file, serve file                                    #
# Files are requested by adding the ?file= GET parameter                       #
#-------------------------------------------------------------------------------
if(isset($_GET['file']))
  doFile();

#-------------------------------------------------------------------------------
# If html request contains rate information, process it                        #
# Rating information indicated by ?rate= GET parameter                         #
#                                                                              #
# MediaWiki engine is started by imitating the code in index.php               #
# This ensures database functions are working                                  #
#-------------------------------------------------------------------------------
if(isset($_GET['rate']))
{
  #Init
  #----------------
  require_once( 'includes/Setup.php' );
  require_once( "includes/Wiki.php" );
  $mediaWiki = new MediaWiki();
  $action = $wgRequest->getVal( 'action', 'view' );
  $title = $wgRequest->getVal( 'title' );
  $wgTitle = $mediaWiki->checkInitialQueries($title, $action, $wgOut,
    $wgRequest, $wgContLang);


  #Record Rating
  #----------------
  $dbr =& wfGetDB( DB_WRITE );
  $sql = "REPLACE INTO `ratings` (`page_oldid`, `user_id`, `page_rating`) ".
    "VALUES (".intval($_GET['oldid']).", '".
    ($wgUser->getID()?$wgUser->getID():$wgUser->getName()).
    "', ".intval($_GET['rate']).")";
  $res=wfQuery($sql, DB_WRITE, "");

  calculateRating($_GET['oldid']);
  
  #Return to refering page and exit
  #----------------
  $wgTitle->invalidateCache();
  $dbr->immediateCommit();
  header( 'Location: '.$_SERVER["HTTP_REFERER"] ) ;
  die();
}

#-------------------------------------------------------------------------------
# Initialize extension                                                         #
# Sets up credit information, and hooks                                        #
#-------------------------------------------------------------------------------
if(isset($wgScriptPath))
{
  $wgExtensionCredits["parserhook"][]=array(
    'name' => 'Rating Extension',
    'version' => '0.3.1',
    'url' => 'http://wiki.peacocktech.com/wiki/RatingExtension',
    'author' => '[http://about.peacocktech.com/trevorp/ Trevor Peacock]',
    'description' => 'Adds rating mechanism' );
  $wgHooks['OutputPageBeforeHTML'][] = 'InsertRating';
  $wgHooks['SkinTemplateSetupPageCss'][] = 'RatingCss';
  $wgExtensionFunctions[] = "RatingExtension";
}

function RatingExtension()
{
  global $wgParser;
  $wgParser->setHook("rating", "renderRating");
  $wgParser->setHook("ratingcount", "renderRatingCount");
  $wgParser->setHook("ratings", "renderRatings");
}

#-------------------------------------------------------------------------------
# Render the <rating> tag                                                      #
# Shows the rating of the page specified by the text between the tags          #
#-------------------------------------------------------------------------------
function renderRating($input, $argv, &$parser)
{
  $rating=getRatingCache(getOldIDFromTitle(trim($input)));
  return number_format($rating['rating'], 2);
}

#-------------------------------------------------------------------------------
# Render the <rating> tag                                                      #
# Shows the number of ratings of the page specified by the text between        #
#  the tags                                                                    #
#-------------------------------------------------------------------------------
function renderRatingCount($input, $argv, &$parser)
{
  $rating=getRatingCache(getOldIDFromTitle(trim($input)));
  return number_format($rating['count'], 2);
}

#-------------------------------------------------------------------------------
# Displays a list of ratings based on the parameters given between the tags    #
#-------------------------------------------------------------------------------
function renderRatings($input, $argv, &$parser)
{
  $parameters=splitParameters($input);
  $sortdata=getSortData(isset($parameters['namespace'])?
    $parameters['namespace']:array());
  $ratings=getRatingList(isset($parameters['namespace'])?
    $parameters['namespace']:array(), $sortdata['column'], $sortdata['order'],
    isset($parameters['limit'])?$parameters['limit'][0]:10);
  $output=;
  $displayCount=(isset($parameters['displaycount'])?
    $parameters['displaycount'][0]:'false')!='false';
  $displayRating=(isset($parameters['displayrating'])?
    $parameters['displayrating'][0]:'true')!='false';
  $pattern=isset($parameters['pattern'])?$parameters['pattern'][0]:
    '[[$1|$2]]'.($displayRating?' ($3)':).($displayCount?' ($4 ratings)':);
  $limit=isset($parameters['limit'])?$parameters['limit'][0]:10;
  foreach($ratings as $rating)
  {
    $output.=str_replace(array('$1', '$2', '$3', '$4', '$5'), 
      array(($rating['namespace']==6?':':).$rating['title'],
        strtr($rating['title'], '_', ' '), 
        number_format($rating['rating'], 2), 
        $rating['count'], $rating['oldid']), $pattern)."nn";
    if(--$limit==0) break;
  }
  return renderWikiText(trim($output), $parser);
}

#-------------------------------------------------------------------------------
# Insert rating to top of page                                                 #
#-------------------------------------------------------------------------------
function InsertRating($parserOutput, $text) {
  global $wgArticle, $allowedNamespaces;
  if(!$allowedNamespaces[$wgArticle->getTitle()->getNamespace()])
    return;
  $oldid=getOldID($wgArticle);
  if($oldid)
    $text='<div id="ratingsection">'.
      getRatingHTML(getRatingCache($oldid), $oldid).'</div>'.$text;
}

#-------------------------------------------------------------------------------
# Add CSS into skin for rating                                                 #
#-------------------------------------------------------------------------------
function RatingCss(&$css) {
  global $wgScriptPath;
  $css = "/*<![CDATA[*/".
  " @import "$wgScriptPath/?file=rating.css"; ".
  "/*]]>*/";
  return true;
}

################################################################################
# Supporting Functions                                                         #
################################################################################
# These functions support the core functions of the extension                  #
################################################################################

#-------------------------------------------------------------------------------
# Processes sortby parameters to determine how to sort ratings                 #
#-------------------------------------------------------------------------------
function getSortData($sort)
{
  $column='rating';
  $order=-1;
  foreach($sort as $item)
  {
    switch($item)
    {
      case 'rating':
        $column='rating'; break;
      case 'name':
        $column='title'; break;
      case 'count':
        $column='count'; break;
      case 'ascending':
        $order=SORT_ASC; break;
      case 'descending':
        $order=SORT_DESC; break;
    }
  }
  if($order==-1)
  {
    switch($column)
    {
      case 'rating':
        $order=SORT_DESC; break;
      case 'title':
        $order=SORT_ASC; break;
      case 'count':
        $order=SORT_DESC; break;
      default:
        $order=SORT_DESC;
    }
  }
  return array('column'=>$column, 'order'=>$order);
}

#-------------------------------------------------------------------------------
# Processes the given text using the mediawiki parser engine                   #
#-------------------------------------------------------------------------------
function renderWikiText($input, &$parser)
{
  return $parser->parse($input, $parser->mTitle, $parser->mOptions, 
    false, false)->getText();
}

#-------------------------------------------------------------------------------
# Returns the page rating given the pages string title                         #
#-------------------------------------------------------------------------------
function getOldIDFromTitle($title)
{
  $title=Title::newFromText($title);
  if($title==null)
  {
    echo "NoArticle";
    return 0;
  }
  $oldid=getOldID(new Article($title));
  if(!$oldid)
  {
    return "NoOldID";
    return 0;
  }
  return $oldid;
}

#-------------------------------------------------------------------------------
# Completes ratings_cache table for any revisions without ratings              #
#-------------------------------------------------------------------------------
function doFillCache()
{
  global $allowedNamespaces;
  $namespacestring=;
  foreach(array_keys($allowedNamespaces) as $space)
  {
    if(is_numeric($space))
    {
      $namespacestring.=', '.$space;
    }
  }
  $namespacestring=substr($namespacestring, 2);
  $sql="SELECT `page_latest` AS `oldid` FROM `page` WHERE".
    ' `page_namespace` IN ('.$namespacestring.') AND'.' `page_latest` NOT IN '.
    '(SELECT `page_oldid` FROM `ratings_cache`);';
  $articles=runQuery2($sql);
  foreach($articles as $article)
    getRating($article->oldid);
}

#-------------------------------------------------------------------------------
# Returns a list of pages in the specified namespaces                          #
# Optionally it may return sorting rating information for results              #
#-------------------------------------------------------------------------------
function getPageList($namespace=array(), $ratings=false, $sort='title',
    $order=SORT_ASC, $limit=100)
{
  global $renderLimit, $allowedNamespaces;
  $limit=($limit>$renderLimit?$renderLimit:$limit);
  $namespacestring=;
  $filterByNamespace=false;
  foreach($namespace as $space)
  {
    if(is_numeric($space) && isset($allowedNamespaces[$space]))
    {
      $namespacestring.=', '.$space;
      $filterByNamespace=true;
    }
  }
  if($filterByNamespace)
    $namespacestring=substr($namespacestring, 2);
  $sql="SELECT `page_title` AS `title`, `page_namespace` AS `namespace`, 
    `page_latest` AS `oldid` FROM `page`".($filterByNamespace?
    ' WHERE `page_namespace` IN ('.$namespacestring.')':"").';';
  if($ratings)
    $sql="SELECT `page_title` AS `title`, `page_namespace` AS `namespace`, ".
      "`page_latest` AS `oldid`, `page_oldid`, `page_rating` AS `rating`, ".
      "`page_rating_count` AS `count` FROM `page`, `ratings_cache` ".
      "WHERE `page_latest`=`page_oldid`".($filterByNamespace?
      ' AND `page_namespace` IN ('.$namespacestring.')':"").
      ' ORDER BY '.$sort.($order==SORT_ASC?:).
      ($order==SORT_DESC?' DESC':).' LIMIT '.$limit.';';
  return runQuery2($sql);
}

#-------------------------------------------------------------------------------
# Returns a list of pages and their ratings for all pages in the specified     #
#  namespaces                                                                  #
#-------------------------------------------------------------------------------
function getRatingList($namespace=array(), $sort='title', $order=SORT_ASC,
    $limit=100)
{
  global $wgNamespaceNamesEn, $renderLimit;
  doFillCache();
  $limit=($limit>$renderLimit?$renderLimit:$limit);
  $articles=getPageList($namespace, true, $sort, $order, $limit);
  $ratings=array();
  foreach($articles as $article)
  {
    $rating=formatRating(array('count'=>$article->count,
      'rating'=>$article->rating));
    $ratings[]=array('title' => $wgNamespaceNamesEn[$article->namespace].
      ($article->namespace==0?:':').$article->title, 
        'rating' => $rating['rating'], 'count' => $rating['count'],
        'oldid'=>$article->oldid, 'title_name'=>$article->title,
        'namespace_name'=>$wgNamespaceNamesEn[$article->namespace],
'namespace'=>$article->namespace);
  }
  return $ratings;
}

#-------------------------------------------------------------------------------
# Splits parameters from the wikitext.                                         #
# Each parameter should be on its own line in the format  parameter = value    #
#-------------------------------------------------------------------------------
function splitParameters($input)
{
  $parameters=array();
  foreach(split("n", $input) as $parameter)
  {
    $parameter=split('=', $parameter, 2);
    if(count($parameter)==2)
    {
      foreach($parameter as $key => $val)
        $parameter[$key]=trim($val);
      if(isset($parameters[$parameter[0]]))
        $parameters[$parameter[0]][]=$parameter[1];
      else
        $parameters[$parameter[0]]=array($parameter[1]);
    }
  }
  return $parameters;
}

#-------------------------------------------------------------------------------
# Use the mediawiki engine to run the given sql code and return an object      #
# containing the first result of the query                                     #
#-------------------------------------------------------------------------------
function runQuery($sql)
{
  $dbr =& wfGetDB( DB_SLAVE );
  $res=wfQuery($sql, DB_SLAVE, "");
  if(wfNumRows($res)>0)
    return $dbr->fetchObject( $res );
  else
    return null;
}

function runQuery2($sql)
{
  $dbr =& wfGetDB( DB_SLAVE );
  $res=wfQuery($sql, DB_SLAVE, "");
  $array=array();
  while($item=$dbr->fetchObject( $res ))
    $array[]=$item;
  return $array;
}

#-------------------------------------------------------------------------------
# Return the oldid of the current page                                         #
# If oldid=0 (most current revision) take the latest oldid from the database   #
# for the current article                                                      #
#-------------------------------------------------------------------------------
function getOldID($article)
{
  $oldid=$article->getOldIDFromRequest();
  if($oldid!=0)
    return $oldid;
  $dbr =& wfGetDB( DB_SLAVE );
  $sql="SELECT `page_latest` AS `oldid` FROM `page` ".
    "WHERE `page_id`=".$article->getID().";";
  $res=wfQuery($sql, DB_SLAVE, "");
  $row=$dbr->fetchObject( $res );
  if($row->oldid)
    return $row->oldid;
  return null;
}

#-------------------------------------------------------------------------------
# Performs a SQL query to fetch a rating for page oldid                        #
#-------------------------------------------------------------------------------
function getRatingData($oldid)
{
  $sql="SELECT COUNT(*) AS `count`, AVG(`page_rating`) AS `rating` ".
    "FROM ratings WHERE `page_oldid`=".intval($oldid)." GROUP BY `page_oldid`;";
  return runQuery($sql);
}

#-------------------------------------------------------------------------------
# Returns the ID for the revision before $revision                             #
#-------------------------------------------------------------------------------
function getPreviousRevisionID( $revision ) {
$dbr =& wfGetDB( DB_SLAVE );
return $dbr->selectField( 'revision', 'rev_id',
'rev_page=(SELECT `rev_page` from `revision` WHERE `rev_id`='.
intval( $revision ).')'.' AND rev_id<' . intval( $revision ) . 
' ORDER BY rev_id DESC' );
}

#-------------------------------------------------------------------------------
# Fetch and calculate a rating for page oldid                                  #
# If there are not enough ratings for the current revivion, cycle              #
# older revisions to gather a minimum number of ratings                        #
# Updates cache table with calcaulated values                                  #
#-------------------------------------------------------------------------------
function calculateRating($oldid)
{
  global $wgTitle, $countLimit;
  $origid=$oldid;
  $ratingdata=getRatingData($oldid);
  $finalrating='?';
  $currentcount=number_format($ratingdata->count, 0);
  #If there are not enough ratings for the current revision
  if($ratingdata->count<$countLimit)
  {
    $count=$ratingdata->count;
    $rating=$count*$ratingdata->rating;
    #cycle older revisions looking for more ratings
    while($oldid=getPreviousRevisionID($oldid)) 
    {
      $ratingdata=getRatingData($oldid);
      #If still not enough ratings
      if($count+$ratingdata->count<$countLimit)
      {
        $count+=$ratingdata->count;
        $rating+=$ratingdata->count*$ratingdata->rating;
      }
      else #found enough ratings
      {
        $rating+=($countLimit-$count)*$ratingdata->rating;
        $count=$countLimit;
        $finalrating=$rating/$count;
        $oldid=false;
      }
    }
  }
  else
    $finalrating=$ratingdata->rating;

  $dbr =& wfGetDB( DB_WRITE );
  $sql = "REPLACE INTO `ratings_cache` (`page_oldid`, `page_rating`, ".
    "`page_rating_count`) VALUES (".intval($origid).", ".
    (is_numeric($finalrating)?$finalrating:0).", ".$currentcount.")";
  $res=wfQuery($sql, DB_WRITE, "");
  
  return array('rating'=>$finalrating, 'count'=>$currentcount);
}

#-------------------------------------------------------------------------------
# Formats rating array for insertion to page rating section                    #
#-------------------------------------------------------------------------------
function formatRating($rating)
{
  $finalrating=$rating['rating'];
  $currentcount=$rating['count'];
  
  #format rating data
  if(is_numeric($finalrating) && $finalrating>0)
    $finalrating=($finalrating-1)*1.25;
  $ratingarray=array('display'=>
      (is_numeric($finalrating)?number_format($finalrating, 2):$finalrating).
      " ($currentcount ratings)",
    'count'=>$currentcount,
    'rating'=>(is_numeric($finalrating)?$finalrating:0));
  return $ratingarray;
}

#-------------------------------------------------------------------------------
# Calculates rating from database for specified revision                       #
# Updates rating cache table                                                   #
#-------------------------------------------------------------------------------
function getRating($oldid)
{
  return formatRating(calculateRating($oldid));
}

#-------------------------------------------------------------------------------
# Fetches the ratings from cache table                                         #
# Calculates rating if it is not found in cache                                #
#-------------------------------------------------------------------------------
function getRatingCache($oldid)
{
  $sql='SELECT * FROM `ratings_cache` WHERE `page_oldid`='.intval($oldid).';';
  $rating=runQuery($sql);
  $rating=array('rating'=>$rating->page_rating,
    'count'=>$rating->age_rating_count);
  if($rating->page_oldid==null)
    $rating=calculateRating($oldid);
  return formatRating($rating);
}

#-------------------------------------------------------------------------------
# Given the rating array and the page oldid, generate HTML code to be          #
# displayed                                                                    #
#-------------------------------------------------------------------------------
function getRatingHTML($rating, $oldid)
{
  global $wgTitle, $wgScriptPath, $countLimit;
  $html=;
  #generate stars
  for($x=0;$x<=4;$x++)
  {
    $html.='<a href="'.
      $wgTitle->getFullURL('oldid='.$oldid.'&rate='.($x+1)).'" rel="nofollow">'.
      '<img src="?file=Star';
    if($rating['rating']>=$x+1) #larger than current star : filled
         $html.='4'.($rating['count']<$countLimit?'b':);
    elseif($rating['rating']>=$x+0.75) #3/4 current star : 3/4 filled
      $html.='3'.($rating['count']<$countLimit?'b':);
    elseif($rating['rating']>=$x+0.5) #1/2 current star : 1/2 filled
      $html.='2'.($rating['count']<$countLimit?'b':);
    elseif($rating['rating']>=$x+0.25) #1/4 current star : 1/4 filled
      $html.='1'.($rating['count']<$countLimit?'b':);
    else #less than current star : empty
      $html.='0';
    $html.='.png" align=bottom/></a>'."n";
  }
  #add text rating 
  $html.=' '.$rating['display'].;
  $html=($rating['count']<$countLimit?'<b>'.$html.'</b>':$html);
  return $html;
}


#-------------------------------------------------------------------------------
# Return a file and exit.                                                      #
# File determined by ?file= GET parameter                                      #
#-------------------------------------------------------------------------------
function doFile()
{
  switch ($_GET['file'])
  {
    #Star .png files
    case "Star0.png":
    case "Star1.png":
    case "Star1b.png":
    case "Star2.png":
    case "Star2b.png":
    case "Star3.png":
    case "Star3b.png":
    case "Star4.png":
    case "Star4b.png":
      header("Content-type: image/png");
      echo readFile('extensions/rating/'.$_GET['file']);
      die();
    #extension css styling
    case "rating.css":
      header("Content-type: text/css");
        ?>
#ratingsection {
  float: right;
  margin-top: -3.7em;
  padding: 3px;
}

#ratingsection b {
  color: red;
  padding: 4px;
}
<?php
      die();
  }
}

<div style="page-break-before: always;"></div>

Appendix 2. Survey

• What is your age?
• What is your gender?
• What is your primary mode of study? (Internal/Distance)
• Which course are you studying?
• Rate your ability with computers. I feel confident ...
  • using a personal computer
  • starting a computer program
  • using word processing software to format a letter or essay
  • copying and moving files to removable media (floppy disc, CD, or USB thumb/flash drive)
  • deleting files when they're no longer needed
  • learning how to use new software
  • using help features in software
  • locating information on the Internet
  • checking email
  • sending email
  • using instant messaging
  • using an online forum
  • writing a blog
  • writing web pages
  • understanding HTML
  • installing software
  • troubleshooting software problems
  • troubleshooting hardware problems
  • using a scripting programming language
  • using a functional programming language
  • using an object-oriented programming language
  • writing computer applications
• Before completing this subject I ...
  • knew what a wiki was
  • had read a wiki without knowing at the time it was a wiki
  • had read a wiki knowing it was a wiki
  • had read from Wikipedia
  • had contributed to a wiki
• I now feel confident
  • adding text to a wiki
  • making links in a wiki
  • participating in discussions in a wiki
  • adding a signature to a wiki discussion
  • using headings in a wiki
  • using lists (numbered lists or unnumbered bullets) in a wiki
  • creating articles
  • understanding which is the correct name space to use
  • uploading files to a wiki
• A rating system was added to Kakapo wiki. Please rate the following statements:
  • Did you understand how to use the rating system
• Did you rate any articles?
• Please rate the following statements.
  • Do you think your ratings were accurate?
  • Do you think other peoples ratings were accurate
  • Did you find the ratings useful
  • Do you think you were inclined to rate your friends articles more favourably
  • Visual quality of the article is important when rating
  • The factual quality of the article is important when rating
  • The writing style of the article is important when rating
• Do you feel there is another important factor when rating (free response)
• Please rate the following statement.
  • I found the pod exercises easy
• Describe two things you found difficult
• Please rate the following statement.
  • I found understanding the game I chose easy
  • I felt comfortable submitting my words to the wiki
• Choose three words that describe how you felt placing text on the wiki (e.g. excited, enthusiastic, proud, confident, indifferent, nervous, unsure, confused, shy)
• Explain why you felt this way
• What general comments would you like to make

Appendix 3. POD Exercises

POD Activity 1

Exercise 1

Visit http://ispg.csu.edu.au/kakapowiki/ and create an account. Read the help and learn how a wiki works and how to use it.

Experiment by <u>writing your own user page</u>.

N.B. Its best to be logged in when editing so the marker can be sure that you actually posted the content that you want to be marked on.

Also, <u>create an article</u> for your POD group. Have each member add a short description of themselves, including your full name, a link to your user page, and whether you are a distance or internal student. Use this page to communicate with the world what your group is doing.

Exercise 2

As a group, research and choose a <u>Multiplayer Online Game</u> or <u>Massively Multiplayer Online Game</u> that you can all play. Sign up and start learning about the game. You can if you wish choose the same game as another POD group, or choose a different game. You may like to keep a journal or notes as you learn about the game. You may need them for Exercise 2. For your journal/notes, you may wish to use your user page in the wiki.

To find a game to play, try the following:

• http://www.mpogd.com/games/name.asp

Alter the filters to find one that suits. Eg. choose "Browser" for Client type if you don't want to install any extra software, or choose a 2D or text based game if you don't have a fast computer.

• http://www.free-games.com.au/Free_Online_Multiplayer_Games/
• http://en.wikipedia.org/wiki/List_of_MMORPGs
• http://en.wikipedia.org/wiki/Category:Browser-based_games

Exercise 3

Each person must write at least 500 words of content in the wiki.

Examples of content are:

• An article on the game you chose as a group
• An article on a theory or a principle used online games eg.
  • Synthetic economies
  • Free-form games
  • Characterisation / Alternate online identity
  • Online Romance / Relationships
  • Addiction
  • Motivations of playing
  • comradeship / support / help and friendship
• An article on an online game or online collaborative space you are interested in or have used

Keep the following points in mind:

• Creating your user page or POD page does not count. your 500 words must be in the form of a standard article.
• your 500 words may be in one article, or spread across multiple articles
• more than one person can contribute to an article. You might like to write a single 2000 word article as a group.
• you must reference any information you use in the article

Resources:

• http://www.nickyee.com/daedalus/

POD Activity 2

Choose a theory or principle exemplified in the game you played and write an article on it. Include a section detailing your experience, how you discovered/learned about it, and your views on it.

Again you may choose the same topic as another person, in which case you must still contribute 500 words, including your own personal view.

Articles should contain an expanded definition of the topic with references. This may be followed by several personal experiences.

Ensure the section detailing your experiences is clearly marked as a personal experience, and separate from the main article.