Thesis/Analysis of Data

From Researchwiki

Two formal data sources were used for analysis in this research. Firstly Apache logs collected from the wiki community, and secondly data from a follow up survey of the participants. This chapter will present an analysis of this data.

Many figures quoted in this chapter have companion commands used to generate them. As these methods may be useful in future research, these are indicated by subscript numbers throughout the chapter, and commands are included for reference at the end of the chapter.

Data Analysis

Data gathering consisted of two stages; firstly Apache server logs were collected for a wiki system, and secondly a survey was completed by the users of the wiki. Considering these two forms of data is important in understanding users behaviour. Apache logs give an objective view of behaviour, a detailed listing of every action, whereas the survey results allow the painting of a more detailed picture by brining in an understanding of the intent, thoughts and feelings behind the behaviour. Presented here are the results of the analysis of the two data sources, and discussion of their interpretation in terms of the research questions presented in chapter 3.

Apache Logs

Data Overview

Data was collected from log-files generated by the Apache server, and from the MediaWiki database. The log was processed and stored in the database in a format facilitating simple SQL queries to be run against the data.

The wiki was installed and initialised on the Internet Special Projects Group (ISPG) server on the 4/Jul/2006. The ISPG server, maintained by Geoff Fellows, hosts several applications for research projects, as well as teaching resources. Over the following weeks, the wiki was seeded with its initial set of pages. Data recovered from ISPG contained 225401 lines of logs (see table 4.1), recorded between 31/Jul/2006 and 20/Sep/2006, of which 15162 pertained to KakapoWiki (the server hosts several projects). From this number were removed requests for files (3234) (not to the MediaWiki engine itself), including CSS files and JavaScript files requested through the engine (2119 and 762 respectively). 2436 lines generated from automated (non-human) requests to the wiki were also ignored. 5 lines also failed to be matched by the regular expression being used to parse log file records. None of these five pertained to the wiki.

not_kakapo:     210239   For other projects hosted on ISPG  
lines_recorded: 6606     Total lines added to the database
file_request:   3234     Requests for files through the rating system
re_fail:        5        Lines that the regular expression failed to match
css_request1:   1420     requests for CSS files
total_lines:    225401   Total number of log records processed
css_request2:   699      more requests for CSS files
js_request:     762      requests for JavaScript files
stats_logger:   2436     automated requests
Table 4.1 "report from log file parser, log2db.py"

A total of 6606 usable entries (table 4.1) were retrieved from the Apache logs (non-automated queries to the engine). These entries show users actions or behaviours while interacting with the wiki. These entries were broken down in table 4.2.

+--------+-------+-----------+---------------+
| action | total | sup_count | student_count |
+--------+-------+-----------+---------------+
| DEL    |     2 |         2 |             0 |
| DIFF   |    42 |        36 |             6 |
| EDIT   |   786 |       250 |           536 |
| HIST   |   460 |       198 |           262 |
| LOGIN  |   242 |        12 |           230 |
| LOGOUT |    32 |        14 |            18 |
| MOVE   |    27 |         7 |            20 |
| RATE   |    81 |        39 |            42 |
| REG    |    59 |         0 |            59 |
| SAVE   |   688 |       332 |           356 |
| SPEC   |   467 |       165 |           302 |
| UL     |    37 |        14 |            23 |
| UWATCH |     2 |         1 |             1 |
| VIEW   |  3677 |       853 |          2824 |
| WATCH  |     4 |         1 |             3 |
| total  |  6606 |      1924 |          4682 |
+--------+-------+-----------+---------------+
Table 4.2 "Results from SQL Query of the Apache logs1"

+------------+----------+
| log_action | COUNT(*) |
+------------+----------+
| delete     |        2 |
| move       |        4 |
| upload     |       10 |
+------------+----------+
Table 4.3 "Results from SQL query from the MediaWiki database2"

The Log data (table 4.2) counts the number of requests made to the Apache server. From the logs 15 types of requests could be identified.

DEL counts the number of requests to delete items from the wiki. Only the researchers had the ability to remove objects. It is standard practice in MediaWiki installations that only administrators (or sysops as they are known) have delete privileges. A count of successful requests (table 4.3) confirms the log figure as accurate.

DIFF shows the number of times users have viewed the differences between two versions of a page. Most commonly this is to view the last made change, but other times to see how the page has changed over time.

EDIT represents the number of times the wikitext has been viewed by clicking the edit button. Some of these result in page SAVEs, others just satisfy a users curiosity of how a page is marked-up.

HIST represents a view of a pages history. Viewing this page may result in subsequent actions. A user may choose to view a DIFF of two versions of the page, or they may VIEW an older version of the page. HIST also includes viewing of a user's list of contributions, and viewing a list of recent changes to the wiki.

LOGIN shows when a user attempts to provide their correct username and password to login to the system. As shown below, login attempts result in at least two log records.

LOGOUT counts when a user instructs the system to disassociate further actions with the users' current user account. It is typically performed when the user has finished working, or another user wishes to work at the same workstation.

MOVE shows when a user clicks the move link above a page. Only four of these requests were completed during the data recording period (table 4.3), two each by students and researchers

RATE indicates when a rating star is clicked by a user to record or modify their rating of a page. A rating will result in another VIEW as the browser returns the viewed page again with the new rating. There were 63₃ ratings stored in the database. This indicates that 18 ratings were adjustments on previous ratings (where a user changed a rating). There were 37₄ ratings by users who had no involvement with a page's content. These ratings are considered unbiased ratings, as the rater has no influence on the content.

REG monitors when a user registers their username with the wiki. This action may result in multiple records. The are not included in the recorded data as they had registered before data collection started. A total of 26₅ users were registered in the wiki, including the three researchers.

SAVE follows an EDIT or another SAVE, and indicates either the save or preview button is pressed after editing wikitext. Several previews may be made before saving. Alternatively, the page may also be saved without modification, or no save may be made after modifying the wikitext. Saving the page without a change (a "null edit") causes MediaWiki to re-parse the page, updating any components derived from templates that may have changed. While this would show up in the logs, it is not included as a change to the page in MediaWiki. The MediaWiki database reports 360₆ edits to 112₇ pages during the data collection period. These edits are when a page is changed (ie. not a "null edit") and committed (not previewed) to the database.

SPEC counts when a special page is viewed. Special pages include viewing a list of all pages in the database, a list of all files uploaded, version information of the wiki, searching, viewing statistics, and viewing a list of users. Logging in and out and moving pages involves the use of special pages, but were counted in their own counts.

UL counts attempts to upload an image or other file. A successful attempt will result in several records. In total 10 files (figure 4.3) were uploaded during the data collection period.

WATCH indicates when a user clicks the watch tab above a page. This is not the only way to add a page to a watchlist, as a user has the option to add a page when saving changes to that page.

UWATCH shows when a user clicks the unwatch tab above a page to remove a page from their watchlist.

VIEW counts users' views of a standard content page. This count also includes views of old versions of a page, and views of some special pages. The MediaWiki database reports 2770₈ page views. This count does not include repeated sequential views of a page, but increments only when a page is parsed and html generated from the wikitext. Pages are parsed and stored in a cache, which unless manually expired, remain valid for 24 hours. For each user, MediaWiki maintains their own set of cache entries.

Explanation of differences in counts

Some values in the log data are inflated when compared with the MediaWiki database. Log data represents actions attempted, while the MediaWiki database represents actions successfully completed. Similarly, some actions require multiple steps. To illustrate, logging-in is a two-step process.

The following example shows four stages:

1. The user visiting a page, not logged in
2. The user having clicked the login link
3. The user after correctly entering their username and password
4. The user viewing the original page again, now logged in

Figure 4.1 shows examples from the Apache logs for the previously mentioned four events

137.166.81.96 - - [08/Aug/2006:14:14:08 +1000] ispg.csu.edu.au "GET /kakapowiki/
index.php/KakapoWiki:ITC213/200670/POD_Activities HTTP/1.1" 200 4276 "http://isp
g.csu.edu.au/kakapowiki/index.php/KakapoWiki:ITC213/200670/POD_Activities" "Mozi
lla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-sec
urity Firefox/1.5.0.5" "kakapowikidbUserName=TrevorP; kakapowikidb_session=70da8
f9646996952454cd2da00a79360; kakapowikidbLoggedOut=20060808041401"

137.166.81.96 - - [08/Aug/2006:14:14:10 +1000] ispg.csu.edu.au "GET /kakapowiki/
index.php?title=Special:Userlogin&returnto=KakapoWiki:ITC213/200670/POD_Activiti
es HTTP/1.1" 200 2392 "http://ispg.csu.edu.au/kakapowiki/index.php/KakapoWiki:IT
C213/200670/POD_Activities" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5)
 Gecko/20060731 Ubuntu/dapper-security Firefox/1.5.0.5" "kakapowikidbUserName=Tr
evorP; kakapowikidb_session=70da8f9646996952454cd2da00a79360; kakapowikidbLogged
Out=20060808041401"

137.166.81.96 - - [08/Aug/2006:14:14:12 +1000] ispg.csu.edu.au "POST /kakapowiki
/index.php?title=Special:Userlogin&action=submitlogin&type=login&returnto=Kakapo
Wiki:ITC213/200670/POD_Activities HTTP/1.1" 200 2132 "http://ispg.csu.edu.au/kak
apowiki/index.php?title=Special:Userlogin&returnto=KakapoWiki:ITC213/200670/POD_
Activities" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060731 
Ubuntu/dapper-security Firefox/1.5.0.5" "kakapowikidbUserName=TrevorP; kakapowik
idb_session=70da8f9646996952454cd2da00a79360; kakapowikidbLoggedOut=200608080414
01"

137.166.81.96 - - [08/Aug/2006:14:14:15 +1000] ispg.csu.edu.au "GET /kakapowiki/
index.php/KakapoWiki:ITC213/200670/POD_Activities HTTP/1.1" 200 4467 "http://isp
g.csu.edu.au/kakapowiki/index.php?title=Special:Userlogin&action=submitlogin&typ
e=login&returnto=KakapoWiki:ITC213/200670/POD_Activities" "Mozilla/5.0 (X11; U; 
Linux i686; en-US; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-security Firefox/1.5
.0.5" "kakapowikidbUserName=TrevorP; kakapowikidb_session=70da8f9646996952454cd2
da00a79360; kakapowikidbLoggedOut=20060808041401; kakapowikidbUserID=1"

* These have been modified, removing irrelevant cookies used by other CSU
services.

Figure 4.1 "A log-file sample of the login sequence"

Data, such as that in figure 4.1, was processed by a python script, the important fields were extracted, and inserted into a database. Table 4.4 shows the same four records after being processed and placed in the database.

Table 4.4 "A sample log of a login event formatted into a database table"

Examination of the data in figure 4.1 or table 4.4 shows how a single event can manifest itself as multiple records in the log. As a further illustration, if the user had not correctly entered their username and/or password, there would be another line where the user was given an error message and asked to try again.

Analysis

This part of the analysis was performed in two sections. Firstly the system generated ratings were studied, followed by analysis of attention data.

Ratings

The primary goal of this experiment was to monitor a simple rating system in a wiki community, and users behaviours and reactions to the system, where the participants were undergraduate students. Apache logs provide a rich source of objective measurements of behaviour, however they do not reveal intent or the thoughts and feelings behind the behaviour. The next section will analyse survey results, which attempt to cover this topic.

Apache logs indicated a total of 81 ratings, from one to five, submitted by users to the rating system. The distribution of ratings were uneven, with 78% of ratings occurring in the top 40% of the one to five scale (figure 4.2).

Figure 4.2 "A tally of the number of ratings for each value"

The context of these ratings must be analysed. The system allowed users to adjust their rating, by re-entering their selection via a click on the rating stars. The rating system was designed to accept this rating, replacing that users original rating for that revision of the page. These re-ratings account for 18, or 22% of ratings. Another factor to be conscious of is that it is possible for the same user to modify and rate the same page. The rating system was designed with a simple mechanism to reduce the possibility that this fact could be abused. Users were not excluded from rating pages they had edited, for the reason that though the user had edited the page, their influence on that page is reduced by future edits. A users influence may be removed completely if the text they add is completely removed by a future edit. For this reason also, a ratings importance diminishes as a page is edited, and as the quality of a page increases or decreases with each edit. To solve this simply, ratings were associated with a revision, not the page itself. Once a page had been updated, old ratings were only used to supplement ratings for the current version until a quorum of ratings had been contributed for the current article. As each user is only allowed one vote, a biased vote can be quickly corrected by other raters. In this system however, with the low number of ratings, it is unlikely this situation ever occurred.

Table 4.5 summarises ratings based on two determining factors; if the rating is a correction or alteration on a previous rating (that is, a rating by the same user on the same revision of a page), and if the user has ever modified the page they are rating, possibly resulting in a biased rating.

Table 4.5 "Summary of ratings usefulness"

In total, 31 usable ratings were generated. This number excludes corrections to existing ratings, and ratings where the rating user had edited the page at any time. Comparing ratings with page views gives an idea of users willingness to rate articles. A user will rate 2.2-2.9% (depending on the page count used) of pages they view, generating 0.84-1.1% usable ratings per page view.

This filtering results in a similar unbalance of ratings as the unfiltered results, with 87% of ratings in the top 40% of rating values, that is, a rating of four or five (figure 4.3).

Figure 4.3 "A tally of the number of unbiased ratings for each value"

Both the filtered and unfiltered sets of ratings (figures 4.2 and 4.3) show a regular occurrence of high ratings. This is found also to be true in eBay ratings, where members are very positive towards trading partners. Resnick and Zeckhauser (2002) suggests this is in keeping with social norms to treat others well. Dellarocas (2001) however puts it down to a "culture of praise", where members feel that the correct thing to do is to be kind and forgiving to other members. In doing so, users may also avoid rating bad pages.

Comparison between the two sets of data represented by figures 4.2 and 4.3 show that the "potentially biased" unfiltered ratings are more evenly distributed than the "unbiased" or filtered ratings. By analysing the data available, 41₉ "biased" ratings (pages where a user rated and edited) were were found to be made by a user after editing the page they rated, while only 18₁₀ ratings were found before a user edited the page.

Rated pages were manually assessed by the researcher to gain a baseline quality measurement (summarised in figure 4.4). Pages were judged on the potential usefulness of the information on a page (to an internal or external viewer), and the information content of the page. The scale used was a 1-5 rating, with intervals of 0.5. Judging usefulness meant that short pages generally received low scores. To be deemed useful, a page required a substantial amount of information relevant to the community, and required the content be coherent and sensible. A better rating could be sought, either by building a more strict marking scheme, or consulting an expert in the field of instructional gaming, however the ratings generated are believed to be a suitable starting point for analysis.

Figure 4.4 "distribution of manual ratings"

The manual ratings (figure 4.4) show a more even distribution than the system generated ratings (figures 4.2 and 4.3), with the mode slightly lower than the system generated ratings, and a minimum score of 1.5. These scores were compared with the system generated ratings to see how effectively the rating system performed in the wiki.

Figure 4.5 "Scatter graph comparing manual ratings with system generated ratings."
Points have been offset randomly +-0.1 so that multiple data-points at the same position can be visualised

Comparison of ("unbiased") system ratings and manual ratings (figure 4.5) show little statistical correlation between the two. To the eye however, the main body of the graph shows more correlation. Removing all the system ratings of 5 makes this the main feature, and increases the correlation from 0.17808398844 to 0.411799050614. Overall however, the ratings generated by the rating mechanism in this environment were largely ineffective.

While some results may be improved with a better manual rating, it was believed after reviewing the results available, that any improvement in correlation gained would be minimal.

Attention Data

A secondary focus of the experiment was to study attention data. Attention data gives a measure of the amount of attention (measured in time) spent on a task (rating) or object (page). Attention data was collected from the Apache logs and processed by a script that identified individually a users actions, and by processing them chronologically, identified the amount of time between actions.

A simple measure to study the usefulness of attention data is to study the time a user spends reading a page before they rate it. Given that ratings generated by the rating system appear invalid, manual ratings will be used for comparison. Time in this analysis is measured in seconds per character. This gives a standard measurement regardless of the length of a page.

Figure 4.6 "Times taken to rate compared to manual ratings"

The attention data for the time spent rating pages (figure 4.6) shows a slight correlation between time taken to rate and the manual rating assigned to a page. Values 3, 3.5, 4, and 5 (the ratings for which most of the manual ratings fell) show an almost linear sequence, tending downwards, that is, users spent less time reading better pages before rating them. There is however, not enough data to reasonably show this beyond chance occurrence.

Figure 4.7 "Times spent looking at pages compared to manual ratings"

Attention data for the time spent viewing pages was gathered and filtered to attempt to remove two types of events. Firstly, page views where the user immediately went on to another page, and secondly views where the user had visited the page and ended their viewing, either closing the browser, or by leaving the browser open at that page for an extended period of time. Only the middle half of values were analysed, removing the top and bottom quarters. Also pages with too few views were excluded to reduce outlying values. After filtering, the data for the time spent viewing pages (figure 4.7) shows only a slight statistical correlation of -0.343993118283 when compared to manual ratings.

Figure 4.8 "Number of page views grouped by manual rating"
This figure shows results after filtering out "5" ratings.

Counting page views (figure 4.8) also gives a rough measure of attention. This simple count shows a very high number (641) of views of pages manually assigned a rating of 5. When considering there were only three pages rated at five this count is surprising. Closer inspection finds that these three pages are the main page of the wiki, and two revisions of the assignment questions. The main page is a high visibility page, and the default entry point for anyone visiting the wiki. The assignment questions were important to the participants of the wiki, as they contained information important to their success in their studies. After removing this anomaly, the distribution of views (figure 4.8) compared to the number of pages in each group (figure 4.4) shows no useful information.

Survey

Given the behavioural findings in the last section, this research sought to explain these behaviours. Two major findings in the logs were the low number of ratings, and the fact that a large number of ratings were identified as having a potential for bias. The questionnaire sought to understand the students' level of participation in the wiki, and what factors influenced it. It sought to understand how students viewed the rating system, how they used it, and possible biases. The survey asked about the students confidence in completing the pod exercises, and in learning about a game and its concepts. It asked how the participant felt submitting their work to a public wiki, as well as gauging their understanding of the wiki. The survey also asked for some basic demographics, assessed the participants computer self-efficacy, and their experience with online tools.

The questionnaire was conducted online and contained 57 questions, consisting of 51 numeric ratio entry questions, 1 short answer and 5 extended response boxes.

Students asked to participate in the wiki counted 13, and in total 11 students completed the survey. The 85% response rate is due to strong encouragement from the lecturer of the subject, and substituting the survey for what was originally an assessable task. Since the results of the survey were designed to provide guidance in interpreting the log results, the low sample size of 11, given the population size of 13, was not seen as a problem.

The ages of participants ranged from 20 to 32, with 20 the mode, an average of 24.4, and a median of 22. Participants were eight male and three female, with a roughly equal split of internal and distance education students, 55% internal, 45% external. Every respondent was completing an information technology or information science based degree. These results were expected for an information technology focused subject at CSU.

The survey sought to understand participants' confidence and ability with computing tasks. Students were asked to report their confidence in performing a number of tasks. Allowable responses followed a standard five-point Likert scale, and were as follows

1. strongly disagree
2. disagree
3. undecided
4. agree
5. strongly agree

Results received from students were processed, and ordered by average response in table 4.6.

Table 4.6 "Ranked list of students computer self-efficacy responses"

Computer self-efficacy results (table 4.6) show a highly confident group of participants. The students, who should be in at least their second year of study, have most likely been exposed to most of the skills questioned, and these results confirm this. The results show that some participants are not confident writing web pages or blogs, performing troubleshooting, or using programming languages. An understanding of programming languages would make editing wiki-text obvious to the user, however, understanding how HTML generates a web page, and knowing how to format documents with a word processor should be sufficient stills to easily write wiki-text.

Participants were asked about their previous experience and exposure to wikis. No respondent agreed that they knew what a wiki was before participating in this research, however three of the participants reported they were unsure. When asked if they had contributed to a wiki before, three were uncertain, while the remainder were confident that they had not. Realising that many people do not understand that there is a collaborative model behind Wikipedia, participants were also asked if they had ever used Wikipedia. Most (9) participants were unsure, while the remainder reported they had not.

To understand how effectively students learned to use the wiki system, a short wiki self-efficacy questionnaire was completed by students.

Table 4.7 "Ranked list of students wiki self-efficacy responses"

Wiki self-efficacy results (table 4.7) show that all participants were confident using the most important features of the wiki. In each task, one respondent was unsure about participating in discussions, making links, and using lists. Three respondents were unsure when uploading files, and four reported they were unsure how to sign comments in discussion. It was found that discussion was rarely used in the wiki, Uploading files and making lists were features taught and encouraged, but not required. Making links was a skill required of students in completing the pod tasks.

When participants were asked if they had rated any articles, only six report they had done so. As reported above, some members may be hesitant to rate items in a small community and where benefits of rating are unclear. Rashid et al. (2006) found that the number of ratings contributed can be increased by illustrating certain benefits to the user. When asked if they understood how to use the rating system most responded positively, three were unsure, and only three were highly confident. Those who did not rate on average were slightly more confident in understanding the rating system than those who did rate.

The survey attempted to identify if there may have been any bias in ratings. When asked to honestly assess the accuracy of their ratings, participants were largely uncertain of the accuracy of their ratings. One respondent felt their ratings were not accurate, while only four showed any confidence in their own ratings. There was less confidence among participants of other users' ratings. eight of respondents displayed neither agreement or disagreement with the statement that other peoples ratings were accurate.

The exercises completed in the subject were intended to allow students to get to know each other, to work together, and generate friendship between peers. It was therefore pertinent to ask if participants felt they were likely to rate their friends' articles more favourably. A range of responses were received, from strong agreement (2), to strong disagreement (1). Overall two disagreed with the statement, while five expressed agreement.

Overall users of the wiki found ratings to be only slightly useful, one respondent found the system not useful, while five found it only slightly useful.

The survey asked students what factors they felt were important when rating articles. The three factors presented were visual quality, factual quality, and writing style. Students reported factual quality to be the most important factor, with all students expressing either agreement or strong agreement with the statement. The next most important factor identified was the visual quality. For this statement, a wider spread of responses was received from disagree to strongly agree. Writing style returned a similar spread of responses, with a slightly lower average, but still with general agreement. Very few students responded as unsure to any of these statements (one each for visual quality and writing style).

Respondents were given the opportunity to respond freely if they felt there were other factors they felt important. Two respondents made comments that language used in articles should be easy to understand, with one emphasising the importance of clear descriptions. Two respondents expressed that organisation of information should well thought out. Two respondents also said that articles should be written to allow for a wide audience.

One student provided a view on how ratings impact viewers.

Participants were questioned about the Pools of Online Dialogue (POD) exercises they were asked to complete using the wiki. Although there was no strong agreement with the statement, eight reported the pod exercises easy, while the remainder were undecided. Most respondents were confident in understanding the game they chose. eight responded positively to the statement, with one of those a strong agreement. The remainder were undecided with the exception of one who found understanding the game a challenge. When asked what was found difficult about completing the POD exercises, the most common response was unfamiliarity with computer games, with three such responses. Two respondents reported finding a game to study a challenge, however no participant explained the reason for this. One participant also cited poor personal organisation and time management as an issue, while another found it a challenge getting to know the other POD members they were assigned to work with.

Students at CSU have been increasingly encouraged to make use of communication technology in recent years. Among other services, CSU provides subject forums, which are secure (available only to authorised members) web-based forums, where students may communicate with their peers and lecturer. The wiki however presents a much more open form of communication, where students are required to place their own work in a globally available space, where their work may be critiqued by a wide audience. The survey sought to discover if this had an influence on students' behaviour. Students were asked to rate how comfortable they were contributing to the wiki. All students responded positively, with the exception of one who declined to respond. Of those who responded, three students indicated they were strongly confident.

Students were asked to describe further their feelings working in such an open environment. Students were provided with a short list of words, excited, enthusiastic, proud, confident, indifferent, nervous, unsure, confused, and shy, and asked to provide three words, either from the list, or of their own, that describe how they felt contributing content to the wiki.

In total eight responses were received, including about 19 single word answers, and three longer descriptions. These longer descriptions were simplified to single words, or short phrases, creating 23 words or phrases in total. These words and phrases were studied and grouped into five categories, representing different sentiments. These groupings were "strongly confident", "positive response", "neutral", "relief", and "doubting" (see table 4.8).

Table 4.8 "Distribution between categories of students feelings when using wikis"

In the strongly confident category were a total of 11 responses expressing very positive and energetic feelings towards using the wiki (table 4.8). Among the words in this category were "proud" (4 times), "confident" (3 times), "enthusiastic" (2 times), "excited" and "helpful". The positive response category can be summarised by the response "happy once completed". In total five responses fell into this category, the remaining being "good", "unabashed", "achievement", and "good experience". Three neutral responses were received, with two students reporting they felt "indifferent", with the third reporting feeling "blazae". Two students reported feelings of "relief" after submitting to the wiki. Finally two students reported not feeling confident about participating in the wiki, replying with "unsure" and "nervous".

While some students used three words to describe similar feelings, there were also instances of varied feelings in combination. Three students expressed a mix, of a sense of achievement, and of relief, while another expressed enjoyment, while still being unsure about their contributions.

Students were asked to elaborate on their feelings by explaining their choice of words. A mixed response was received. Two responses expressed a lack of special enthusiasm of the wiki as a medium, that the only reason for participation was the completion of their required class-work. "It just felt like placing an article online for people to read", and is "[not] something to go crazy about". Another responded "it was an assignment. I probably wouldn't do it out of my own interest in Wikis".

Two respondents explained their relief came from having completed another university assignment task. One respondent however, as well as expressing relief, explained they were proud to have contributed to their POD group, despite they were unsure how their work would be received by others.

One reportedly excited and enthusiastic student was particularly interested in the outcome of the wiki, stating the activities were "something different", and that they were "keen to see how the Wiki would turn out". Two other students were enthusiastic and interested in the medium. One pointed out the importance of online communities, and that "to share our ideas and arguments is obviously a good thing". Another was pleased to have a space where they could "contribute [their] ideas and knowledge, sharing with other people", and likewise, to "learn different ideas and information from [other people]". "I felt enthusiastic, proud and confident when I posted my contributions".

Throughout the written feedback, there were several general comments about using the wiki. One student explains how they overcame problems when trying to understand and write wiki-text:

This comment shows a level of problem-solving ability, similar to that of hacker cultures (Raymond 2006), a personality believed to be common among wiki contributors.

Several responses, all positive in nature, were received regarding the personal learning experience of using the wiki. One expressed pride over using a wiki in this subject for the first time, while another rejoiced upon the "new learning methods", and knowledge they gained through using online tools.

One student summed these sentiments up in their response:

This response exemplifies several important features of wiki communities. Stallman (1999) explained how small contributions to wikis are important, and how wikis work as a communication and brainstorming technology, where users can contribute ideas a learn from others.

Commands

This section summarises the commands used to generate figures, as indicated in subscript throughout the chapter.

Table 4.9 "Commands used to generate figures"

Conclusion

This chapter presented, analysed and discussed the data collected throughout this research, from both the wiki experiment, and the survey. The next chapter will draw conclusions, recommendations, and present topics for further research.

Bibliography