Links are not enough: Using originality reports to improve academic standards, compliance and learning outcomes among postgraduate students

Training students on the interpretation of originality reports generated by an electronic evaluation tool can assist with the reduction of unintentional plagiarism. An initial trial by the Sydney Business School, a postgraduate faculty of the University of Wollongong, has demonstrated that a proactive approach, based on pedagogical principles, can have a positive impact on the improvement of student writing skills when compared to a retributive justice approach reliant on a student's ability and initiative in accessing internet support resources. 
 
This paper argues that higher education should not rely on links to internet-based information, policies, and systems, to educate students in highlighting the seriousness and consequences of allegations of plagiarism. The trial at SBS supplemented the use of an electronic plagiarism detection tool with instructions given by the lecturer, related to the subject assessment tasks, and discussions both on the benefits of using originality reports and how to use these reports effectively to improve students' writing, thus providing positive motivation and consistent academic support and guidance. This paper proposes that this more proactive 'informed' approach can ultimately achieve better results for students, academics, and institutions.

• Students are overwhelmed with information when they first arrive at university, particularly from overseas. Links to plagiarism policies are often not used or understood. Recent literature indicates a shift in Australian universities towards education on good practice in academic writing.
• Sydney Business School first trialled the use of Turnitin in 2006, using it to help students to see where their assignments were over-reliant with direct quotations, whether or not these were correctly acknowledged.

•
The trial was successful both in terms of student perceptions and performance. The trial was extended in 2008 and made mandatory for all subjects in the faculty in 2009. Guidance was given to staff on how to interpret the originality reports consistently and fairly.
• In 2009, a further trial demonstrated how student performance improved earlier and faster when students were given more information and support on academic writing, including how to use originality reports to help avoid plagiarism.

Introduction
New students are bombarded with an overwhelming amount of information when commencing their course of study. This ranges from the provision of general information relating to the institution through to academic processes, procedures, and lecture timetables, before students even start to acquire subject information and understand the expectations and requirements in relation to attendance and assessment in their chosen courses. Students who move to another residence, and particularly another country, must assimilate all this introductory information in addition to finding accommodation, familiarising themselves with new transport systems, and integrating themselves into a new environment, and often another language and culture.
Reliance on the provision of links to institutional and external websites, to ensure individual students comprehend codes relating to conduct, ethics, and academic processes, places the onus of compliance on the student. Links can be seen as evidence which demonstrates that students have been provided with resources to become informed on the definition, use and application of institutional and academic policy. However it is not easy to demonstrate that students actually take the time to access available information, nor is it obvious that students fully understand the purpose or the outcomes intended by a faculty academic policy, such as the use of text matching programmes, e.g. Turnitin, which are used for the identification of possible plagiarism offences. Carroll (2003, p.13) argues that even when this information is highlighted at induction sessions, it is still not enough. Duff et al. (2006) suggest that the remedy for plagiarism lies not in punitive measures but in making Western expectations of scholarship clear. Joyce (2007) carried out an extensive literature review of Australasian research relating to plagiarism and found a shift since 2003 towards education and support. Indeed Devlin (2006) describes a 'sea change' in Australian universities, away from a primary focus on detection, towards a more holistic strategy of helping students to improve their academic skills, including acknowledgement of sources. This is consistent with the approach adopted at Sydney Business School (SBS), a postgraduate faculty of the University of Wollongong, where we use Turnitin as part of a wider approach to improve students' understanding and academic writing.
This paper first reviews the literature relating to student understanding of plagiarism, followed by a discussion of the university's responsibility in relation to educating students, rather than assume they will follow web links and read relevant policies. The paper then describes the application of Turnitin at Sydney Business School and draws conclusions based on the outcomes to date.

Student Understanding of Plagiarism
Understanding the concept of plagiarism, and the functionality of tools that can be used to detect it, are sometimes assumed by institutions and academics as 'given'. Once the links to information and rationale underpinning acknowledgement of sources is provided to students and they have been warned about the serious consequences of plagiarism, the onus is on students to ensure  (Dahl, 2007, p.173). What may be detected as plagiarism can range from minor errors in referencing and citing, confusion about paraphrasing (Keck, 2006), deliberate actions to recycle a student's (or other student's) previous submissions, through to purchasing material to avoid the writing process and meet assessment deadlines (Evans, 2006). Beute et al. (2008, p.203) identified a range of areas where students had difficulties, including: in-text referencing, overreliance on direct quotation, retaining too much of the wording or style of the original in paraphrasing, not using a standard referencing system, not being consistent in citing or referencing practices, not providing full bibliographic details, not accrediting graphic sources, and a general overuse of sources.
A number of these issues should not be classed as plagiarism, but rather as poor referencing, e.g. not being consistent in the format of citations or providing incomplete bibliographic details. Clearly here the intention is to cite the source, but the student has not yet developed the skill of referencing correctly, described by Park (2003, p.475) as stemming from "difficulties in learning the appropriate research and writing skills".
The ease of access to information sources does not guarantee that students allocate time to review governance requirements. This same ease of access is sometimes seen as the underlying cause for increases in events that have the potential to be classified as, or may in fact be, plagiarism. For many overseas students, the actual concept of plagiarism is not readily understood, as the requirement to attribute ideas to originating authors is not promoted, policed, or in many cases does not exist in their home countries or institutions. Other nationalities can have a different perspective regarding the sharing of ideas, or have different words to describe plagiarism that may or may not have the same implication of unacceptable behaviour as it does in the English language. Thatcher (2008) notes that rather than being unacceptable behaviour, the Chinese regard copying as a way of learning from and paying respect to past masters.
A survey of international postgraduate students at an Australian university found the primary reasons for plagiarism were a lack of awareness of Western defined writing and associated referencing skills, and secondly the students' limited language skills which led to their reluctance to re-word what had been written by experts (Song-Turner 2008). In other words, many students are not deliberately cheating, but they have not understood the different requirements in Western universities. Lund (2005) relates the students' difficulty to their different educational and cultural traditions, in particular the reverence for the master, a lack of critical thinking skills, and a concept of ideas as belonging to all, rather than to individuals.
However Maxwell et al. (2008) found no difference in the understanding of plagiarism between international and domestic students in two Australian universities, with students from both groups displaying confusion on the meaning of plagiarism, and similar assessments of how serious/not serious an offence it is.  Brown and Howell (2001) examined how the provision of information regarding plagiarism influenced student attitudes towards understanding what it was. Their research however was limited in that text describing plagiarism policy was provided to students and tested under research conditions. The tests did not examine whether the students could actually locate where their institution provided this information, and whether or not they would actually take the time to read it. Integrating education on acknowledgement practices within a subject appears to be preferable as the context, relevance and importance are clear to students.

University responsibility
Many students appear not to share the same understanding of plagiarism as their lecturers. Nor is the importance of understanding these concepts, definitions and rules appreciated in the early stages of a student's course, and sometimes not until a case of plagiarism is alleged, and a plagiarism investigation takes place. Yet, as Elander et al. (2009) point out, approaches based on detection have limitations and may not lead to students modifying their behaviour. Abasi and Graves (2008) note that university policies on plagiarism contain little information on successful academic writing. Devlin (2006, p. 2) also comments that "policy related to plagiarism contained little, if any reference to an educative approach to plagiarism". The University of Wollongong Acknowledgement Practice /Plagiarism Policy (http://www.uow.edu.au/about/policy/UOW058648.html) does provide many examples of correct and incorrect acknowledgment practice. As observed by Beute at al. (2008) however, having a policy is not enough: students need to be educated about policy intention and application. Similarly, Macdonald and Carroll (2006) warn that statements that the information is available in a course handbook or online would probably be insufficient, if challenged, to demonstrate that the university had taken adequate steps to ensure students understood the policy before penalising them for breaching the policy. Instead Macdonald and Carroll advocate the promotion of good scholarly practices.
Is it acceptable to 'link and forget', or is it the responsibility of institutions and those who work in the Australian higher education sector to ensure that students fully understand the rules and procedures that govern their studies and research as well as the conventions that apply in their particular discipline? Briggs (2003) describes this as a moral approach, presuming we have the knowledge and will act in an appropriate way with regard to the application of a rule relating to plagiarism. However, Vatz (2009) notes that there are many disincentives to academics investigating plagiarism, in particular the time it takes to investigate and respond to appeals, as well as the possible damage to the academic's reputation and their student evaluation scores. In order to ensure equity for all students and to ensure standards are maintained, academics cannot choose to ignore the problem of lack of attribution of sources, hence the use of software to make this task easier is being explored in many universities. One such application is discussed next.

Background
The University of Wollongong's Sydney Business School first trialled the use of Turnitin in 2006. Turnitin is a well-documented text matching system, which allows students and staff to see how much of their assignment is exactly similar to other sources such as information on websites and articles on some commercial databases (Buczynski 2005;Crisp 2007).
Sixty-one international students enrolled in a Master of International Business subject in 2006 submitted their assignments to Turnitin. The assignment was set up to allow students to submit multiple times before the due date, so that students could see for themselves if they had issues. Some students submitted up to six drafts before they were satisfied with their results. All students submitted final assignments with similarity rankings below 5%, this percentage typically made up of repeating words in the essay title or common definitions of the topic. There were huge benefits to students who were not deliberately plagiarising but who either had not understood the need to cite sources in-text or who had not realised how much of their essay was made up of direct quotations/close paraphrasing. A small number of students improved their citations but were still over-reliant on direct quotations. This allowed the lecturer to focus on this aspect of writing and how to use the research the students had found to support their argument and demonstrate critical analysis, rather than spend time investigating potential plagiarism. Students were referred to special Learning Development consultations to help them understand how to integrate evidence in various ways beyond the use of direct/indirect quotations.
Given the positive experience, Turnitin was used in the faculty with a further range of subjects over the following two years. A series of Learning Development workshops were made compulsory in 2008 on the Sydney campus. In 2009 the use of Turnitin was mandated for use in all subjects. Crisp (2007) noted that allowing students to check their originality reports and re-submit can be a powerful teaching tool as students can practice and improve before their work is assessed against both the marking criteria and the university policy on plagiarism, hence this can reduce the incidence of plagiarism. He noted however that only 28% of academic respondents at the University of Adelaide set up assignments on Turnitin to enable this. At Sydney Business School, the decision was taken to make this the default setting. There were some teething problems as staff familiarised themselves with setting up assignments and interpreting reports on Turnitin. In some cases, students waited until the deadline to submit their assignment to Turnitin and did not receive their originality report in time to address the issues identified. Hence, they did not avail of the opportunity to improve the quality of their writing.
As noted by previous authors, e.g. Barrett and Malcolm (2006), Turnitin originality reports must be reviewed by academics before any decisions are taken. Text matching programmes report on similarities, but cannot effectively determine whether a similarity is an actual case of plagiarism.
Through an analysis of student Turnitin submissions over two sessions in late 2008/early 2009, it soon became apparent at SBS that a consistent set of guidelines was required by both staff and students, to ensure equity and to avoid confusion in interpreting the reports. Sydney Business School guidelines now (a) use of incorrect punctuation to identify quotes, or within quotes (b) incorrect formatting of references (c) use of matches due to restatement of the assignment question (d) use of common words, phrases or popular authors (e) a large number of <1% matches (usually due to use of common words or phrases)

Faculty Processes
The guidelines were linked with the University of Wollongong Student Academic misconduct policy which stresses that poor acknowledgment of sources may represent 'poor academic practice or scholarship rather than academic misconduct'. In such cases, the University allows the Subject Coordinator to focus on the education of the student. This perspective is supported by Wheeler (2009) who concluded from his research with students in Hokkaido University in Japan that writing that could be construed as plagiarism was caused by students "lack[ing] the experience needed in order to properly cite sources" (p.25). Crisp (2007) noted a split between academics who regard a plagiarism offence as an 'education and training' issue and those who regard it as an 'honesty and reputation' issue (Crisp 2007, p. 3). This also addresses the problems raised by Flint et al. (2006) when staff have different interpretations of plagiarism, and hence apply their institution's policy in varying ways.
At Sydney Business School, we regard a first offence as an education and training issue, unless the level of plagiarism is high or the student admits they intentionally plagiarised. The lecturer or Subject Coordinator reviews the assignment with the student, checking whether the student understands the need to acknowledge all sources (of ideas as well as quotations), how to reference sources correctly, that direct quotes must be in quotation marks, and highlight that the use of extensive direct or indirect quotes or close paraphrases does not demonstrate critical analysis. This is followed by a referral to Learning Development workshops to ensure that students can apply these concepts in their writing. This process is in line with Pittam et al.'s (2009, p. 154) recommendation for adopting a broader range of approaches to assist understanding of the issues related to plagiarism and the use of plagiarism and writing improvement tools.
It is rare for students to have the same issues in later assignments when they have attended Learning and Development workshops. Sydney Business School guidelines also include a range of possible penalties, to ensure consistency as recommended by Carroll and Appleton (2005) who argue that consistent penalties encourage students not only to comply with regulations, but to adopt the beliefs and values of academic integrity.
Beneficial as this practice has been, having to submit assignments to Turnitin as well as to the Faculty can seem to students like an additional burden. If students do not appreciate how to use Turnitin to help them improve their work, they may submit at the last minute merely to meet the submission requirement. Students are genuinely distressed when they unintentionally plagiarise and are penalised heavily because of it. We are therefore moving to be proactive, explaining to students why acknowledging sources is important, how they can do so to strengthen their argument, and how they can use Turnitin to help them identify any potential issues for example, where sources are not appropriately acknowledged. A pedagogical approach rather than one based on threats can help students' understanding of potential and unintended plagiarism issues and how using Turnitin can improve their writing. Promoting the benefits of using an originality assessment tool "designed to help students avoid unintentional plagiarism"  3) changes the focus from a negative process, designed to achieve retributive justice, to a positive one of improving student and graduate outcomes.

The need for a formal approach
During a discussion about an upcoming assessment in early 2009, a small class of domestic students raised the issue about their lack of understanding about Turnitin, what it did, and how it was to be used. Some guideline work designed to assist Faculty members formulate a consistent method of reviewing originality reports was used to demonstrate how similarity matches are presented, and how the information could be used to improve assignments.
As a result of this discussion, coupled with the examination of class reports from previous sessions, it was very quickly realised that if domestic students had difficulty in understanding the use of Turnitin, international students were even less likely to comprehend the information provided by the system, and the potential benefits available through repeated submissions. It became obvious that the links to web based support materials on the university website were not effective, and that another approach was required.

Pilot
Following on from the informal discussion with the domestic students, the Sydney Business School piloted a proactive approach with two classes of postgraduate students. Again, students from the Master of International Business course were selected for the pilot sessions, and the majority of these students did not have English as their first language. The demographics of the pilot groups are shown in Table 1. , and acceptable events (use of common words and phrases, or a restatement of an essay question). The lecture component of the session was for 20 minutes. A lively question and answer question followed the formal presentation, with students becoming actively engaged in understanding how the system worked, rather than listening passively to warnings that 'plagiarism can lead to serious consequences', when they may not have truly understood what plagiarism actually means.
Linking the workshops to specific assignments helps in making warnings about plagiarism relevant (East, 2006). Warn (2006) suggests that approaches to controlling plagiarism are likely to be more effective if embedded within the course objectives. As McGowan and Lightbody (2008) suggest, educating students on referencing appears to be more effective if it forms part of an assessable component of a core subject, rather than a standalone workshop. Furthermore, having a low value first assignment allows students the chance to improve in their subsequent assignments, which is helpful for students who still need some additional help in fully understanding the requirements. The pilot demonstrated the potential to create a variety of concurrent benefits for students, lecturers, and educational institutions.

Results
Turnitin results for classes exposed to some form of instruction on the system are summarised in Table 2. Class A showed a 5 fold reduction in the similarity values >24% (i.e. from 20% to 4%) achieved by students between Assignment 1 and Assignment 2. This improvement followed a Turnitin tutorial conducted between the assignments by the lecturer. The positive feedback received on the session conducted for Class A resulted in a formal presentation being prepared based on the information covered in the tutorial session, including interactive examples of reports. score of less than 5% (71% of the class), and 39% of the class (or 24 students) achieving a score of 1% or below. The 3% of students whose scores exceeded 25% had not attended the class where the Turnitin session was presented.

Benefits
One of the main benefits achieved through the proactive workshops integrated within the subject was that students gained an understanding of how to utilise the tools actively to improve their writing, rather than complying with a requirement to submit a paper for originality report just prior to the assignment deadline -promoting the concept of writing, reviewing and editing. Errors in referencing formats were highlighted, providing the opportunity for self correction by students prior to final submission. Many students took advantage of this opportunity, some resubmitting their work up to 7 times. The majority of students in Class B resubmitted their work at least 3 times. It could be argued that resubmissions by students were aimed at reducing matched texts to avoid allegations of close paraphrasing or 'cut and paste' copying. However, the fact that the students took the time to resubmit their work a number of times prior to the deadline indicates a shift towards reading, interpreting and acting on results.
Secondly, the lecturer recognised a vast improvement in the quality of original work received from student compared with the results achieved with previous comparable cohorts. This resulted in the lecturer spending less time addressing scholarship errors, together with a reduction in the number of comments required, due to the progress made with the standard of referencing and citation. Of prime importance was the time saved in post assessment interviews as fewer instances of unintentional plagiarism, paraphrasing, and poor referencing techniques were identified. The number of appeals also fell as students could see for themselves if their essay contained large chunks of unattributed quotations and most did not argue with the originality report. The visual nature of the reports, with their colour coding and numerical matching of material from different sources, seemed to make it obvious to students how much or little of their assignment had been written by themselves and how much was taken directly from other sources, with or without acknowledgement.
Some academics may posit that by educating students on how text matching systems like Turnitin operate will only succeed in teaching how to avoid having copied work detected, for example using reverse language processing (double translations), or the purposeful use of poor spelling and grammar to minimise matches. However this notion does a disservice to the majority of students who undertake studies to improve their knowledge and earn their qualification. As educators, we have a responsibility to focus on those students who wish to learn, rather than being distracted by a deviant minority. We should ensure that our systems and practices assist in developing skills, while implementing practices and processes to effectively manage those students who may try to gain a qualification without the appropriate demonstration of individual academic capabilities.
Within Sydney Business School, our approach is to place improving academic writing at the heart of our engagement with our students so that instead of a climate of threats, the students can focus on rapidly improving their approach to integrating evidence in their assignments and improving the quality of their argument. For the university, the approach has the twin benefits of ensuring high standards of academic integrity and raising standards of student academic achievements.

Limitations and Further Studies
The results reported are based on the formal analysis of Turnitin Originality reports for two classes of post-graduate students at one SBS campus, and the informal feedback of students during and after the session. In addition the observations of the lecturers and resulting submissions have encouraged an extension of the pilot. Further pilots, operated under similar conditions, have recently been conducted at the Wollongong campus. The results are yet to be fully analysed, however initial indications highlight similar improvements. Neither pilot included a student or Faculty survey component where the direct opinions and feedback of students and academics were sought. These aspects will form part of a follow up study to be conducted during 2010.

Conclusion
For the faculty, a defined process is now in place to focus on improving academic writing and to ensure a clear understanding of requirements, processes and penalties -a pro-active approach to managing intentional and unintentional plagiarism. This is a positive way to implement university policy and achieve the aims of the policy, which is to eliminate/reduce incidences of plagiarism. It meets what Handa and Fallon (2006) term the moral responsibility of universities to include the development of academic skills within faculty classes, and not only as optional centrally available workshops. Sydney Business School also addresses another recommendation from Handa and Fallon which is to factor in the skills level with which students begin their studies. Students at SBS whose academic skills are weak are educated on how to use electronic tools to improve the standard of their submissions. Those who take the chance to cut and paste from electronic sources have a greater awareness of the risks they are taking and how much easier it is for academics to identify plagiarised work. Students who are putting great effort into ensuring that they do not plagiarise have expressed their pleasure that there is now a level playing field and that fellow students are not 'getting away with it'. A similar reaction was reported by Ledwith and Risquez (2008) who reported, from a study of Irish students, that students perceived their academic environment as fairer since the introduction of Turnitin.
Our approach ensures that students develop an informed responsibility of authorial acknowledgement. Any subsequent failure by the students to ensure that their submissions comply with the codes and standards of the institution can be investigated in the knowledge that the students have not only been informed but actively educated on the requirements and their responsibilities. This limits the opportunities for appeals based on a lack of awareness and understanding, and ultimately provides the institution with an additional layer of protection in terms of breaches of copyright and non acknowledgement of original work and ideas. Due to the encouraging results achieved, the interactive approaches developed will be used at other University of Wollongong campuses to help students improve their writing, with the educational use of Turnitin an integral part of this approach. A research project will include surveys and focus groups of academic staff and students, as well as data relating to the performance of students in their assignments and other indicators such as number of appeals.
The use of text matching software can be a powerful aid to help students improve their writing and to help staff identify potential plagiarism. However academic judgement should prevail, as there are many examples, some of which are cited above, of how a high percentage match can be obtained without any plagiarism taking place. Alternatively, unattributed use of ideas or quotations from books will not be picked up by Turnitin. Consistency in the promulgation and use of these tools is required. The active demonstration of the use and application of plagiarism detection tools to student cohorts is one step in the process. Consistency in the assessment and interpretation of report data by academic staff is another. If the initial results of some minor steps in addressing this issue through the interactive demonstration of the tools are any indication, a concerted effort in helping students understand the topic is a worthwhile investment of time and resources.