Towards consistent penalty decisions for breaches of academic regulations in one UK university

This paper investigates the extent to which policies and procedures designed to achieve fair, consistent treatment of student plagiarism are effective in one UK university. The case study uses a model more frequently linked to reliability in assessment decisions to inspect penalty decisions and consider ways of achieving greater consistency. The study notes activities that are important in achieving greater consistency such as sharing assumptions and information between decision makers, introducing specialist officers, and restricting penalty choice. The authors describe a tariff system for choosing appropriate penalties and evaluate its potential for future monitoring of decisions. Areas for continued attention are identified in the case study, because fully consistent treatment cannot yet be demonstrated.


Introduction
calls for higher education institutions "to develop cohesive frameworks for dealing with student plagiarism that are based on prevention supported by robust detection and penalty systems that are transparent and applied consistently" (p. 483-4). In a subsequent paper, he described how his own university rewrote their policy, specified acceptable punishments, appointed specialist officers, and redefined roles for course leaders and assessors (Park, 2004). Other universities are following a similar approach. In a 2005 survey of nine UK Further and Higher Education institutions Carroll and Duggan (forthcoming) report that the primary reason for reviewing policy was the pursuit of effective, consistent handling of cases. Advice on how to achieve this goal was proposed by the authors in 2001 (Carroll & Appleton, 2001) using strategies similar to the approach described by Park and which also stressed the link between consistency and UK Human Rights legislation as well as responsibilities under race, sex and disability discrimination. This connection was reinforced by Deech (2004) who described the interest of the UK Office of Independent Adjudicator's concern about inconsistent and potentially unfair treatment of student plagiarism.
Achieving this goal of fair consistent treatment requires addressing the many opportunities for inconsistency that a student may encounter. For example, Simon, Carr, McCullough, Morgan, Oleson and Ressel (2003) showed that academics' trust in procedures designed to deal with plagiarism has a direct influence on willingness to use them and that their levels of trust varied. Macdonald and Carroll (in press) found the use of different procedures depending on perceptions of the time required to pursue cases. Barnett and Cox (2005) showed that 51% of the lecturers who responded to their survey overlooked cases of plagiarism and collusion and, in the same survey, they documented variations in academics' concepts of collusion. All these aspects and many more will result in variations in how an individual student is treated. In our own experience, we regularly encounter practices that threaten consistency, such as having several markers for a cohort, treating students differently early in their studies compared to later on, academics who hold different expectations of international students, and administrators' who understand different things by the word 'plagiarism'. This list of potential causes of inconsistency is far from exhaustive. However, in this paper, we focus on one potential source of inconsistency: decisions which are made once a case of plagiarism or collusion has been confirmed. For a wider review of issues, see Marsden and Hicks (2003) or Park (2003). We ourselves advocated the development of the procedures to, amongst other things, deliver more consistent, defensible, fair and transparent penalty decisions. Now, five years later, we were seeking evidence that the changes were effective in delivering some or all of these goals. In this paper, we describe the actions of one UK university with 17,000 students and 650 academic staff using procedures that were adopted in 2001 for dealing with confirmed cases of plagiarism.

Inconsistency in punishment decisions
To demonstrate that inconsistency within and between decision-makers was a widespread problem, one of the authors asked approximately 1000 participants at over 30 workshops and conferences in four countries between 2003 and 2005 to consider three cases that would be considered plagiarism in the authors' university.
Case 1: A final-year nursing student submitted a 3000-word piece of coursework, 75% of which was a verbatim copy from a website without in-text acknowledgment of the source or the words themselves. The site was not listed under 'Bibliography'. When interviewed, she defends the practice by saying this is what she has done for the last three years and no one else has said it was unacceptable. Case 3: A mature-entry student (previous 12 years spent working as a chef) submits an essay at the beginning of the second term with highly variable language -some fluent, some choppy and ungrammatical. There is no in-text citation.
Workshop participants were asked to select an appropriate consequence/action from the list below for each student. [All actions except (1.) and (2.) include a discussion and record the decision]: Participants were asked not to confer, to assume they had authority to use all penalties and to write the number of their decisions on a whiteboard. The results consistently showed that wide variation existed within each group and between groups. The overall results are not presented here as the argument for this paper is not which were the most popular choices but what might an individual student receive in this kind of situation. Tables 2 and 3 show two typical results from the workshop exercise and are offered as examples of this variation. The example in Table 2 shows consensus as to which cases were more serious, on equivalent treatment for Students 2A and 2B, and on the need to signal unacceptable behaviour to Student 3. However, what the student might experience will differ greatly since students will not experience the average choice or even the most popular choice for this group. The student will receive one of these decisions and in some instances, this would vary widely. The most inconsistent treatment might happen to [hypothetical] Student 1 as she might have a recorded conversation with her teacher or be expelled (although in reality, expulsion would not be within an individual lecturer's remit in almost all cases). Therefore, Table 2 shows inconsistency within the group and this was universal in the workshop exercises. The same inconsistency was shown between groups as illustrated in Table 3 where another group of 19 were asked to deal with the same cases and possible actions.   Tables 2 and 3, the examples demonstrate that Case 3 in Table 3 was treated both more severely and more leniently compared to the same case in Table 2; there was more agreement within the 19 respondents in the treatment of Case 1 although apparently, two made no decision. Other differences could also be mentioned.

student corrects work, full marks
Workshop participants seemed to take the task seriously and considered a range of factors to reach conclusions each saw as appropriate. They seemed genuinely surprised to discover that others followed the same process yet came to markedly different conclusions. In fact, the variability mirrors other university activities and in particular, grading and marking. Many studies (for example, Lamming, 1990;Newstead and Dennis, 1994;Webster, Pepper & Jenkins, 2000) have shown that individual markers, left to their own devices, will award quite different marks to identical pieces of work. When individual markers are asked to award a grade for the same piece of work at different times, these marks can also vary widely. Interventions to reduce inter-marker variability and improve grade reliability include: • introducing criteria such as how the work is structured, the language used, the currency of sources etc (Price & Rust, 1999); • asking markers to discuss what the criteria mean so as to ensure a shared understanding of what is meant by criteria such as 'current research', 'appropriate structure' or a 'professional level' of language (Saunders & Davis, 1998); • multiple marking of some or all scripts (Dracup, 1997); • moderation to monitor and checking that criteria are used (Baume, Yorke & Coffey, 2004); and • intervening and reacting when consensus is threatened by an individual's decision. Rust, O'Donovan and Price (2005) describe this approach to generating a shared understanding of marks as "a social constructivist approach …. [which] argues that knowledge is shaped and evolved through increasing participation within different communities of practice" (p.232). They provide a model for those such as ourselves who are seeking consistency in student plagiarism cases.

Reducing the number of choices
Before 2000, academic staff in the authors' university used more than 15 penalties for breaches of academic regulations. These were reduced to five. 1. having a recorded conversation with the student and issuing a warning 2. requiring the marker to disregard plagiarised material and assess the remaining coursework 3. requiring the student to correct the piece of work, then resubmit it for reduced marks, either as a stated percentage reduction or capped mark 4. awarding a zero grade for the piece of work 5. awarding a zero grade for the module 1 If the case warranted a more severe punishment than failure of the module, the revised procedure required referral to a University disciplinary committee which could exercise a much wider range of penalties including removal of credit, denial of professional accreditation and exclusion from the university.

Reducing the number of decision-makers
Before 2000, all academic staff (c. 650) were authorised to deal with breaches of academic regulations, whereas after 2000, 17 were designated as Academic Misconduct Officers (subsequently renamed Academic Conduct Officers or ACOs) and charged with determining plagiarism, imposing a penalty from one of the five listed in Table 4 or, if appropriate, referring cases to a university-wide disciplinary committee. ACOs handled all student plagiarism which had been identified by assessors in their own schools and recorded their decisions on a university-wide register.

Ensuring ACOs met regularly
Regular meetings were convened to allow discussion, argument and sharing of practice. Email discussions kept ACOs in contact between meetings, allowing them to pose questions and consult on difficult cases. They created standard letters, refined their data collection and did small-scale studies 2 . Some ACOs were seen as experts who could answer academics' questions and help their schools better manage academic integrity. Over time, a 'community of practice' along the lines described by Rust et al (2005) for assessment began to develop. Half of those appointed in 2000 remain in the role five years later, and induction for replacement appointments has been improved.

Creating a shared tariff of punishments
Anecdotal experience, backed by annual reviews of the data and discussions amongst ACOs from 2001 onwards showed the need for consensus about the circumstances under which each of the designated penalties should be applied. We decided this was best achieved by asking ACOs to first determine which of three levels of breach the student had committed, then designating which of the five penalties were appropriate for each level of breach. We called the match between level and penalty a tariff. Considerable time in ACO meetings from 2001 onwards was given to drafting and agreeing on a paper describing the tariffs. The next section describes the process of determining a tariff, then offers a retrospective inspection of decisions in order to establish a base line for judging future consistency improvements.

Tariffs for penalising academic misconduct
The tariff paper acknowledges that the issues are complex and that there is a need for ongoing discussion. In addition to guidance on selecting penalties, the tariff paper describes 'compound' penalties for students who did not comply with ACO decisions and reinforces the roles of ACOs in relation to Examination Committees. In this paper, we focus on the process for deciding the level of breach of academic regulations relating to the attribution of student work and the criteria for allocating a penalty. The levels were designated as negligent academic practice, academic malpractice and academic misconduct.

When is it a case of 'negligent academic practice'?
This category describes small breaches where there are good reasons to believe this arose through ignorance or carelessness rather than deliberate deception. 'Small' would be demonstrated by: • involvement of only a small portion of the work (e.g., less than 2%) and/ or • involvement occurring in a part of the work of lesser importance (e.g. the methodology section or appendix rather than the analysis and conclusion) and/or • breaches arising solely from misuse of citation conventions (as opposed to inclusion of entirely unattributed material) as, for example, when the student cites sources in the bibliography rather than within the text and if unattributed quotes do not exceed approximately 2% of the total.
'Ignorance or carelessness' would be indicated by: • occurrence early in the student's career at the University and/or • having well-founded reasons to assume the student did not understand academic conventions.
It is not sufficient for the student to assert misunderstanding; the ACO needs to seek reasons to demonstrate it.
'Negligent academic practice' can not be considered if there is evidence that the student had the intention of gaining unfair advantage 3 or the student had already committed any of the three categories of breaches.

Actions for 'negligent academic practice'
All students who, on the basis of evidence presented, are likely to fall within this category are interviewed by the ACO. If the interview confirms the ACO's judgement that negligent academic practice has occurred, the interview focuses on the failings of the submitted work, with the ACO advising the student on how to correct them and future consequences should the student not heed this advice. A record is kept centrally that would allow ACOs in other schools (should the student claim the same circumstances of ignorance and carelessness at some point in the future), to judge that similar treatment would be inappropriate.
Following the interview, the student's work is assessed, taking into account the 'negligent academic practice' that occurred during its generation. The ACO can recommend any of four options: • assess the work as if elements not properly attributed had been so attributed (i.e., signalled as someone else's work) and the work derived from collusion had been attributed to a third party [penalty 1 in Table 4]. • Disregard the elements which are not the student's own work and which are not correctly attributed, then assess the remaining work [penalty 2 in Table 4].
• Require the student to make necessary amendments to their work and re-submit it for a reduced or capped mark as determined by one of the above methods [penalty 3 in Table 4].
Criteria for choosing the best option from penalties 1-3 include the extent of the material not properly attributed, the exact nature of the breach, the tradition and style of the subject area, the time available to the staff concerned with dealing with the case, and above all, the over-riding principle that the student may not gain an advantage (other than the increased understanding arising from discussion with the ACO). A student will learn from amending and resubmitting work, but this must be balanced with the additional staff time which the process would consume. If work is resubmitted, the ACO must check that no other changes have been made and that remarking has not resulted in a mark higher than that which would have been gained by the original piece of work.

When is the work 'academic malpractice'?
'Academic malpractice' can be attributed when breaches arise from misunderstanding but are not excessive. This would happen when: • the extent mirrors 'negligent academic practice' but it is not reasonable for the student to show such a level of ignorance or carelessness. For example, all breaches as part of a third year dissertation and small breaches occurring after a previous finding of negligent academic practice would be deemed 'academic malpractice'. or • ignorance or carelessness is demonstrated but the breach is too extensive to consider 'negligent academic practice'.

Academic malpractice cannot be considered if
• there is any evidence that the student intended to gain unfair advantage • the student's record shows any previous offence of academic malpractice or negligent academic practice • the extent of the breach means that no reasonable person would have concluded it was academically appropriate.
For pragmatic reasons, the last category would be treated as interchangeable with a breach so extensive that disregarding the plagiarised/colluded elements would render the work unassessable.

Actions for 'academic malpractice'
All 'academic malpractice' must result in the student's mark being less than would have been the case if only penalty 2 in Table 4 was awarded (i.e. disregarding of the material in breach of regulations). The penalty should be: • reducing the mark by a specified number of percentage points [penalty 3 in Table 4].
• requiring the student to resubmit the same piece of work for a mark that will then be reduced by a specific number of penalty points [penalty 3 in Table 4]. • requiring the student to resubmit the same piece of work for a mark capped at a specific percentage point [penalty 3 in Table 4].
The ACO must specify the reduction or cap at the time of imposing the penalty. The work is then assessed and moderated through the normal assessment processes. Care is needed on resubmission to ensure the student does not gain a potential advantage by checking the amendments are only those required to remove the breaches and that any cap is designated significantly below the mark appropriate to the quality of the original work.

'…..but sometimes it really is 'academic misconduct'
In general 'academic misconduct' arises when: • the scale of the breach is so great that no reasonable person should conclude that it was appropriate academic behaviour; • the student has a recorded offence of 'academic malpractice' or 'academic misconduct'; and • the student admits or, on the balance of probability, the ACO determines that the student intended to gain an unfair advantage by their behaviour.

Actions for academic misconduct
The penalties under this category cover the widest range. At the minimum end, a piece of work where a first-year student commits a first offence consisting of a relatively small breach, admits attempting to gain unfair advantage and shows contrition, may only warrant a resubmission penalty such as that awarded to 'academic malpractice'. At the other extreme, on a second or even third occasion where a student submits multiple pages lifted directly from the Internet or a first offence that involves a student stealing another student's work and submits it, thus depriving the second student of any marks,would probably warrant referral to a disciplinary committee with a recommendation of suspension or exclusion from the University.
Most cases of 'academic misconduct' have as the appropriate penalty either the award of a zero for the piece of work [penalty 4 in Table 4] or for the module [penalty 5 in Table 4]. Penalty 4 is more appropriate for a first significant breach, or one repeat of a minor breach. Penalty 5 is appropriate where there is clear evidence of the intention to cheat. However, the ACO will also have to consider all the relevant factors such as stage of the student's career, the intentions of the student (i.e., whether they intended to gain unfair advantage), the number of previous offences, the extent of the student's knowledge of the concept of academic misconduct and the effect on the student's progression/ programme. Using the tariff paper, we could retrospectively classify the hypothetical cases cited in Table 2 and deem Case 1 to be misconduct, Student 2A to be malpractice, Student 2B to be misconduct and Case 3 as negligence. The shaded areas in Table 6 reflect decisions in line with tariff agreements, although the penalties themselves and the numbering system between the exercise and tariff paper differ. It would now be theoretically possible (although arguably not useful) to determine the degree of inconsistency in this group of decision-makers.

Retrospective inspection of available data
Once the tariff paper was agreed, we reviewed the records of actual decisions in our own university taken between 2001 and 2004 with a view to seeing how closely they complied with the tariffs and where we might anticipate future deviations. Because data had not been collected in uniform ways across the university or in some cases, in sufficient detail to allow retrospective classification of all cases, we were only able to inspect decisions made in 377 cases. Table 7 shows the percentage of the 377 cases which fell within tariff guidelines.

Discussion
The changes made in 2001 to deliver more consistent penalties proved to be workable and effective in handling a large and growing number of cases of student plagiarism in the authors' university. Reducing the number of decision makers allowed for induction, monitoring and above all, regular communication and discussion between ACOs. However, reorganisation of schools reduced numbers from 17 to 12 by 2004, at the same time that the number of cases rose substantially, leaving some ACOs unable to handle cases quickly. Whilst reduction in the number of decision makers was beneficial, we found that too low a number meant there was a threat that expediency rather than other values might determine decisions.
Reduction in the number of penalties did reduce the opportunity for individualised decisions but nevertheless offered sufficient choice to allow ACOs to take account of the student's individual characteristics and circumstances. However, the original five penalties need further review and refinement. For example, Penalty 2 (Table 4) which requires the assessor to disregard plagiarised material and mark the remainder, has proved troublesome, and unauthorised use of penalties suggest the need to revisit the list. Academics who are not ACOs, and therefore not aware of the reasoning behind the current five penalties, often view penalties as too harsh or too lenient, depending on their own beliefs as to appropriate punishment. Academics in general do explain their reluctance to refer cases on these grounds. It remains important that ACO discussions leading to the tariffs continue, are shared widely and that colleagues' views are sought and considered.
It is likely that the most significant contributor to consistency was the growing community of practice which developed amongst ACOs through regular meetings and shared information about practice. This mirrors experience concerning assessment since consistency in both cases could only occur through discussion of the tacit and implicit knowledge which underpin decisions. It may be significant that an increase in use of unauthorised penalties (i.e., penalties outside the designated five in Table 4) during the 2002/3 academic year coincided with a period when ACO meetings, for a range of reasons, became intermittent, then ceased at the same time as new ACOs were appointed.
Recent introduction of standardised templates for recording each case should allow future reviews to be more complete, detailed and reliable and should avoid the difficulties encountered in this review. Nevertheless, it is clear, even with limited information, that consistency cannot yet be demonstrated. Noticeable deviations from tariff boundaries included: • decisions about 'academic malpractice' that were more severe than the tariff in about a quarter of all reviewed cases.
• decisions that were too lenient for 'academic misconduct' in 20% of the inspected cases in 2002/3 and 14% in 2003/4. The data, although incomplete, indicates a need to clarify and review the level of breach designated for 'academic malpractice' and to focus future discussions on enhancing a shared understanding of its use.
Lenient treatment of misconduct is also noticeable. Where misconduct cases were assigned a recorded warning (penalty 1) or markers were told to disregard the portions of a student's work that breached regulations (penalty 2), records always mention extenuating circumstances that led the ACO to choose penalties outside the tariff. The tariff paper does not mention extenuating or mitigating circumstances as criteria for allocating a penalty but this is clearly a problem. If too much regard is paid to extenuating circumstances and no criteria are agreed for factoring them into the tariff, then consistency is threatened. Alternatively, if no regard to extenuating circumstances is given, fairness becomes difficult or impossible. This issue warrants further attention.

Conclusion
Changes made at one university that were designed to foster increased consistency, transparency, fairness and defensibility have had a positive effect. Penalty decisions are markedly more consistent than was the case in 2000 and other benefits have also occurred, such as the creation of a cohesive ACO group, better and more systematic record keeping, and more efficient administrative procedures. Review of the records points to areas for future effort. However, complete consistency was not achieved and, if more complete data was available, may be even less evident than was demonstrated in the subset of cases reviewed against tariff boundaries for this paper. We are reassured with the progress thus far and plan to continue to monitor the use of the tariffs in future, with the current study serving as a baseline.
Discussions about consistent treatment of student plagiarism rarely if ever define what is meant by the term 'consistent' and indeed, in some instances seem to adopt a meaning for the term which is closer to 'conformity' or perhaps mean consistency within a very narrow range of variation. This is neither realistic nor reasonable in the case of our own university dealing with a large number of cases of student plagiarism. ACOs deal with hundreds of cases, each one of which represents a different and often difficult context, and each requiring management within limited resources. A realistic definition of 'consistency' would take account of this context. Bodies such as the Office of Independent Adjudicators in the UK which have begun investigating and challenging universities on their inconsistent procedures need to be sensitive as to the limits of consistent treatment. However, even where complete consistency remains a goal, it is possible as in this case, to have transparent procedures that explain decisions and that demonstrate ways in which decisions are monitored and reviewed. In our case, this is probably the most significant achievement over the last five years.