Can a policy change practice? An evidence-based approach to developing policy

Universities agree that there is a need to educate students about academic integrity and that the quality of the awards they confer on their students is compromised if students gain credit for work where breaches of academic integrity have been overlooked or not dealt with in a consistent matter. This paper describes how one university developed a new institutional framework and accompanying policy document in order to simplify the processes, ensure that students received educational assistance when required and respond to the dissatisfaction of academic staff with the existing policy and processes. The context for these changes included perceptions in the press that the numbers of cases of academic misconduct were increasing and an ever-increasing reliance on electronic sources of information. The development of the institutional framework and the related policy are analysed using a policy cycle model and conclusions are drawn about the success of the policy implementation based on practice aligning with policy.


Introduction
There are many different views about what constitutes academic integrity and how it should be managed, with some commentators on academia such as Dr Dale Spender saying that universities are wasting their time and resources trying to police plagiarism in the context of unacknowledged appropriation from the internet in the 21st century (Trounson, 2008). Others still see a need for citation and integrity and consider that "'cut and paste' is bad form even in the midst of the electronic heartland" (MacFarlane, 2008). Sharman and Wilshire (2007, p. 506) bring some realism to the debate when they say that anyone who does not think that universities should be in the business of making judgments on academic integrity just does not understand the nature of a university. Most universities have a policy covering academic misconduct in their institution and this was confirmed by a project undertaken by the Teaching and Learning Centre at Murdoch University (2005). The Academic Integrity Project (Teaching and Learning Centre, Murdoch University, 2005) conducted an audit of 42 Australasian institutions with the resultant finding that "almost every institution recognised that there was a need to educate students better about academic integrity, but they also recognised a need to make detection and disciplinary procedures more efficient and transparent." Griffith University is no exception to this finding and this paper will focus on how this particular university sought to improve the way they managed academic misconduct and promoted academic integrity firstly through the development of good policy and then by promoting good practice.

Policy development
Policy development does not take place unless a problem is first identified and then makes it onto the agenda of an institution. Often important issues do not make it onto the agenda because other more publicised issues are receiving all the attention. "There are common steps in how problems develop. Interest groups, officials and politicians identify a particular problem, and strive to make it of concern to the public" (Cobb & Elder cited in Althaus, Bridgman & Davis, 2007, p. 48).
A number of high profile cases of plagiarism at Australian universities (e.g. University of Newcastle in 1998; University of New England in 2006) reported by the media has piqued public interest in the academic integrity of the nation's publicly funded universities. By regularly reporting the number of cases and the ways in which cases are dealt with, the media has taken on the role of watchdog for the higher education sector and forced tertiary institutions to respond. At some stage this response requires the institution to review its existing policy thereby drawing it into the policy cycle. In reflecting on the development and subsequent implementation of a new policy on academic misconduct at Griffith, it is useful to review it against the application of the Australian Policy Cycle model ( Figure 1) described by Bridgman and Davis (2000, p. 24). Bridgman and Davis (2003, p. 99) believe that policy developers are informed by patterns which are helpful in assisting them but "no policy model can claim universal application". Griffith's policy on academic misconduct is a good example of the 'Australian Policy Cycle' in action, with one modification and that is the implementation step duplicated after a formal evaluation. The first implementation was a trial prior to formal evaluation to allow for any improvements prior to final implementation.  (Bridgman & Davis, 2000, p. 24)

Identifying the issues and policy analysis
Identifying the real policy problem to be solved is a challenge in itself, highly influenced by the University's reputation, the potential risk to that reputation, media opinion, bureaucratic convenience and those with access to Executive Management. As Althaus, Bridgman and Davis (2007, p. 50) state: Most issues emerge through these familiar processes but the government agenda can be expanded through extensive use of data and indicators, evaluations of policy effectiveness and a willingness to look beyond the easy subjects.
Highly influential in Griffith's approach to looking beyond the easy subject was Deputy Dean (Learning and Teaching) Professor Anna Stewart, who has an extensive background in criminology and juvenile justice. As a newly appointed Deputy Dean (Learning and Teaching) Professor Stewart consulted with academic staff to identify issues in learning and teaching. They identified their most pressing concern was the reactive, punitive and bureaucratic nature of the University's policy on academic misconduct. This coincided with a Freedom of Information request from the Courier Mail newspaper asking the University to document all cases of academic misconduct dealt with by the University over the past two years. This information was difficult to find and was not documented consistently across the faculties.
The first steps in the process of identifying the issues and analysing policy responses included getting agreement on the problem and then engaging in institutional research. This culminated in the following outcomes:


a comprehensive literature review about the causes of and responses to academic misconduct was commissioned (November, 2006);  Griffith academic staff were surveyed on their attitudes to academic integrity using the UK Joint Information Systems Committee (JISC) (2005) Plagiarism Advisory Service Survey (N=234);  an honours student, James Ogilvie (supervised by Professor Stewart), surveyed students' (N=500) attitudes to plagiarism;  a visit to Oxford Brookes and Lancaster Universities in the UK was undertaken because in the literature they were cited as examples of international best practice in managing academic integrity;  key individuals from across the University were identified and approached to participate in the Academic Integrity Reference Group (AIRG) and assist in developing an academic integrity framework.
This process provided necessary evidence and confirmed earlier anecdotal evidence that the existing academic misconduct policy was unpopular with academic staff and the information about identified cases was not being consistently recorded. Further, it showed that when students were identified, the policy was being inconsistently applied across the academic elements with quite diverse figures recorded. For instance, in 2005 there were 257 cases recorded across the University and in 2006 the number was 109.

Policy instruments
The research conducted in these first two stages of the Australian policy cycle (identify issues and policy analysis) provided the necessary evidence to move to the third stage and develop the policy instruments necessary to address the issues. According to Hood (1983) there are "five common policy instruments used in Australia" (Hood cited in Althaus, Bridgman, & Davis, 2007, p. 89). The University has used two types of policy instruments to construct a framework for promoting academic integrity ("a policy through advocacy") and to manage cases of academic misconduct ("a policy through law"). The objective was to build an institutional response based on the principles of procedural fairness that was easy for staff to use, encouraged them to take responsibility in responding to concerns, ensured that repeat concerns were tracked across the institution and provided students with fair, transparent, and consistent responses. The product of these first three stages, viz. identify issues, policy analysis and policy instruments, was the first draft of an institutional Framework ("a policy through advocacy") that provided for a two-tiered response to academic misconduct. This evidence-based holistic framework focused on prevention and educational responses and was to be centrally administered by an Academic Integrity Manager, to facilitate whole of institution consistency and tracking. The Framework was based on three interrelated foundations. These are the concepts of prevention, promotion, and management.
The two-tiered approach recognises that some students who engage in academic misconduct do so inadvertently because of poor time management, inadequate study skills and lack of familiarity with academic writing conventions. Therefore, Tier 1 cases are handled by the Course Convenor and the policy provides for an educational response such as resubmission, a reduced mark or referral to a learning adviser or to the Student Academic Integrity Tutorial. Tier 2 acknowledges that some students engage deliberately in academic misconduct with the intent to deceive. These conscious, premeditated forms of academic misconduct are considered serious and are formally managed by the Dean (Learning and Teaching) for the appropriate academic group. The Academic Misconduct Policy, "a policy through law", provides for penalties to be imposed on these students such as nil marks, fail grades and, as a last resort, exclusion from the university.

Consultation and coordination
Consultation was an important part of the process as it offers an opportunity to engage in debate and develop a solution (Bridgman & Davis, 2000, p. 25). This allowed the stakeholders to have some ownership of the policy. The Framework was taken to senior University committees and a new University assessment committee was formed to provide ongoing governance. Professor Stewart was a member of a project board that was responsible for implementing 'text-matching' software at Griffith. Liaison with other University support groups facilitated the development of staff and student resources.
During the initial trial in three academic faculties, it was found that the policy was effective and efficient in recording and identifying multiple incidences of academic misconduct, effective in providing information about a student's academic performance and effective in identifying the types of students likely to engage in academic misconduct. For instance, it was usually students with a GPA within the range of 3.5 to 4.5. This data was readily available by interrogating our centralised database. However, there were a couple of areas where greater clarity was required. Academic staff were unsure of the role of the

Decision and implementation of trial
In October 2007, the Deputy Vice Chancellor (Academic) presented the framework to the University Council. Council approved a trial of the Framework in three academic elements (25% of students) and appointed an interim Academic Integrity Manager (Karen van Haeringen). Early in 2008, a full-time Academic Integrity Manager (Jennifer Martin, now called Student Academic Integrity Coordinator) was appointed.
In mid-2008 the trial was extended University-wide. After a reasonable period of time, the trial was to be evaluated with the eventual goal being university-wide implementation. The Academic Integrity Reference Group had now become an Academic Integrity Change Management Group and through representation on this group academic staff from across the disciplines and staff from support areas were able to give ongoing feedback. The Semester 2, 2007 trial identified a number of minor problems with the processes. Academic staff wanted more guidance when deciding on the seriousness of the case and with making consistent responses. In response to this request, the seriousness matrix, developed by Yeo and Chien (2007) at Curtin University was adapted with the permission of the authors for use at Griffith. In order to validate this instrument in the Griffith context, the Deans (Learning and Teaching) participated in a workshop considering a number of hypothetical cases of academic misconduct. The result was a high level of agreement with regard to levels of seriousness and outcomes for students leading to the adoption of the matrix. It has since been validated by academic staff in their responses to questions about the usefulness of the matrix. One of these staff members is quoted below: Yes -this was very useful. It helped me consider the different types of misconduct and compare and contrast the extent of the misconduct across students.
In a survey of Deans and Deputy Deans, Learning and Teaching nine out of ten said they were satisfied with the matrix (Holman, 2009).

Evaluation
During the Semester 2, 2007 trial feedback was requested from every academic immediately after they dealt with a case through a small number of short structured questions. Then at the end of the 2007 trial period, interviews were conducted with randomly selected academic staff that had processed cases in the period from October to December 2007.
In 2008 the University was audited by the Australian Universities Quality Agency (AUQA, 2008) and received a commendation "for the adoption of an evidence-based approach to academic integrity underpinned by research literature and practice". AUQA later followed this up with an invitation to include Griffith's Institutional Framework for Promoting Academic Integrity among Students in its Good Practice Database. In February 2009, to further assess the effectiveness of the Framework after its university-wide roll out, the Academic Integrity Manager/Student Academic Integrity Coordinator prepared a self-review report summarising the development of the Institutional Framework and the implementation process, providing statistical information on academic integrity cases and reporting qualitative feedback from academics that had used the process. The self-review also summarised the University's position at that time using the JISC Plagiarism Advisory Service Roadmap (2005) set of questions. The University further commissioned a review from an external consultant which was conducted in March 2009. The evaluation report recommended that "the policy and processes set out in the Institutional Framework for Promoting Academic Integrity among Students be adopted as the policy and processes for dealing with student academic misconduct across the University" (Holman, 2009). It was also recommended that the University "approve a policy for dealing with student academic misconduct that satisfies all the legal requirements, together with a companion document, a Framework, modified from the existing framework". The policy for dealing with student academic misconduct represents "policy through law" and the Framework represents "policy through advocacy". It is this combination of policy instruments that has facilitated widespread acceptance by the University community of the importance of upholding academic integrity.

Implementation
The University Council approved two new policies to take effect from the beginning of 2010, the Student Academic Misconduct Policy and a modified Institutional Framework for Promoting Academic Integrity among Students, with full implementation universitywide. This implementation coincided with the launch of a new Student Academic Integrity Management System (SAIMS) built as an add-on in the PeopleSoft Enterprise System that allows academic staff to submit identified cases of academic misconduct to an online system which is centrally administered by the Student Academic Integrity Coordinator. All the evidence and correspondence is stored in the system. Academic staff receive continuing support from the Student Academic Integrity Coordinator and ongoing education is provided in the form of presentations to staff, websites for staff and students, a booklet of guidelines for academic staff, good practice guides, assistance with assessment design, job aids and self-paced tutorials on using SAIMS.

Practice
The processes for managing academic integrity at Griffith are set out in the flow charts below (Figure 2).

Figure 2: Student academic misconduct process
Generally, universities situate academic integrity within either of two paradigms, these being the paradigm of crime and punishment and the paradigm of learning and teaching (Hartle, Kimmins, & Huijser, 2009, p. 3) Under the previous academic misconduct policy, the University was managing academic misconduct within the crime and punishment paradigm. The new policies attempt to move the majority of cases into the learning and teaching paradigm with only the more serious ones remaining within the crime and punishment paradigm. One of the influences on this decision was Professor Stewart's academic background in crime prevention. Professor Stewart (2008) argues that there are "two predominant crime prevention approaches: developmental and situational". Therefore, using the developmental approach, Griffith has sought to reduce the risk factors and provide students with the resources that assist them to navigate their way through the conventions of academic writing and also give them the opportunity to learn from their mistakes. However, the situational approach recognises that some students will weigh up the costs and benefits of indulging in academic misconduct and therefore, there must be wellpublicised robust processes for dealing with serious cases of academic misconduct and a perceived greater likelihood of getting caught.
One of the developments that has contributed to the perception that there is a greater likelihood of getting caught has been the implementation of SafeAssign text-matching software. Presently 734 courses are using SafeAssign for submission of assignments and a large number of cases of plagiarism that are submitted to the Academic Integrity Management system have been detected using the SafeAssign software. However, this software can also be used as a learning tool and students are given the opportunity to submit a draft assignment so that they can learn about what may be considered to be plagiarism and seek assistance to produce work that satisfies the academic conventions of academic writing.

Two-tiered process
Our decision to manage cases of academic misconduct using a two-tiered process allows us to move seamlessly between the two paradigms. Often, a concern about a breach of academic integrity is initially thought to be a Tier 1 case but on investigation more serious concerns emerge and the case can then be referred from the Course Convenor to the Dean (Learning and Teaching) for a decision. Conversely, sometimes mitigating circumstances can result in a Tier 2 case receiving an educational response rather than a punitive one. Our centralised tracking system allows the Student Academic Integrity Coordinator to assign cases from repeat offenders to a Tier 2 decision-maker. These cases may appear to be Tier 1 cases initially but the process allows for a firmer response for subsequent offences, particularly where the student has been directed to educational resources and given the opportunity to learn from their mistake the first time. Bretag and Green (2009, p. 13) argue that commitment from those in charge of applying the policy is as important as having a clear policy. Early in the trial, we decided it was important to get the Deans (Learning and Teaching) involved in the development of the policy and the practical implications of consistency of application across the University. They would then promote the policy to their discipline groups.

Academic engagement
Our academic community has engaged with the new processes in a very positive way with a subsequent increase in the numbers of cases being reported. There have been 2,057 cases reported from the beginning of the trial in October 2007 to the end of 2010. This is in clear contrast to the numbers being reported prior to the development of the Framework. So academic staff are clearly aligning their practice with policy which was not the case prior to 2007. Table 1 shows the numbers of cases for 2010 separated into disciplines. In 2010, there were 43,280 students enrolled in the University, so these figures represent only 2% of the total student population. Integral to the success of the process has been the importance of having a central administrator that they can approach for advice.
The position of Academic Integrity Manager is critical to the effective operation of the Academic Integrity Framework. The Academic Integrity Manager is the pivotal person managing each student misconduct case, supporting the academic decision-maker through the steps of the process with forms, draft correspondence (correspondence and forms are now handled online) and advice as needed. The Manager maintains the records, and is hereby able to ensure that a student's past history, anywhere in the University, is taken into account in deciding a penalty. The Manager also plays an important role in raising awareness and promoting the Framework. (Holman, 2009)

Decisions
Data collected on outcomes in 2010 shows that about 75% of the decisions result in educational outcomes for the student (Figure 3). These responses do not mean that the student's marks are not affected. They can be referred to educational resources and also for help from a learning adviser but they may still receive a reduced mark based on marking the portion of the assessment that is not plagiarised or if they are given the opportunity to resubmit the work after obtaining help then they are not eligible to receive any more than a 'pass' mark. The penalties referred to in the data are nil marks, fail grades and exclusions. From Figure 3 it can be seen that although 35% of cases are referred to a Tier 2 decision-maker only 25% of those cases result in penalties being imposed. The reasons for this divergence will include mitigating circumstances being taken into account, remorse or lack of intent on the part of the student and the inexperience of the student. This aligns practice with the intent of the policy to manage the majority of cases within the learning and teaching paradigm.

Appeals
As a percentage of cases managed, the number of students appealing to the University Appeals Committee has decreased. In 2005 there were 23 appeals, 19 of which were upheld out of a total of 257 cases (9% appealed) and in 2006 there were 21 appeals, 10 of which were upheld out of a total of 109 cases (19% appealed). In 2010, there were only 23 appeals, 3 of which were upheld out of a total of 839 cases (2.7% appealed). This would imply that students perceive that the process is fair and equitable and that the decisions made are able to be vindicated when open to the scrutiny of the appeals process.

Concluding statement
These policies and processes have allowed the University to respond to breaches of academic integrity in an equitable and timely manner, foster the continued development of a culture of integrity and reduce the administrative burden on academic staff. They benefit students by improving the quality of their learning experiences and providing an opportunity for academic staff to intervene and direct them to educational resources to assist them. These processes have emphasised the importance of assuring the quality of assessment as policy and practice in the area of educational integrity is dependent on good assessment design. The University is presently developing a new assessment policy that will assure the quality of academic standards and provide new governance arrangements for assessment.