The assessment of ethics

That Australian universities value the development of qualities broadly related to ethics is evidenced through their inclusion in institutional statements of graduate attributes (GA). Early GA implementation strategies largely emphasised the mapping of specific attributes against existing programmes or courses. There is now a growing acknowledgement that authentic implementation does not occur unless GAs are embedded in assessment. The assessment of GAs is a problematic and challenging task, a situation attributed partly to difficulties in conceptualising GAs in ways that facilitate their operationalisation in teaching and assessment and partly to inadequacies in the development of assessment strategies and instruments. For many academics, the moral dimension of ethics so intensifies the assessment challenge that they are often not assessed at all. While these difficulties are acknowledged, this paper argues the case for the explicit inclusion of ethics in course 
teaching and assessment plans and illustrates some of the contexts, including the student university learning experience, in which the development and assessment of ethics can be undertaken.


•
While some statements refer explicitly to ethical practice in (future) professional lives, others have added application in personal and social contexts.
• Assessment is now acknowledged as providing strongest evidence of successful implementation (Barrie 2004), but is problematic (Hughes & Barrie in review) because of inadequate or naïve GA conceptualisations (Barrie 2006) or inadequate assessment strategies or instruments (Carroll 2004).
• There is a need for explicit inclusion of ethics in planning for teaching, learning and assessment.
• The assessment of ethics involves (1) articulation of learning outcomes, (2) the selection of assessment methods (3) the basis for judgements and (4) the role of students in the assessment process.
• Many current assessment contexts and practices already provide opportunities for assessment of "Ethics" -e.g. contributions to group work.
• Explicit attention to the behaviours that encompass Ethical Awareness can:  more overtly articulate institutional values and expectations in everyday teaching and learning practices; and  support the collection of evidence of GA implementation effectiveness.
Discussion Question 1 What teaching, learning and assessment behaviours suggest opportunities for the development and assessment of ethics?

Introduction
Whether we consciously plan for ethical outcomes or not, they happen. We teach implicit lessons on ethics; we model ethical (or unethical) behaviour or reasoning, often unconsciously; and our practices deliver certain results, both to students and employees as well as to society. What we as universities and university employees do, deliver and produce has an impact. If we do not plan for desired impact, we can become "strategically tragic" -delivering undesirable consequences to the external world and society and to those internal to the university whom we serve as well (students and employees). (Moore 2008, 1) Universities have traditionally included the development of student integrity as an intrinsic, but sometimes implicit, component of their mission. In recent times, this aim has been explicated through reference to some form of ethical outcomes in the statements of graduate attributes articulated by most Australian universities. The term 'Graduate Attributes' is commonly used in Australia to refer to the overarching outcomes of a program of university study of several years duration. 'Graduate' or 'generic' attributes and in relatively common use in Australian and the UK and equate to the 'student learning outcomes' or 'program outcomes' more commonly used in the United States and some European educational curriculum documents.
Policy expressions of Graduate Attribute usually imply a notion of application in relation to the ethical practices of a profession -To understand and apply ethical professional practices: a field of study -An awareness of and sensitivity to ethics and ethical standards on interpersonal and social levels, and within a field of study and/or profession: and, personal life -To respect, understand and apply ethical practices personally and professionally (italicised sections are extracts from Australian University Graduate Attribute Statements, The National Graduate Attribute Project [GAP] website).
Though policies employ expressions such as ethical practice, reasoning or understanding, the more general term ethics is used throughout this paper other than when citing the terminology of particular literature.
A recent study (The National GAP), funded by the Australian Learning and Teaching Council (ALTC) scoped the embedding of graduate attributes in curriculum and teaching. Findings from this study supported Barrie's (2004) earlier proposition that assessment provides more convincing evidence of graduate attribute policy achievement than earlier input indicators such as the mapping of graduate attribute coverage against existing course or program outlines, a practice found to be perfunctory or otherwise problematic (Sumison & Goodfellow 2004).

The problematic nature of the assessment of graduate attributes
However, the assessment of graduate attributes, especially when undertaken for summative purposes, is itself problematic (Hughes and Barrie forthcoming). Knight and Page's (2007)  they include ethical practice, may not be amenable to assessment as it is conventionally understood. The conceptual and practical challenges often associated with this type of attribute are due to the fact that they: • are usually represented by a combination of knowledge, skills, attitudes or dispositions and practices • are seldom able to be fully specified • develop slowly -progress may not be evident within the span of a single semester • require descriptions of criteria and context for understanding of judgements of performance • rarely lend themselves to reporting assessment judgements in terms of marks (summary from Knight and Page 2007, 11).
So, while ethics is considered an important attribute by many academics, when these intrinsic difficulties are considered in combination with other personal and contextual factors such as academics' lack of expertise or interest, and the prioritisation of subject content in an already crowded curriculum (Matchett 2008), low levels of engagement with either the development or assessment of ethics is unsurprising. A recent study (de la Harpe et al 2009) reported that Australian academics consistently ranked ethical practice second last (ahead of only Information and Communication Technology Literacy out of nine attributes listed) for emphases given to teaching and assessment and the confidence and willingness with which they approached these tasks.
The importance of assessing ethics Matchett (2008, 27) however, while acknowledging the magnitude of such difficulties, argues against their acceptance as a justifiable excuse for the omission of ethics from curriculum and assessment as ethical questions are at the core of most disciplines. Knight and Page (2007) also identify disadvantages of failing to assess competencies such as 'ethical practice' as what is not assessed is not taken seriously by students or by teachers: employers seek information on such attributes from applicants: governments expect evidence that these attributes are promoted and assessed: and, assessment helps students and teachers identify ways of enhancing development (summary from Knight and Page 2007, 13).
Even among those who consider ethics an important attribute to be developed through the university experience, there is a belief such development will occur through socialisation and that intervention, such as explicit attention through teaching and assessment, is unnecessary. Of course, the socialisation model does hold true to some extent as what has been termed the hidden curriculum (Snyder 1973) -non-academic experiences and interactions with academics, administrators and peers -shapes much of what students learn about ethical behaviour at university (Matchett 2008). Implicit, socialisation approaches however ignore the strong relationship between ethics and cognition and this is the basis of the argument for a more overt and systematic approach to embedding ethics in curriculum and assessment. It should be noted that this discussion addresses ethics as a graduate attribute; the teaching of Ethics as a field of study is a matter of adherence to the same general principles and practices of effective teaching and learning as apply to any discipline area and therefore beyond the more general focus of this paper.

The assessment of ethics
Embedding ethics in assessment involves decision-making in ways appropriate to a particular educational context or situation. These decisions involve the selection and articulation of learning outcomes, the ways in which students will provide evidence of their achievement of the learning objectives, the basis of assessment judgements and the ways in which students will be involved in the assessment process.

Learning outcomes related to ethics
One of the key problems in articulating ethics learning outcomes is that graduate attributes both collectively and individually may be conceptualised in quite different ways. Barrie's (2006) research identified four distinct orientations to graduate attributes. While the first, precursory, can be discounted as having any relevance to this discussion, the remaining three suggest possible alternatives for framing the outcomes of a curriculum with explicit attention to the development of ethics: • Complementary -a generic skill set that is separate from and secondary to the learning of disciplinary knowledge: complementary to but not altering or interacting with disciplinary knowledge in any way. • Translation -an ability to translate or apply disciplinary knowledge: closely connected with and shaped by the parallel discipline learning outcomes so not generic but characteristic of particular disciplines and allowing the application of discipline knowledge in the real world. • Enabling -abilities that transcend disciplinary boundaries though initially developed within a disciplinary context; enable students to reshape and transform knowledge to meet new challenges in contexts far removed from that of the original discipline; intellectual and personal attributes that are the keys to enquiry and learning in many aspects of life, not just formal study. Translation approaches which prioritise application to professional responsibility may also extend this conceptualisation into the area of enabling through the inclusion of references to broader societal outcomes. Steneck's (1999) conceptualisation of ethics as encompassing (1) technology and society (2) engineering and society, and (3) ethical reasoning, and with outcomes related to an understanding of the inter-relationships between technological development and the welfare of individuals, society, and the environment (ibid 13) is an example of this.
The four areas of 'learning outcomes for ethical deliberation' summarised by Matchett (2008, 35 citing Ozar 2001 correspond even more closely to Barrie's (2006)  Each distinct orientation to ethics implies a particular approach to teaching and learning as well as assessment. The following discussion, in many respects, therefore applies to teaching and learning as well as to assessment based on the acceptance of Boud's (1998) assertion that a worthwhile assessment activity also serves as a worthwhile teaching and learning activity.

Generating evidence of achievement
Assessment is the making of judgements about the quality of learning based on consideration of evidence of achievement in relation to criteria and standards. The role of the teacher is to provide opportunities for students to generate evidence through assigning formal assessment tasks or through drawing student attention to other contexts (e.g. work placements, service learning, study abroad, student clubs or associations) in which evidence of learning may be generated. The student role is to generate appropriate evidence and make it available for judgement. This is generally undertaken by teachers, tutors, peers or students themselves, or by other stakeholders (industry representatives, workplace supervisors) who are assuming greater roles in assessment processes in some (mainly) professional programs.
Numerous methods are available for assessing the cognitive and affective dimensions of ethics in formal ways. In going beyond the traditional examination or essay, Balogh (2002) suggests alternatives such as responses to case studies, the recording of observations in media diaries, journals or logbooks, role plays, skits and debates. However, no specific form of assessment is inherently appropriate for any specific conceptualisation of ethics. A written examination could be used for each of Barrie's (2006) conceptualisations listed above but each examination 'instrument 'would require a distinct set of student responses that would constitute qualitatively different forms of evidence of achievement. Validity would be assured by the extent to which the assessment task, and consequently the evidence generated, was aligned with intended learning outcomes (Biggs 2002). Steneck (1999) has illustrated an attempt to align Criteria (Learning outcomes) and Assessment tools (tasks) in the integrated engineering ethics curriculum adopted by the University of Michigan. As Table 1 illustrates, in this course the assessment of ethics draws on a number of different strategies and techniques.  As the assessment examples in Table 2 illustrate, evidence of achievement of the cognitive dimensions of learning may be generated through familiar tasks such as a quiz or contextually appropriate formal examination. However, the affective dimensions of attributes such as ethics pose a different type of assessment challenge. The critical incident reflective essay task designed to assess Critical disposition was selected because of the strength of Knight's (2002) argument that some aspects of student achievement such as disposition or behavioural intentions cannot be warranted in the normal way through traditional or formal assessment. Instead he proposes that students are enabled to lay powerful claims to achievement which they could substantiate with material drawn from (sources such as) learning portfolios (ibid 7) or reflective journals. This approach allows the assessment of ethics to be based not only on evidence generated by students as a response to the formal assessment tasks but also permits the inclusion of evidence generated as they observe or confront authentic ethical dilemmas and apply their knowledge and cognitive skills. The reflective task assigned effectively requires students to make a claim that they have developed a critical disposition and to offer evidence in support of that claim. Knight (2002) stipulates that the success of such approach is dependent on a number of factors including many opportunities for low stakes formative assessment, the provision of effective feedback, the development of grade indicators including threshold descriptors to complement the learning outcomes and the development of student capacity for self and peer assessment. Alverno College in the United States is a frequently cited example of this approach in practice (Loaker 2000).

The basis of assessment judgements
The development of grade indicators or threshold descriptors to be used as the basis for the assessment of ethics has been approached in various ways. Kohlberg's (Kohlberg et al 1983) 'Stages of moral development, ' and Perry's (1970) 'Scheme of intellectual and ethical development' are commonly cited resources specific to this area. In addition, the more general Bloom's taxonomies in the cognitive (Krathwohl 2002) and affective (Krathwohl et al 1964) domains are applicable in articulating the basis for assessment judgements. The Association of American Colleges and Universities (2009)  Four standards or levels are described for each of these criteria. At the time of writing the ethical reasoning rubric is in draft form with feedback invited which may address some of the drawbacks common to rubrics in which the distinction between levels is determined by a single word in each verbal standard -hence Ethical perspectives/concepts may be applied in fully, adequately, partially, or minimally accurate ways -a distinction whose meaning is unlikely to be shared by those who have no opportunity for some form of moderation or access to exemplars.

The role of students
An active role for students in the assessment process is essential for successful outcomes of any programs seeking to develop ethics. Students will only develop ethical sensitivity, reasoning or practice through opportunities to consider the ethical implications of their own and others' actions; to apply frameworks and processes to ethical decision-making and to reflect on and evaluate the basis of their own ethical choices in a range of authentic contexts.
Ethical reasoning is involved in much of the decision-making that occurs in personal and professional lives which means that there is an abundance of material suited for selection as the basis of student learning and assessment activities. As a result of identifying of a gap between students' responses to an ethical question posed in two different contexts -(Would they report another student for cheating? When on the job, would they report a colleague for falsifying reports?) - Steneck (1999) suggests examples from the context of students' university experiences as a potentially engaging introduction to ethics and that the consideration of professionalism and professional codes is more effective if delayed until students are further into their professional course. Many aspects of student life suggest issues and dilemmas that can be used to develop  Moon 2005), group work behaviours such as social loafing and the issues raised in Taylor's letter to his student s (e.g. 'putting yourself in a position to make fruitful contributions to class discussion', respecting other students' 'opinions', giving full and proper credit to sources').
While students can be actively engaged in the exploration of the ethical practices applicable to future professional life through many class-based activities, work placements, internships, service learning are real-world contexts containing examples of ethical conflicts and competing values and therefore provide more authentic contexts for learning and assessment tasks Balogh (2002).
On broader ethical issues, print and electronic media are a rich source of material for the contextualisation of learning and assessment activities. Mladenovic (2009), for example, reported success in developing accounting students' ethical reasoning through following an ethical conflict resolution model and a variety of ethical perspectives to explore ethical issues reported in the press. Background material addressing the ethical issues associated with particular areas of professional practice (e.g. 'Accounting and auditing', Health and medicine') or topics of broader social significance (e.g. 'Globalisation', 'Indigenous issues'), readily available on the website of the St James Ethics Centre (see also Longstaff 1995), constitutes a useful resource for academics and students.
Whatever the context in which learning and assessment are undertaken, it is important that independent, lifelong learning is encouraged and supported. Assessment needs to develop student knowledge and cognitive skills and their capacity to understand and apply meta criteria and standards to their own ethical reasoning practices and decisions and to those of others and this will only occur when they are active rather than passive participants in the assessment process (Boud 2000).

Conclusions
It is as true for the development of ethics as for the development of any other graduate attribute -critical thinking, communication, creativity -that teaching and assessment are most effective when a whole of programme approach is taken. While fragmented modules or courses will impact on student learning to some degree, the strongest message we can give students that the development of ethics is valued is for it to permeate an entire program. The 'Ethics Audit' available through the Higher Education Academy's Centre for Bioscience (2008) website has been designed to 'help teachers considered the content and design of a programme of learning with respect to the relevant ethical issues appropriate to the discipline'.
This exploration of the assessment of ethics, while acknowledging the problematic nature of this undertaking and identifying key barriers, has argued that the development of ethics is too important to be left to chance. A range of literature has been drawn on to illustrate different orientations to the nature of ethics, types of assessment activities and contexts for learning and assessment. It has also been argued at the active engagement of students is essential to the success of this enterprise. (In the words of a student) the graduates of today are going to