Accuracy of referencing and patterns of plagiarism in electronically published theses

Research on accuracy of referencing has shown deficiencies in both students’ and researchers’ efforts to cite accurately. Students’ attitudes and understanding about what constitutes plagiarism has been researched extensively, but there is much less evidence about actual patterns of plagiarism. This study aimed to analyse both accuracy of referencing and plagiarism in electronically published theses. The data consisted of a sample of theses published in Theseus database, which is an electronic publication forum for universities of applied sciences in Finland. Altogether 91 theses in the areas of health and business studies formed the purposive sample for this study. Thesis texts were analysed by comparing the in-text citations to references, and in case of frequent inaccuracy a Google search was used to scrutinise possible plagiarism. The accuracy of referencing was classified in four categories: accurate, some inaccuracy, constant inaccuracy, and misleading referencing/plagiarism. Examples of inaccurate referencing, misleading referencing and plagiarism are presented in the article.


Introduction
In higher education students are supposed to learn academic writing skills and to use and present cited material in text. Accuracy of referencing and avoiding plagiarism are highlighted in numerous writing manuals aimed towards higher education students. The writing manuals guide students in not only how to plan, write and organise a text but they also offer very detailed instructions on how to cite references (how to quote and use direct quotations of sources, how to paraphrase) and how to write a consistent reference list. The students are expected to write in a way that the reader knows exactly what part of the text is written by the author and what is cited from referenced sources. In-text citations and the reference list offer the reader a possibility to find further information and, in the context of education, a possibility to evaluate students' knowledge about the subject. As Masters (2005, p. 282) puts it: "In academic writing, the need to quote and reference accurately is a given. To ensure accuracy, reference systems are designed in great detail …".
In Finland, the writing manuals higher education institutions provide on their websites are fairly homogeneous. Students are advised to cite the sources and give detailed information about the source in the reference list. Plagiarism is not accepted under any circumstances. All higher education institutions have agreed to follow national guidelines for good scientific practice provided by the Finnish Advisory Board on Research Integrity (2012, p. 32) in which plagiarism is defined as "unacknowledged borrowing", and "representing another person's material as one's own without appropriate references". The definition also specifies that "plagiarism includes direct copying as well as adapted copying" (p. 32).

Students as writers
Digitisation of literacy and writing (e.g. Mangen & Velay, 2010) has had a significant impact on the way students in higher education find and use material in their studies. Before, students needed to visit the library where it was possible to explore scientific publications, books and journals. Today, the worldwide libraries have come to students' laptops, tablets and smart phones, as library services are available electronically. A vast selection of electronic books and scientific journal articles are available to be read and also copied by just a click of a mouse (Moore, 2010).
Ondrusek (2012) has conducted a literature review of students' writing skills. Even if there is not much research evidence available about how students in higher education master (scientific) writing, she reminds us that writing plays an integral part in higher education, "and fluency with basic writing skills is a pre-requisite to advanced academic writing. This fluency depends on a student's understanding of how to select a topic, organise ideas, and employ rules of writing mechanics" (p. 184).
Research on student writing has, however, not focused on writing skills but plagiarism on different educational levels. New concepts like cyberplagiarism (e.g. Skaar & Hammer, 2013), electronic source plagiarism (e.g. Ellery, 2008), and online plagiarism (e.g. Selwyn, 2008) have been developed to describe the re-use of existing texts in essays and theses. Skaar and Hammer (2013) report on a study in which upper secondary students wrote an essay with the option of using the internet. They use the concept 'plagiarism as a writing strategy' in a situation where students minimise effort when writing an assignment. The results are telling of cyberplagiarism, as 75% of the participants plagiarised from online sources while writing the essay. Ellery (2008) reports on a study of an essay assignment of first-year geography students, in which 25% of students plagiarised despite explicit instruction in academic writing. Vardi (2012), on the other hand, reports very low levels of plagiarism when only 1.88% of 2,500 first-year university students were reported for any type of plagiarism in two assignment tasks in communication. Pittam, Elander, Lusher, Fox and Payne (2009) conducted a study in which they explored beliefs and attitudes about authorial identity in academic writing among psychology students. They defined authorial identity as "the sense writers have of themselves as an author and the textual identity they construct in their writing" (p. 154). One of the aims of their study was to prevent plagiarism by promoting greater student authorial identity. Plagiarism was discussed in focus groups and connected with mistakes in citation, quotation and referencing. Results of this study showed that after discussions about referencing, both undergraduate and postgraduate students were, to some extent, still confused about quotations and paraphrasing.

Accuracy of referencing
Higher education students are encouraged to read scientific articles in their own subject area, and use the latest knowledge offered in articles especially when writing the theoretical base for the thesis. Articles written by researchers are also used as models of scientific writing. Accuracy of referencing in scientific writing has interested many researchers (e.g. Eichorn & Yankauer, 1987;Haussmann, McIntyre, Bumby, & Loubser, 2013;Siebers & Holt, 2000). In 2003, Simkin and Roychowdhury published the result of their estimation of how many people, who publish in scientific papers, have actually read what they cite. The result is significant as their estimation is that only 20% of authors have read the original text that they cite. Simkin and Roychowdhury based their estimation on misprints in scientific citations, such as false page numbers or mistakes in names of cited authors. As there are multiple possibilities to make a mistake in a number or a letter, only repeated misprints in references were seen as being copied from someone else's reference. Copying a reference is a form of inaccurate referencing and can be seen as one pattern of plagiarism. As Martin (1984, p. 184) has pointed out, one "indication of secondary source plagiarism is the copied mistake".
In an examination of referencing in five leading medical journals Siebers and Holt (2000) found the percentage of errors to vary between 4 and 40%. Eichorn and Yankauer (1987) found 31% of 150 randomly chosen references in public health journals to have citation errors in the author's name, title of article, volume, year or page numbers. Another approach to citation accuracy is to assess if the citation in text is actually from the original source. Haussmann, McIntyre, Bumby and Loubser (2013) examined citation accuracy in 120 articles. The results showed that 19% of citations that were examined did not provide support for the statement of the author. Eichorn and Yankauer (1987) found this kind of quotation error in 30% of references as the citation differed from the original author's text and content. Pritchard (2013) has expressed a concern for the quality of scientific articles offered to be published. From the editor's point of view she has noticed a trend where "authors utilise sloppy and inconsistent citation practices, turning in manuscripts with badly formatted footnotes, missing or inaccurate author names, partial pagination", "concepts we thought were supposed to be learned at a far earlier stage in one's education" (p. 339).
There is less evidence of students' or graduates' referencing accuracy as research tends to focus on plagiarism rather than on accuracy of referencing. Vieyra, Strickland and Timmerman (2013) have noticed that studies about students' use of sources have concentrated on first-year students in university. Studies about referencing and plagiarism among university graduates are rare which may be due to the assumption "that plagiarism issues disappear by graduation as students learn how to appropriately use and cite resources" (p. 36). One study in which the focus is on referencing in finalised theses is a study conducted by Harinarayana, Chikkamanju and Vasantha (2011) who analysed the citation accuracy of all references in five psychology doctoral theses accepted at the University of Mysore. The results showed that 77% of references had citation errors. The errors were classified to be major errors (error in journal, book or author name or error in publication year, volume or pages) and minor errors (punctuation and format errors). Major errors were found in 40% of cited references.
In Finland, no research has been published about the accuracy of referencing in students' or graduates' texts. However, when Kämäräinen (2012) analysed the use of Wikipedia as a source for theses of students in universities of applied sciences, he recognised over 30 different types of information sources cited in theoretical frameworks. He also made remarks of the reference lists, which were unfinished to an extent that Kämäräinen (2012) ends up stating: "It is obvious that the reference lists of the theses in data had not been checked and this can only mean that not even the teachers read them".

Plagiarism in theses
According to Gilmore, Strickland, Timmerman, Maher and Feldon (2010) investigations into graduate student plagiarism have been relatively limited. The focus in plagiarism research has been on undergraduate student plagiarism and especially on students' self-reports. For example Selwyn (2008) analysed 1,222 undergraduate students' self-reports of plagiarism and found that 60% of the students reported at least moderate plagiarism in the past 12 months. Selwyn concludes that "students' use of the internet for the plagiarism of others' work is certainly a prominent and prevalent feature of higher education in the early twentyfirst century" (p. 476). Gilmore et al. (2010) explored plagiarism in graduate students' research proposals. Originally there was an assumption that graduate students were able to properly cite and paraphrase, but the researchers came to discover that plagiarism was frequent. Almost 40% of research proposals contained plagiarism, "including copying and pasting of text from websites, failure to paraphrase, and failure to put quotation marks around direct quotes" (p. 23). When McCullough and Holmberg (2005) explored randomly selected electronically published master's theses in the area of science and technology they used a Google search to find word for word plagiarism in 210 theses. Suspected phrases were searched and potential occurrences of plagiarism were found in 27% of theses. Later, in another study by the same authors (Holmberg & McCullough, 2006) possible plagiarism was found in 68% of 68 theses. Almost 60% of the theses with suspected plagiarism contained more than one phrase (at least seven words) that was also found in a previously published text.
Plagiarism in students' texts was seen as a growing problem in the European project Impact of Plagiarism in Higher Education Across Europe (IPPHEAE), and the project focused on how it is dealt with in higher education institutions. The project found that there is "lack of a common understanding of what constitutes plagiarism" in different countries (IPPHEAE Project Consortium 2013, p. 38). Within the same project, Glendinning (2013, p. 18) paid attention to the lack of statistics of plagiarism in Finland, and stated that "it is impossible to know how much plagiarism is occurring, how much is being found and how much is being ignored". Comparison to the neighboring country Sweden gives light to significant differences in detected plagiarism. In Sweden, a total of 517 cases of sanctioned student plagiarism in higher education in 2011 were reported by Högskoleverket (2012) and 480 cases in 2012 were reported by Universitetskanslerämbetet (2013).

Research design
This study aimed to clarify the accuracy of referencing in electronically published theses of graduates of Finnish universities of applied sciences (polytechnics). In Finland, there is a dual system of higher education consisting of universities and polytechnics (Ministry of Education and Culture, 2014). Both sectors can grant bachelor's and master's degrees, but doctoral degrees are only granted in research universities. Many institutions have started to publish students' theses electronically. Some universities publish master's theses on their own web pages, but universities of applied sciences (UAS) have a common publication forum, Theseus, where students can publish their theses (Theseus, 2013).
The data consist of a purposive sample of theses published in Theseus. In early 2013, 40,000 publications from 25 different universities of applied sciences were available in the database. For the purpose of this study, a total of 48 theses in the area of health and 43 theses in the area of business studies have been scrutinised in order to determine the accuracy of referencing. Both bachelor's theses and master's theses were included in the data. There are seven different areas of study available in Finnish universities of applied sciences, and the data were selected from the largest study areas with most publications in Theseus. From each university of applied sciences, the sixth most recent bachelor's thesis in the area of nursing (n=24) and the third most recent master's thesis in the area of social and health care (n=24) were chosen. The same procedure was used in the area of business studies resulting in having both bachelor's (n= 23) and master's (n=20) theses in the data. The collection of data took place from September to November 2012 in the area of health, and December 2012 to January 2013 in the area of business. All the theses were published in 2012.
The analysis began with one research question and without an exact analysing tool. The research question was: 'How accurate is referencing in theses?' First, an overview on the first theses was taken, and notes were made concerning variation of referencing style and accuracy and consistency of reference lists. After examining ten different theses in the area of health, the analysis frame was fixed. From each thesis in the sample, ten randomly chosen in-text citations were checked for referencing style and accuracy, and the citations were compared to reference list information. Accuracy was seen as precision and exactness in names of authors, publication name, year of publication and page numbers (Buchanan, 2006), both in in-text citations and reference list, and this way only the major errors (Harinarayana et al., 2011) were examined.
If inaccuracy was detected in citation style or in the referencing list, possible instances of plagiarism were found using a Google search. It is acknowledged that only the very obvious cases of plagiarism would be found as no electronic plagiarism detection was used. Attention was also paid to possible copying of pictures and figures from previous texts, but this theme is excluded from the article as it would deal with Finnish copyright law and different interpretations of copyright. The 91 theses in the data set were classified into four categories based on the accuracy of referencing. The categories will be explained with text examples that have been translated from Finnish to English. In examples of plagiarism comparisons are made to sources that are understood to be the original (published earlier). These sources are listed separately after the reference list. The quantitative summary of accuracy of referencing by the level of degree and field of study is presented after describing the categories. The author names or names of institutions which published the analysed theses are not mentioned.

Results
When the author of a thesis followed the chosen referencing style consistently, referencing was categorised as accurate. In-text citations were given in a similar style, cited sources were found in the reference list in the correct alphabetical order, and both in-text citations and reference lists provided relevant detailed information about the source. In the reference list, the same style was used consistently. In 50 out of 91 theses, referencing was accurate. In a further 13 theses, some inaccuracy in citations and references was detected (less than 10 inaccuracies found). Examples of inaccuracies included mistakes in year or pages of publication, mistakes in alphabetical order, inconsistency between citation and reference, referrals to webpages instead of referring to the author, flaws in giving the reader the full reference information, or citations being mentioned in-text but not in the reference list.
The number of theses in which the referencing practice was inaccurate and unclear was significant (n=28). The two last categories, constant inaccuracy and misleading referencing/plagiarism are analysed further in the following sections.

Constant inaccuracy in referencing
In 17 of the theses inaccuracy in citing and referencing was constant. Inaccuracy was evident when the in-text citation style varied, the style of giving the references in the reference list varied, there was notable use of secondary/tertiary sources without reference to the original, or confusing details of the source were given in the reference list. These inaccuracies were further categorised into subcategories: inconsistent reference list, confusing reference, the primary source not identified, misquotation and quotations without quotation marks.
In Figure 1, examples of references from an inconsistent reference list are shown. The references were listed inconsistently throughout the reference lists in theses of this category.

Figure 1. Example of inconsistent reference list
The first reference in Figure 1 refers to a Finnish journal, but the title of the article is missing, the volume is not presented and there are no page numbers either. In the second reference the editor is written in English (ed.) and in the third reference in Finnish (toim.) In the third reference, there is no first letter of the first name for the other author, and again, the title is missing. The year of publication is presented in three different styles: at the end of the reference, after the author name, either with or without brackets. Other inaccuracies within reference lists included: giving the name of only the first author or all authors, giving the first name or the first letter of the first name of the author, giving the title in or without quotation marks, giving the page numbers of the article or only some individual page numbers.
A reference is confusing when information about the source is given differently in different parts of the thesis. The following is an example of a thesis in which a figure from an internet source had been copied. The reference is given, but the page number of the original text is not presented. The title of the figure (translated) is: " Figure 4. Sources of every day customer information behind experiencing value (Korkman & Arantola 2009)". In the reference list the student writes the names of the authors in a different order: "Arantola, H.& Korkman,O." and thus there is inconsistency between the in-text citation and the reference list. In the text there are references to different pages of "Arantola & Korkman 2009", to "Korkman & Arantola" without a year or page numbers and to "Korkman et al. 2009" to a page. The Google search was confusing. There was another electronically published thesis in which, in quite the same way, references were made to Korkman and Arantola, but in the reference list the names were written as "Arantola & Korkman". After further searching it was verified that the source "Arantola & Korkman" did not exist, and that the correct reference should consistently be made to "Korkman & Arantola, 2009".
In the data there are numerous instances in which quotations are written without quotation marks. It is acknowledged that in many countries this would be considered plagiarism (see e.g. Gilmore et al., 2010). However, in this categorisation lack of quotation marks was seen as inaccuracy if correct in-text citation was given (Table 1), but if in-text citation was missing this kind of writing practice was considered plagiarism (Table 2). This interpretation is valid in the Finnish context where no plagiarism cases concerning lack of quotation marks have been identified. The data proves that in many instances the primary source is not identified. Thus, secondary sources and even tertiary sources are used in central parts of the theses. Accordingly, referencing remains unclear. General introductory internet pages were also used as a source. Table 1 presents one example in which the student writes about the main concept of her thesis. Primary sources are not cited and direct quotations are presented without quotation marks. The thesis text is copied from Virtual University of Applied Sciences (2012) open access study material, which has some references to primary sources, but the student does not mention them in her text. The copying here is (almost) word for word with minor changes to the text.
If the in-text citation was incorrect and misleading this was categorised as misquoting. On one page in her 50 page master's thesis the student refers to an article by Almeida, Wethington and McDonald (2001) in the following way (text translated): Irritability, fatigue and stress may prevent the parent to notice the child's needs or tighten the relationship between them. Even good educational principles do not necessarily help, if the parent loses his/her head due to stress and exhaustion. (Almeida, Wethington and Macdonald 2001) There is no other reference in the thesis to this source. The content of the text sample could not be found in the original (Almeida et al., 2001), and thus it remains unknown what the correct citation for this text might be. Many inaccuracies regarding this reference can be found in the reference list information (Figure 2 Master's thesis (2012) Virtual University of Applied Sciences (2012) The concept of well-being is difficult to determine because of the different things to different people produce well-being. It is easily understood as a tangible level of living, even if it is larger whole.
Well-being is seen as an individual, communal and societal dimension. It is also a cultural issue; learned and adopted for common methods and standards of behavior, values, attitudes and values are reflected in the behavior of human well-being. People's well-being is the highest goal to which the entire social policy and the various services of endeavor. (Virtual University of Applied Sciences in 2012.) Well-being concept is difficult to define because different things to different people produce well-being. It is easily understood as a tangible level of the body, though the issue may be a wider phenomenon.
Well-being is seen as an individual, communal and societal dimension. It is also a cultural issue: lessons learned and adopted for common methods and standards of behavior, values, attitudes and values are reflected in the behavior of human well-being. People's welfare is the highest goal to which the entire social policy and a variety of services is ultimately a matter of endeavor.
The first thing to confuse the reader is the matter of the page numbers, as they are given backwards. In this reference the student has made many mistakes. Paternal in the original title has become parental, there are mistakes in writing the title of the journal, name of the third author and in reporting the volume and number of the issue. The page numbers also are incorrect. The correct publication details are: Journal of Marriage and Family 63 (2), 417-429.
Another example of misquoting or of giving a false reference is presented below (text translated). There are two in-text citations in this chapter and neither of them is correct: The minimum number of nursing staff is significant above all for nursing staff stability in the organisation. It has been shown that for each additional patient i.e. increase in workload effects nursing staff 15 per cent increase in job dissatisfaction and 23 per cent increase in probability of burnout. Thus, the use of substitutes has significance in the skilled personnel for work and to coping at work. (Pitkäaho 2011, 113;Aiken, Sloane, Cimiotti, Clarke, Flynn, Seago, Spelt & Smith 2010, 904 -906.) In this paragraph, the student has referred to two original texts: Pitkäaho's (2011) doctoral dissertation (in Finnish) and to an article written by Aiken et al. (2010). On page 113, Pitkäaho (2011) does not write about the issue the student writes about in the paragraph, and the content cannot be found in Pitkäaho's dissertation at all, e.g. there is no concept of 'minimum number of staff' or 'job dissatisfaction'. In the student's 67 page master's thesis there is only one reference to Aiken et al. (2010). The student refers to pages 904-906 of the article (with a spelling mistake in Spetz). However, in the Aiken et al. (2010) article those first pages deal with nurse ratios in three US states, and the relation between nurse-patient ratio and patient mortality. Thus, the issues the student writes about in her text cannot be found on those pages (nor in the whole article). It remains unclear where the exact percentages cited in the text originate from, and why references are made in a misleading way to those two sources that do not contain the information.

Misleading referencing/plagiarism
In this study, in the same way as in the Gilmore et al. (2010) study, plagiarism was not originally in the research design as it was assumed that citation skills are learned before graduation and that plagiarised texts would not have been accepted for publication. Only after detecting repetitive inaccuracies in referencing and finding misleading references, was a Google search used to find text similarities with previously published texts. This method would only reveal the most obvious cases of online plagiarism and there were no means to check offline plagiarism (Selwyn, 2008). Word for word plagiarism was identified in 11 theses of the total of 91 theses in the data. The thesis was categorised into misleading referencing/plagiarism if at least two whole paragraphs in the text were word for word copies of another author's text.
Finding plagiarism in accepted theses created a challenge in categorising accuracy of referencing as occasionally references were accurate -but plagiarised. A typical pattern of plagiarism was cut and paste plagiarism, and text had been copied from several, different types of sources. Plagiarised paragraphs were found either in the theoretical part of the thesis or in the methodology section. In some cases the wording had been changed slightly: synonyms were used, the sentences were shortened or word and sentence order changed.
In Table 2 an example of a double error in referencing is presented: there is neither reference nor quotation marks in a direct quotation. Similar sentences in the thesis were found in a document published by the Ministry of Finance (2008) which presented the customer approach of the national IT service unit. The student had copied unconnected sentences from the original document and formed paragraphs consisting of copied sentences. The Google translation shows where words had been changed into synonyms producing slightly different translations.
There are no references in the thesis when the author writes about customer management (two pages).
In Tables 3 and 4, examples of misleading referencing are presented. They represent examples of secondary source plagiarism (Martin, 1984) because the two paragraphs including the references are identical. In Table 3, the in-text citations are accurate and they are consistent with the reference list in both texts, but it is evident that the student, in 2012, has copied the text with references from the other thesis, published in 2010, and that the references are also copied from the original reference list. Further evidence of plagiarism is that an understandable spelling error ("researcheres"), made by the original author in the 2010 thesis, was replicated in the 2012 thesis. The copied link, however, does not work due to removing one character, namely "-" (a dash). The link in the original authors' reference list is accurate and correct. Customer relationship management strategy is to raise the value of the customer. Customer relationship management is carried out by analysis of customer processes and the creation of customer strategies that enable customers to develop. Customer Focus requires that the service company is constantly analyzing their customer base and develop alternative ways to treat the needs of its customers. Inducing will take care of the company's customer relationship strategy. Inducing will be to monitor, report and analyze on a regular basis. The process outcome is a clear understanding of the customer objectives and the measures to achieve the targets.
"Customer relationship management strategy is to raise the value of the customer." "Customer relationship management is carried out by analyzing the customers and creating customer strategies that enable customers to develop." "Customer orientation requires that services organization providing a continuously analyzing the customer base and to develop alternative ways to take care of their customer relationships." "Inducing will take care of customer relationship strategy. It will monitor, report, and analyze on a regular basis. Design of the final results in a spinal understanding of customer objectives and measures to achieve the targets." One paragraph in a master' thesis (2012) One paragraph in a master's thesis (Korppi & Latvala, 2010) Jones (2004 points out that the structures that may restrict the participation of children, should be amended in such a way that participation is possible and meaningful. It is also important to understand the UN Convention on the Rights of the child's inclusion as a right rather than a necessity. The child should not therefore be exported to childhood and give him too much responsibility for decision-making situations, but he is given the right to participation in an age and level of development. York: Unicef.PDF-dokumentti. http:// www.unicefirc. org/publications/pdf/insight6.pdf Read 14.12.2011Jones (2004 points out that the structures that may restrict the participation of children, should be amended in such a way that participation is possible and meaningful. It is also important to understand the UN Convention on the Rights of the child's inclusion as a right rather than a necessity. The child should not therefore be exported to childhood and give him too much responsibility for decision-making situations, but he is given the right to participation in an age and level of development. (Lansdown 2001, 8).  Table 4, yet another example of secondary source plagiarism, recycling of text within Theseus, is presented. In a similar way as in the previous example, the text and references are copied word for word, and the reference list information is identical, except here a new spelling mistake is made in the word "Publisherss".  Tables 3 and 4 verify that text from an existing source has been taken and used as a graduate's own writing. Previously published texts are copied to text, references of this previously published text are copied to the reference list, and nothing convinces the reader that both authors have written an authentic text due to reading the named source and produced identical paraphrases and citations. The citations are accurate and consistent with the reference list, but still a problem of academic integrity arises: readers are given an impression that both authors have individually read the same sources and paraphrased texts with exactly the same wording and referencing.
Quantitative summary of findings Accuracy of referencing in theses (n=91) was classified into four categories. In Table  5, the categorisation is presented by subject field and level of degree which gives an overview of referencing accuracy in theses. No generalisations can be made as this was a purposive sample and the research design was descriptive. It might be worth noticing, however, that constant inaccuracy, misleading referencing and plagiarism were more common in master's theses than in bachelor's theses. One paragraph in a bachelor's thesis (Partanen, 2010) published in Theseus Knowledge management has become more widely across the last 8-10 years. One might almost say that knowledge management has begun to discuss the gradual, and thus introduced among the people. Knowledge management has become an important topic of the speech in the business community. It is also a prominent networked economy. Competence management can also be considered due to increased competition, mergers and new acquisitions. ( Knowledge management has become more widely across the last 8-10 years. One might almost say that knowledge management has begun to discuss the gradual, and thus introduced among the people. Knowledge management has become an important topic of the speech in the business community. It is also a prominent networked economy. Competence management can also be considered due to increased competition, mergers and new acquisitions. ( The results suggest that the majority of students graduating from universities of applied sciences master referencing at the level that is expected of graduate students. In 50 (55%) theses referencing was accurate and followed the generally accepted norms of academic writing. In the category of some inaccuracy (n=13) the texts were not completely finalised, but major mistakes were not noticeable. In 28 theses of the sample major inaccuracies were found. Constant inaccuracy and misleading referencing are categories that overlap with plagiarism and could be categorised differently, depending on how plagiarism is defined.

Conclusion
Research on accuracy of referencing among university graduates is rare even though forming a research design to analyse electronically published data is simple. Vieyra et al. (2013) and Pittam et al. (2009) share the view that lack of research on graduates' texts may be due to the assumption that graduates have learned to write and cite accurately. The analysis of theses in this study shows that there are major inaccuracies in referencing, as nearly one third (31%) of theses had major referencing inaccuracies, or referencing which could be categorised as misleading or plagiarised. The results are, however, consistent with findings by Gilmore et al. (2010) and Harinarayana et al. (2011) who found significant inaccuracies in graduate students' texts and theses. It was also found in the study by Pittam et al. (2009) that graduate students were still confused about quotations and paraphrasing.
Accuracy of referencing is an issue of academic integrity not only for scientific journals but also for higher education. Researchers of citation accuracy in scientific journals emphasise the responsibility of the author to cite and reference accurately (e.g. Eichorn & Yankauer, 1987;Pritchard, 2013;Siebers & Holt, 2000), and in a case of detected plagiarism in an article the publisher, the journal, would retract the article. Also, in the plagiarism reference tariff developed in Britain (AMBER Project, 2010) to standardise penalties for student plagiarism, the responsibility of accurate citation and referencing lies with the author, the student. According to the tariff, penalty points will be applied even in a case where "below 5% and less than two sentences" were found to be plagiarised. In all cases of detected plagiarism the student receives a formal warning and the assignment is failed. The sanctions would be more severe if plagiarism was detected in a thesis.
The results of this study need careful consideration as they show that in Finnish higher education theses containing major inaccuracies have been accepted and published. The results show evidence not only of inaccurate referencing but also instances of plagiarism in published theses, which forms a sharp contrast between writing instructions and actual practices of accepting theses. The readers of electronically published theses, as well as students preparing their own theses, are sent a confusing message about the standards of writing in higher education.
In addition to the research on plagiarism focused on students' attitudes and selfreports that is already being conducted in many countries, accuracy of referencing might be another useful aspect to look at when studying the way students and graduates of higher education write and cite. Comparative research designs could be a possible solution in developing international standards for student writing at the bachelor and master degree levels.