An investigation of assessment in fully asynchronous online math courses

This study is based on a voluntary survey of all listed Fall 2006 State University of New York (SUNY) Learning Network (SLN) mathematics faculty. The survey focuses on assessment in SLN math e-learning. Based on survey data, the study attempts to answer the following questions: What percentage of math e-learning courses overall, by course category and by institution (two year or four year) are proctored? In proctored and unproctored math e-learning courses, which assessment instruments are utilised and how? Comparing the two approaches, how does the practice of assessment in proctored math e-learning courses compare to the practice of assessment in unproctored math e-learning courses? Finally, what are the ten most popular strategies for assessing math e-learning courses? The paper concludes with questions, suggestions for further research and implications regarding the academic integrity of math e-learning courses.


Introduction
The SUNY Learning Network (or SLN), recently recognised by the US Distance Learning Association (USDLA) for best practices in distance learning, is one of the nation's and world's leading online learning networks with more than 100,000 enrolments annually (SLN 2006). For this reason, SLN provides an ideal field for the study of Internet-based education or e-learning.
Of particular interest is the arena of online pedagogy in the instruction of mathematics. Given the young age of e-learning there is little surprise that the research base is slim at best (Engelbrecht & Harding, 2004). Engelbrecht and Harding, writing in their two-part article, Teaching undergraduate mathematics on the Internet (2004), state "research on this mode of teaching is sparse and open research questions are plentiful…little has been done in developing a pedagogy for online mathematics courses..." (p. 1). Along with recognition of the need to create and expand this research base, there is also growing recognition of the need to use new pedagogical approaches when a course is to be delivered in an e-learning modality (Smith & Caris, 2003). Online instructional designers commonly voice the fact that successful online courses are generally not carbon copies of their bricks and mortar counterparts.
While there is much to learn about how we can effectively teach math online, as a major component of pedagogy, how do we successfully and effectively assess learning outcomes? One question that looms large is whether we can successfully assess our students in an entirely unproctored online format. In other words, is the purist view of online instruction, which encourages the maxim of 'anytime, anywhere' which by implication, discourages the use of proctored assessments, applicable to math e-learning? Further and more pointedly, is it applicable to different subset categories of math courses but not to others (for example, developmental versus calculus versus liberal arts)?
As this modality expands, the purpose of this survey is to take a snapshot picture of the evolution of assessment in math e-learning. Three main questions will be addressed: 1.
What percentage of math e-learning courses overall, by course category and by institution (two year or four year) are proctored? 2.
In proctored and unproctored math e-learning courses, which assessment instruments are utilised and how? Comparing the two approaches, how does the practice of assessment in proctored math e-learning courses compare to the practice of assessment in unproctored math e-learning courses? 3.
What do current online math faculty identify as important assessment practices in math e-learning?
Because "assessment drives what students learn… [and] show [s] students what we value and how we expect them to direct their time" (Smith & Wood, 2000, p. 126), by investigating how assessment instruments are used and weighted, we should be able to obtain a greater general understanding of the present nature of math e-learning.

Study premises
• Learning outcomes are course specific standards of mathematical understanding set by individual faculty, departments and/or institutions. • E-learning faculty, like their bricks and mortar counterpart, are responsible to act as agents of their institution to certify the degree to which learning outcomes have been successfully achieved.

•
Various assessment instruments, be they proctored or unproctored, are utilised to measure the degree to which a student has achieved course specific learning outcomes.

•
While pedagogically, the learning process (reflected in formative assessments) is of importance and value, at least equal to or greater than the learning product (reflected in summative assessments), summative assessment measures in mathematics, by definition, ultimately provide the most accurate overall measure of a student's cumulative mathematical understanding. This is to say that, while the learning process is an objective, it does not necessarily reflect the desired learning outcome.

Background
Formative versus summative assessment Assessment can be separated into four categories: diagnostic, formative, summative and accountable (Engelbrecht & Harding, 2004). An example of a diagnostic assessment instrument is a placement test. Examples of formative assessment instruments are homework, discussion, quizzes, projects. Summative assessment instruments include tests, mid-terms, final exams, capstone projects. Attendance may be an example of an accountability assessment instrument. It is clear from these categories that formative and summative assessments typically form the overwhelming bulk of a math student's semester grade.
When focusing on formative and summative assessment, the following distinction is useful: "summative assessment is intended to summarise students attainment at a particular time, whereas formative assessment is intended to promote further improvement of student attainment… [that is] assessment of learning versus assessment for learning" (Crooks, 2001, p. 1). Formative assessments, as the word root implies, serves the primary role of helping students to form an understanding of course concepts and material. Summative assessments serve at least two primary purposes. They help students synthesise course material. They also provide both the student and the instructor an accurate measurement of how (and whether) the learning outcomes have been achieved (Straight, 2002). Tests and examinations, traditional bricks and mortar summative assessment instruments, are generally thought to be critical and primary components of traditional math assessment (Bonnice, 1999;Wood & Smith, 1999;Ross, 1999).

Traditional bricks and mortar versus e-learning instruction
As previously mentioned, there is growing recognition of the need to use new pedagogical approaches when a course is to be delivered in an e-learning modality (Smith & Caris, 2003;Rovai, 2000). Online instructional designers commonly voice the fact that successful online courses are generally not carbon copies of their bricks and mortar counterparts. Rovai (2000), writing in support of this view, states "online instructors must recognize the need to design instruction appropriate to the medium" (p. 141). Further, he states that there must be some actionable recognition that where face-to-face interaction is limited or does not exist, then "assessments become even more important" (p. 144). This is to say that while traditional assessments "usually require that all learners in a class be tested through the same standardized procedure at the same controlled location" and while these traditional assessments are broadly considered reliable and support the current standardsbased reform movement, it remains unclear how to effectively assess e-learning students (p. 142). Echoing this sentiment, Engelbrecht and Harding (2004) write that "some of the most serious errors in the educational design of web-based courses have been…focusing on content rather than outcomes" (p. 3).

Four important characteristics of general e-learning and their influence on math e-learning
When focusing on assessment in fully asynchronous math e-learning, four important characteristic aspects of general e-learning pedagogy must be discussed. These characteristics have a significant influence on the nature of assessment, particularly in math e-learning.

Discussion threads and writing-based assessment
First, the tendency for e-learning, in general, is to rely heavily on discussion threads and "student-centered active learning and performance assessments" such as essays, projects and portfolios (Rovai, 2000, p. 3). These activities tend to naturally lend themselves to writing-based courses (Smith & Caris, 2003) where issues of academic honesty are at least partially addressed when one considers the concept of a "writing fingerprint" (or "intellectual fingerprint") (Jones, Taylor, Irvin, & Faircloth, 2001, p. 10). This concept is based on the idea that every student has either a static or gradually evolving 'fingerprint' that instructors can compare to all other student writing submissions. By this means, instructors can verify academic honesty. Math faculty can piggyback on this concept by including writing-based assessment instruments such as discussion components, projects and portfolios. Relative to assessment, there is little change for writing-based courses which naturally use such instruments for summative purposes in their bricks and mortar counterpart. In math elearning, where summative assessment instruments are typically tests and exams, the idea of using these instruments in a similar fashion may be less clear. Certainly major math projects have promise. They support reform efforts to move away from a "computation-laden… to a concept-driven curriculum" (Vidakovik, Bevis & Alexander, 2003, p. 1). They enable students to "engage [in] in-depth learning on wider areas of their personal interest" (Hibberd, 2005, p. 6). However, related to assessment, this question remains: is it widely feasible for projects or to what degree can projects act as 'a' or 'the' major summative math assessment instrument, replacing the traditional summative instruments (tests and final exams)?
'Open book' tests Secondly, among those actively involved in instructional design for e-learning, there is some wide agreement that all unproctored online assessments should be designed as if every one were an 'open book' test (Holland, 2000;Christe, 2003;Griffin & Clarke, 2002;Baron & Crooks, 2005). For math e-learning, this means that every unproctored assessment should be treated essentially as though it were a take-home test. For comparison, at Australian universities, such tests (open book or take-home) are rare, but what is common is "the use of restricted open book, particularly where students are invited to take an A4 sheet of handwritten notes into the examination" (Wood & Smith, 1999, p. 156). This view, of the rarity of take-home or open book tests, is shared by this author in his experience teaching at three SUNY junior colleges. Similarly, a 1999 US News and World Report article, entitled The Cheating Game writes of a University of Western Ontario bricks and mortar professor that "eliminated take-home exams…after he caught students collaborating on them" (Kliener & Lord, 1999, p. 4). Given the almost universally acknowledged rising rates of cheating (McCabe, 2005;Lathrop & Foss, 2000;Gibbons, Mize, & Rogers, 2002;Kleiner & Lord, 1999) it is not difficult to infer that students are likely obtaining some form of unauthorised help on take-home exams.

Student collaboration
The inference that students are probably obtaining some form of unauthorised help on take-home exams is given much more credence given this third characteristic: general e-learning tends to emphasise and encourage student-to-student collaboration. Related to math e-learning assessment, this kind of activity provides students with "an excellent excuse of habit for unauthorised collaboration on assessments" (Rowe, 2004, p. 4). Certainly collaboration can be a valuable pedagogical tool for both online and traditional math courses. However, with regards to summative assessment in mathematics, any form of collaboration is typically prohibited and is, in fact, labelled as cheating. If general e-learning tends to emphasise and encourage collaboration, how difficult would it be for math e-learning students, in an unproctored course format, to 'switch' gears from collaborating on some assessments and not on others? Given the e-learning propensity to emphasise collaboration, rising rates of cheating and the moral pragmatism with which cheating tends to be viewed, what would prevent students from minimally seeking and receiving coaching on their tests?

Issues of psychological and physical proximity
The last major characteristic centres on the issue of physical and psychological proximity (Bauman, 2002, p. 2) or the "feeling of detachment" afforded by the online medium (Gibbons, Mize & Rogers, 2002, p. 4). This reflects on two characteristic differences of the e-learning versus traditional bricks and mortar medium. First, there is the inability, in e-learning courses, to actively and physically 'witness' a student engaged in the formative or summative learning and assessment processes. This process of "witnessing" can serve to validate measurements obtained by various assessment instruments. Second, this reflects on the deterrent effect of the 'live' physical presence of an instructor. When a student considers acts of academic dishonesty, the e-learning medium relies much more heavily on an individual student's integrity. Whereas, in the bricks and mortar classroom, issues of personal responsibility may appear more lucid as they are reinforced by the mere physical presence of the instructor acting as the certifying agent. While these points of differentiation reflect previously mentioned statements regarding the need to utilise a new pedagogy tailored to the e-learning medium, they also present serious questions regarding the integrity of traditional summative instruments utilised in an e-learning environment.

Cumulative impact of these four factors on math e-learning
Given the idea that, for many math instructors, summative assessment (using instruments such as tests, mid-terms and final exams) is of critical importance, that these math assessment instruments are typically not writing-based (to be able to effectively piggy back on the concept of a writing fingerprint), that take-home math tests are generally rare and results are dubious, that summative math assessment instruments typically prohibit collaboration, and that the online medium relies much more heavily on student self-accountability in issues of academic integrity, one would naturally question the validity of unproctored assessment instruments in math elearning courses. This is the source of great debate as one considers creating and teaching '100% online' math courses. The requisite question in math e-learning is whether it is possible to administer unproctored tests, mid-terms and/or finals in a format and/or fashion that still preserves the integrity of the educational system and the quality of its educational outcomes.

Proctored versus unproctored assessment
Part of the answer to the previous question may be found in a related quantitative study recently conducted by Gregory Wellman of Ferris State University. Wellman (2005) cites numerous articles that have expressed concern that "unproctored, online, asynchronous testing would be compromised by students collaborating without consent of the instructor" (p. 20). In his study he looked at proctored versus unproctored testing in a medical terminology course. While such a course is clearly outside the math discipline area, both courses are similarly objective in nature. This study, while not exploring cheating directly, examined the effects of unproctored courses where there is only self-accountability versus a proctored course where there is outside accountability. The question he poses is how the level of outside accountability impacts the outcome of student learning or the effectiveness of the course (i.e. a focus on success in achieving learning outcomes not student integrity). Wellman concludes that "unproctored testing, paired with online content delivery, was less effective in promoting overall learning and reduced time-on-task for some students" (Wellman, 2005, p. 36).

Participants
Three short survey questions were sent out to approximately 120 math faculty members listed on the SLN website (representing approximately 180 sections of fully asynchronous online SLN math courses being taught in the Fall 2006 semester). While SLN faculty email addresses were readily available, to better the response rate, all faculty institutional email addresses were researched and used for the survey. These addresses were used with the rationale that SLN addresses are most likely secondary faculty email addresses while institutional email addresses are most likely primary email addresses.

Survey instrument
The three questions sent out by email were: Other 2 (please specify): ___, Other 3 (please specify): ___ Question 2: Are any of your assessments proctored? If, yes, which one(s)?
Questions 3: Any words of wisdom you can share regarding how best to assess fully online math students?
Sampling procedure The first email survey was sent out to approximately 120 email addresses on Friday 20 October 2006 with approximately fifteen emails returned as undeliverable. In the following week the undeliverable faculty email addresses along with about three other previously unknown faculty email addresses were researched and new emails were sent out.
Five days later a second email was sent out to all 120 email addresses. This email was sent out as a thank you (just over twenty responses had been received) as well as a reminder requesting at least ten more replies. Additional effort was also made in the second email to allay any concerns related to faculty anonymity and the credibility of the study. Following the second email request over twenty more responses were received.
The survey was voluntary. In an attempt to remedy any possible non-response bias, a follow up survey was sent out to twelve randomly selected SLN faculty representing ninteen non-responding sections (this was accomplished using a numbered list of all faculty names and a random number generator). It was discovered in this process that some faculty were not responding because, though they were listed on the SLN site, they were not using the SLN platform (they were using either WebCT or Blackboard). A follow up email or phone call was sent out to encourage participation regardless of the platform used. Every reasonable attempt was made to contact the twelve faculty (using email and phone messages). However, due to the possibly sensitive nature of the questions (respecting intellectual property rights, disclosure of course information particularly regarding proctoring) and this author's attempts to maintain good relations with his peers by not appearing overly invasive, only six faculty responses were received (six provided answers, two asked not to participate, four did not return emails or phone messages). The six responding faculty represented eleven sections. The response rate for the non-response sample was viewed to be too insufficient as to aid in any claims of generalisability. As a result, this data was rolled into the original survey data, increasing the overall response rate to 39% of all listed SLN math faculty.
Analysis of the data was conducted by dividing the courses into six categories and the assessment instruments into ten categories. The six course categories were developmental (pre-algebra and elementary algebra), general algebra (intermediate algebra, algebra I, II or III, trigonometry and college algebra), calculus (pre-calculus, calculus I, II or III), general liberal arts math, statistics courses and miscellaneous courses (business math, technical math, history of mathematics, and so on). The ten assessment instrument categories were: homework, portfolio, discussion, attendance, project, quizzes, tests, final exam and two 'other' categories. Mid-terms were included in the 'test' category. Examples of assessment instruments in the 'other' categories were activities such as self-assessment, completing a proctor form, sharing solutions with the class, group problem-solving or journaling. Statistical analysis was based solely on sections using particular assessment instruments. Values were either rounded to the nearest whole number or tenth.

General results
Tables Table 1.

SLN listed institutions by campus, type and response
Overview: The response sample indicates that survey responses were received from at least one faculty member representative from just over three-quarters of all SUNY SLN campuses with one faculty representative from almost 85% of all two year SLN colleges and 50% of all four year SLN campuses.

Table 2. Response rates and use of proctoring: Overall and by two or four year institution
Overview: The overall response rate was a somewhat robust figure of 39% of all sections and 39% of all faculty responding. The percentage of faculty utilising some form of proctored assessment instrument was 36% compared to 64% of SLN faculty not using any form of proctored assessment instrument. When controlling for type of institution (two versus four year) half of all the two-year colleges who responded used proctoring and no four-year campus used any form of proctoring.   Table 3 indicates some significant variations in how proctored assessment instruments are being utilised. Developmental math courses generally favour using proctoring. General algebra, statistics and miscellaneous category math courses favour not using any proctoring, with general liberal arts math courses clearly tending towards the use of no proctored instrument. Calculus e-learning courses are almost evenly split in their use of proctoring. According to type of math course, the sample course category distribution closely mirrors the actual population distribution. Table 5 indicates that compared to proctored math e-learning courses, unproctored courses use typical formative assessments such as homework and discussion more often and, when used, they weight results from these instruments significantly higher. Focusing on typical summative instruments such as tests and final exams, the survey data indicates the opposite trend: when compared to proctored math e-learning courses, unproctored courses use typical summative assessments less often and, when used, results from these instruments are given less weight. Quizzes are a cross between formative and summative instrument. When compared to proctored math elearning courses, unproctored courses use quizzes less often but, when used, they weight these results significantly higher. Finally, the survey data indicates that math projects are used dramatically more often (over 300%) in unproctored math e-learning courses and, when used, are given more weight as a portion of the overall course grade. A closer look at the data also reveals that just over half of all unproctored courses using projects use no tests or final exams.  Table 4 Table 5 Summary of comments received Approximately four single-spaced pages of comments were received. What follows is a summary of all the strategies shared for assessment in online math courses (in order of predominance):

Tables 4 and 5. Percent difference on usage and weighting: Unproctored to proctored
The ten most popular strategies for assessing fully asynchronous online math courses 1. Use of Proctoring (nine mentions): Use of a proctored assessment (stated as summative assessments such as midterms and final exams). Many were somewhat dogmatic about the necessity of using proctored assessments. Faculty espousing this position used statements such as "true assessment of a student's ability" (more than one faculty respondent used the words "true assessment").

2.
Use of a writing component (nine mentions): Example: "Gives the instructor a feel for the authenticity of the student". 3.
Use of Projects (six mentions): Several respondents wrote about the usefulness of projects. Sample statements include: "[provides] evidence that they understand the material", "'Helpers' are not into doing final projects from the initial discussion of potential topics, through the review of sources, gathering information, and a (sic) outline before the project is submitted." However, as one faculty member stated, this also appears to be reflected in the assessment data: "I use projects/case studies. This is easy to do with statistics, but much more difficult with, say an algebra class". 4.
Use of web-based math software for homework and testing (six mentions): Many faculty spoke well of web-based assessment such as Pearson's MyMathLab. These systems use algorithms that essentially provide each student in a class with the same set of questions (the question order may be scrambled) with different numerical values and thus different answers. One faculty member mentioned being "pleased" to use this software. In contrast, as one faculty member shared, "helps a bit… does not at all address the possibility of 'coaching' during a test that is not proctored". 5.
Use of a variety of instruments (five mentions): Many shared the importance of "a variety" of assessment instruments 6.
Use of a more "applied learning approach" and "questions that require more thought, innovation, or reflection". As one faculty stated, "I don't think the conventional math questions such as textbook type questions are sufficient" (two mentions). 9.
Use of "progress reports" to keep close track of students' performance (one mention). 10. Use of "self-assessments" (apart from a regular "formal assessment") (one mention).

Comparison of proctored to unproctored math e-learning courses
When comparing unproctored to proctored course assessments, the following is noted: • Unproctored courses tend to use homework more often and, when used, at a significantly greater weighting.

•
Unproctored courses are dramatically more likely to use a project component and, when used, at a higher weighting.

•
While the use of a discussion grade as an assessment instrument is almost identical for both course modalities, the average weighting is significantly higher for the unproctored courses.

•
While quizzes are not used as frequently in unproctored courses, when they are used, they are weighted significantly higher than in proctored courses.

•
There is both a decrease in the use and weighting of tests and a final exam in unproctored courses.

•
For unproctored courses, instruments in the 'other' category (constituting either accountability, diagnostic or formative assessment instruments), tend to be used less overall but when used they are weighted significantly higher than they would be in proctored course sections.
The significance of these results is that they indicate a clear overall trend towards reduced usage and weighting of summative assessment instruments in unproctored sections (see Table 4: for example usage rates in unproctored sections were 68.2% for tests and 50% for final exams, compared to 96% and 100% usage for proctored sections) coupled with an overall increased usage and weighting of formative and accountability assessment instruments (see Table 4: for example, homework was used 34% more and weighted 69% more, discussion was used 10% more and weighted 67% more). Since assessment directs the focus of the student's learning and shows them what faculty value and how we expect them to study (Smith & Wood, 2000) these results seem to indicate that unproctored courses value student effort more and student outcomes less than their proctored counterpart.

Use of projects
Unproctored e-learning courses were four times (317%) as likely to use projects, and weighted them 44% more than proctored courses (see Table 5). As some of the comments reveal, the data appears to support the idea that a number of online instructors are partially or completely substituting projects for traditional summative assessment instruments (tests and exams). The strongest support for this contention is the fact that just over half of all unproctored courses using projects use no tests or final exams.
Overall, of all unproctored courses using projects, the data indicates that homework is used and weighted more (used 22% more and weighted slightly more than twice what one would find in proctored courses) and, while tests and final exams are weighted almost the same as proctored courses, they are used significantly less (used 53% and 73% less respectively). This appears to represent a departure from traditional assessment practices where tests and exams are a critical component of assessment (Bonnice, 1999;Wood & Smith, 1999;Ross, 1999;Engelbrecht & Harding, 2004).
While projects are not new, the question, as previously raised, is whether or to what degree projects can serve as 'a' and/or 'the' primary summative assessment instrument. While it seems apparent that projects are beneficial in serving to help students synthesise course material, it is less apparent, given the ability to consult animate or inanimate resources, how accurate a picture projects can provide of the degree of success an individual student has attained in achieving the course learning outcomes. In contrast, there is little doubt that typical proctored assessments (timed, monitored, regulated resources) would provide a fairly accurate summative assessment.
As suggested by some of the respondents, the thinking that projects can indeed provide such an accurate picture appears to be rooted, at least in part, in the belief that the sheer amount of work required to complete a project acts as a disincentive for students to be able to obtain unauthorised help. As one faculty member commented in relation to projects: "'Helpers' are not into doing final projects". Projects generally require a substantive volume and consistency of work, fit well with the e-learning emphasis on collaborative activities, and may also require text-based inputs (that serve to create an 'intellectual fingerprint' that helps to ensure student validity and academic honesty). However, how effectively can we expect them to serve as 'a' and/ or 'the' major summative assessment? Furthermore, are projects more appropriate in some course categories than they are in others (for example: statistics versus developmental)?
The following faculty statement sums up the current situation well: "there are many conflicting theories on this topic and at this point there seems to be no accepted standard". As many other comments suggest, given the infancy of the e-learning modality, it is clear that many online instructors and course designers are searching for answers. Highlighting this search for answers, while only 36.2% of sections use a proctored assessment instrument, the use of proctoring ranked at the top of suggested assessment strategies. In the math e-learning community, this likely reflects the intensity of the debate regarding the academic integrity of unproctored math courses.

Summary of major findings
Overall, a minority of math e-learning courses are proctored (36%). By institution, no course section originating from a four year institution was proctored. Half of all course sections originating from a two year institution were proctored. By course category the percentage of proctored courses vary greatly with developmental courses using proctoring to the greatest extent (78%) to liberal arts math courses using proctoring the least (8%).
Comparing unproctored with proctored sections, there is a clear trend indicating that offering math e-learning courses in a '100% online' (i.e. unproctored) format leads to a heavier reliance on formative assessment instruments and a decreased reliance on summative assessment instrument. This appears to indicate that unproctored courses place greater value on student effort and decreased value on student outcomes than their proctored counterpart. Projects are more than four times as likely to be used and are weighted significantly higher in unproctored courses compared to proctored courses. This indicates a departure from traditional assessment practices where tests and final exams are the primary summative assessment instrument. The key strategies suggested by faculty for assessing fully asynchronous online math courses is the use of proctoring and the use of a writing component. This contrasts with the current reality that a minority of these courses use proctoring and indicates the intensity of the debate over proctoring in the math e-learning community.
Regarding the shift in use of formative versus summative instruments, the inference is that faculty teaching unproctored course sections tend to value student effort (process) more than student learning (product). Regarding the increased use of projects, the concern in math e-learning, in view of a cheating continuum from the use of a surrogate to receiving coaching, is to what extent can projects act as a reliable and effective summative assessment instrument that can certify a student's mathematical understanding? As half of all unproctored sections in this sample were doing, is it feasible for projects to entirely replace tests and exams and reliably act as the sole summative assessment instrument? Coupled together, these two concerns (use and weighting of formative versus summative assessments, and reliability of projects as a summative instrument) lead one to question the quality of the educational outcomes of unproctored courses in math e-learning. Are these courses, in any way, of less value than proctored courses or, in the worst case scenario, specious? Despite the fact that current faculty may present anecdotal findings that indicate similar grade distributions in unproctored e-learning math courses as in their bricks and mortar counterparts, these comparisons should not be read as reflecting similar learning outcomes given the uncertainty that unproctored grades may, at least in part, be the result of unauthorised coaching or collaboration. Purist online instructional designers and e-learning futurists may resist the issue of proctoring because it avoids the issue that pedagogical innovations must simply be developed to uphold the 'anytime, anywhere' creed of what may be termed 'true e-learning'. While others may argue whether such innovations are necessary, all should agree on the critical importance that all (proctored and unproctored) math e-learning courses, like their bricks and mortar counterpart, be effective in helping students successfully achieve course specific learning outcomes.
That said, the need for a paradigm shift that addresses the peculiarities of e-learning, particularly in mathematics, has been voiced (Smith & Caris, 2003). In focussing on summative assessments, Smith and Caris (2003) write: "changing teaching without due attention to assessment is not sufficient" (p. 125). As we transition from d-learning (distance learning) to e-learning (electronic learning) and to, perhaps, what now has been envisioned, m-learning (mobile learning), it is imperative that attention be paid to how we can effectively assess student learning outcomes (Englebrecht & Harding, 2004). While the practice of assessment must serve to help students learn and effectively synthesise course concepts and material, they are also needed to provide as accurate an idea as possible of how successful they have been in that endeavor.
Whether, one day, advances in biometric safeguards become widely available and trusted or whether e-learning pedagogical insights or developments provide the answer, there is much work to be done. While, for many math faculty, there may be an "innate conservatism" (Smith & Wood, 2000, p. 131) that resists changes to traditional techniques of assessment, the e-learning instructional modality demands we reassess our approach to assessment. Such changes must, however, continue to steadfastly uphold and ensure the integrity of our educational enterprise. As stated in the recent American Mathematical Association of Two-Year Colleges position paper, we must work to ensure that math e-learning standards "meet the content, pedagogical, and assessment standards used in traditional on-campus mathematics courses" (AMATYC, 2002, p. 1). At this time, in math e-learning, it appears only some form of significant proctored summative assessment instrument will ensure that educational standards and integrity are preserved.
Further research should attempt to answer the following questions: • At this stage in math e-learning, are unproctored math sections placing too much emphasis on the process of learning to the detriment of assuring the learning outcomes (product) have successfully been achieved? • What is the product of unproctored courses in math e-learning? • Can projects serve as at least as accurate a summative instrument as traditional tests or final exams?

Study limitations
Given the survey methodology (almost entirely voluntary responses) it is not theoretically possible to generalise these results to all listed SLN math courses. However, given the close similarity of the make-up of the sample to the general population (see Tables 1 and 2) coupled with the high response rate (close to 40% of all faculty and all course sections), these results would raise serious questions as to whether the general SLN math population does indeed mirror these results.

Author biography
In the last 10 years Sven Trenholm has taught full-time at three State University of New York (SUNY) community colleges. He has a long-time interest in computer technology, an academic background in mathematics and education, and a current and developing interest in research and development of computer-mediated math instruction with a specific interest in math e-learning. He is interested in researching issues of academic integrity specific to math e-learning in the rapidly growing fully asynchronous online modality. He has spoken several times on computer-mediated math instruction, and has attempted to be an advocate within the SUNY Learning Network (SLN) to draw more attention to the unique nature of online math instruction.