Assuring Quality in Distance
Learning
A Preliminary Review
A report prepared for the
Council for Higher Education Accreditation
by The Institute for Higher Education Policy,
Washington, DC
Ronald A. Phipps, Senior Associate
Jane V. Wellman, Senior Associate
Jamie P. Merisotis, President
April 1998
Executive Summary
The Council for Higher Education Accreditation (CHEA) commissioned The Institute for Higher Education Policy to investigate the emerging topic of quality assurance in technology-mediated distance learning programs in higher education. Distance learning is growing rapidly, not only as a supplement to traditional institutions and programs, but also as a replacement for those institutions and programs. Policymakers at both the federal and state level are displaying a keen interest in developing a level playing field in public policies affecting distance and campus-based higher education. Further, distance learning is seen by many as a transformative vehicle for increasing the pace of change and reform in higher education. For these and other reasons, analysis of quality assurance is an essential topic for national, state, and institutional policy development.
This study drew upon information from a variety of sources: a review of the literature on distance learning, visits to distance learning sites and providers, and interviews with distance learning experts. This report presents the most up-to-date information available, with the caveat that the context for distance learning is continually changing and evolving.
Our inquiry has been framed by several interrelated phenomena that have accompanied the rapid growth of information technology. These technological and cultural catalysts are pervasive and transcend the various characteristics of providers of postsecondary education. They are:
The emergence of lifelong learning, which is becoming the norm that is augmenting, and in some cases displacing, school-age education. People of all ages participating in learning are the impetus for the creation of new providers well beyond the traditional educational establishment, expanding not only who gets educated, but by whom.
Efforts to make instruction more learner-centered, which is characterized by three fundamental qualities: instruction is largely self-directed; it is more focused and purposeful; and it employs the appropriate level of faculty mediation.
The desire to provide access irrespective of where a student lives, which embraces the notion that learning does not only have to take place on a college or university campus. A number of courses and academic programs already are being provided to students in venues away from the campus.
The development of "knowledge media," which describes the convergence of telecommunications, computing, and the learning or cognitive sciences, and includes the capturing, storing, imparting, sharing, accessing, and creation of knowledge. Knowledge media provide the opportunity to change the emphasis from the classroom and teaching to the individual and learning.
With this backdrop, the study has attempted to answer the following questions:
- What is the universe of technology-mediated distance learning at the postsecondary education level, and how can the array of learning offerings in this country be categorized and described?
- What are the existing or emerging strategies for quality assurance of technology-mediated distance learning programs?
- How can the educational outcomes and experiences offered by these programs and institutions be measured?
- What specific policies and procedures are required for providing quality assurance of technology-mediated distance learning?
For the purposes of this paper, we define "quality assurance" in distance learning as the means by which the institutions or providers set their program goals and measure results against those goals. The process reviews academic content, pedagogic techniques, resources, and support services to see how they combine to enhance the learning environment and ensure student academic achievement.
What is the universe of technology-mediated distance learning at the postsecondary education level, and how can the array of learning offerings in this country be categorized and described?
There are essentially four types of distance learning providers: the military services; corporate universities; unaffiliated distance learning pro-viders; and postsecondary providers.
The Military Services
The U.S. Army is embarking on the most ambitious undertaking of all of the armed services, dedicating $840 million over a 13-year period to provide global access to training through distance learning. The goal is to use distance learning methods to improve efficiency and effectiveness of military training. Using the full range of technology options, the Army intends to design 35 courses during FY 1998 and develop a total of 535 courses by FY 2003, in addition to building 204 facilities and 745 classrooms.
Corporate Universities
A large application of distance learning today is employee training. In 1995, over $50 billion was spent on training by employers. Though estimates suggest that as many as 1,000 corporate universities exist, the extent to which the corporate sector is using distance learning is difficult to ascertain. Most corporate universities share two common goals: to train all of their employeesÑnot just their professional managersÑand to view training as a way to inculcate key stakeholders in the vision, traditions, and culture of the organization.
Unaffiliated Distance Learner Providers
A variety of learning activities are available, primarily through the Internet, that are not associated with any postsecondary institution. The major difference between these unaffiliated learning activities and other kinds of distance learning is that they are not credit-bearing, degree, or credentialing programs.
Postsecondary Providers
Both collegiate higher education institutions, as well as other entities, provide instruction to degree- or credential-seeking students through learning activities that are typically organized in courses for some form of academic credit. According to a Fall 1995 National Center for Education Statistics survey, a significant number of institutions are attempting to address distance learning. Thirty-three percent of higher education institutions offered distance education courses and another 25 percent planned to offer such courses in the next three years. Twenty-three percent of the institutions that offered distance education courses offered degrees that students could complete by taking distance education courses exclusively. An estimated 753,640 students enrolled in distance education courses in academic year 1994-95.
A key to understanding the organizational structures and approaches used by postsecondary providers in distance learning is to distinguish between the different types of postsecondary providers, both in terms of who provides the education and who awards the degree or certificate. When different organizational arrangements are sorted in this way, there appear to be four basic patterns:
Enhancements to Traditional Campus-Based Instruction
The vast majority of institutions offering distance learning are traditional colleges and universities with on-campus students who also offer some courses and entire programs of study at a distance. Technology-assisted instruction is both a pedagogical enhancement to the regular curriculum and a way to facilitate access to students who either cannot or choose not to enroll in traditional classes. More and more faculty are setting up websites to supplement their courses, including chat rooms for student study halls, and e-mail communication between students and faculty. Students who receive instruction via these means are regularly matriculated, enrolled in the usual courses, taught by the same faculty, and stay on campus most of the time they are studying. The instruction can be offered through off-campus centers as well as on-campus.
Consortia or Collaboratives
Consortia or collaboratives are cooperative pooling and sharing arrangements among institutions, where several colleges and universities join together through a statewide or regional network to offer distance learning programs. The authority to award the degree or credits is retained by the institutions and does not shift to the ÒcooperativeÓ or consortium. Examples include the Education Network of Maine and the Southern Regional Electronic Campus.
Contracted or Brokered Arrangements
Contracted or brokered arrangements are configurations of institutions, faculty, or other providers brought together solely for the purpose of delivering distance learning. In contrast to consortia or collaboratives, in contracted arrangements the degree or certificate authority rests with the contracting or organizing entity, not with the originating institution. Examples of major types of contracted arrangements include Governors State University and the National Technological University.
Virtual Universities
These institutions offer most or all of their instruction via technological means. A variety of teaching and learning styles and delivery mechanisms are provided, but what binds them together is their heavy or exclusive use of technology as the educational delivery device. The British Open University, Western Governors University, and International University are good examples of these types of institutions.
What are the existing or emerging strategies for quality assurance of technology-mediated distance learning programs?
Quality assurance strategies appear to be integrated into the design of most postsecondary education distance learning programs. These strategies look quite similar to those in traditional institutions and programs, focusing on the following four key aspects of quality assurance:
Faculty Credentials, Selection, and Training
Quality assurance in many programs and institutions focuses heavily on review of faculty credentials, selection procedures for new faculty, and faculty training. Often, candidates for a faculty position are required to engage in an intensive training program that includes conducting a mini-lesson, interviewing with current faculty, and being assigned a mentor. In addition to possessing a graduate degree relevant to the field of instruction, faculty are expected to understand the role of technology in a learning environment, be trained in online teaching concepts, and use assessment techniques appropriately.
Time-on-Task Measures
Focus on time-on-task measures, including minimum weeks for courses and monitoring of course "log-ins," is common in distance learning programs. In addition to textbooks, many courses are accompanied by a comprehensive study guide which provides course objectives and key concepts. Students are expected to spend a minimum amount of time per week for study and homework assignments. Interaction with faculty is mandated and, in many cases, students share access to a class group mailbox along with the instructor, which provides a platform for instructor communication at virtually any time.
Student Support Services and Consumer Information
Several of the distance learning providers conduct what appear to be rigorous reviews of student support services as one element of quality control. In fact, focus on adequate student support as an essential element of teaching and learning may be one of the most distinctive features of quality control in distance learning environments.
Goals and Outcomes
Quality assurance in distance learning is distinguished by a strong emphasis on program goals, and assessment of results or outcomes in the context of these goals. The focus on assessment of goals and outcomes does not necessarily mean that distance learning programs have been path-breakers in devising new ways to measure instructional outcomes; in fact, with few exceptions we found that distance learning providers measure learning results in much the same way as do conventional programs.
Comparing "Conventional" and Distance Learning Quality Assurance Strategies
Quality assurance strategies for distance learning tend to be oriented toward institutional assessment activities, and to affirming that the core capacities to assure qualityÑsuch as faculty credentials and student support servicesÑare in place.
Thus, the review is of resources and inputs more than educational value-added, making it similar to quality assurance in traditional higher education.
What seems to be most strikingly different is the process for quality review in distance learning programs. In distance learning programs we found a greater tendency for the assessment process to be led by the administration instead of the faculty, with greater use of outside consultants and assessment "experts" in lieu of internally-generated peer reviews. The quality assurance process therefore appears to be less process-driven, where there is a high value placed on consultation, consensus building and dialogue, and more oriented to "bottom-line" or market-oriented results.
In addition to the differences in the process for quality assurance, other pertinent differences that emerge in comparing the two include:
Difference in Mission and Focus
The major stated commitment in distance learning is to the teaching/learning process, and there is a sharper focus on that as an exclusive goal.
Focus on Client
The student is regarded first as a client of the organization, and the educational activities that the client desires predominate in the design and implementation of programs.
Less Control by Faculty over Curriculum
The tendency to develop or use pre-packaged courses and the preponderance of part-time faculty are characteristics of many distance learning programs.
Less Emphasis on Process
The conventional academic culture relies heavily on process, is substantially consultative, and is consensus-driven. The delivery of distance education programs tends to abandon, at least partially, these traditional quality assurance activities, and appears to be more assessment-driven.
Contracting for Services
Many distance learning programs contract with other entities to provide those administrative and student services not directly related to the teaching/learning process.
How can the educational outcomes and experiences offered by these programs and institutions be measured?
The measurement of educational outcomes and experiences in distance learning is elusive. With few exceptions, we found little evidence to indicate that student learning outcomes are emphasized more heavily in the vast majority of distance learning settings than in the more traditional institution environments. However, some of the more prominent providers are attempting to inject more of a student learning outcomes focus into their overall delivery systems. For example, the University of Phoenix has established comprehensive cognitive and affective assessment systems for working adults. Motorola University employs a process where students are assessed one and three months after completion of a course to determine their retention of the crucial knowledge and skills. Perhaps more importantly, managers expect performance to improve as a direct result of the learning activities. The Western Governors University has developed a system where quality control requires the successful completion of an externally administered, criterion-referenced assessment instrument for awarding certificates and degrees. Although WGU will authorize certain providers to supply academic content, the process by which the student acquires the necessary knowledge and skills is secondary.
What specific policies and procedures are required for providing quality assurance of technology-mediated distance learning?
To aid in the development of a policy and research agenda, we conclude with some suggestions for next steps, including a policy agenda for academic accreditation, options for federal policy development, and topics requiring additional research and analysis.
A Policy Agenda for Academic Accreditation
The accreditation model remains a viable and effective means for public quality assurance in distance learning. The research conducted for this paper shows that the core processes of setting and measuring standards can work effectively in distance learning settings. Yet accreditation is challenged by distance learning to adapt standards that are rigorous, to be prepared to re-evaluate traditional processes, to be open to alternatives, and to provide public evidence of measures of performance against the standards. It also must engage in public discussion about fundamental questions regarding the purpose and values of higher education, particularly in degree-granting collegiate programs.
We believe there are central threshold questions which relate to basic collegiate values and purposes that accreditors must confront because of the prevalence of distance learning. These questions concern the role of faculty and the degree of professional autonomy and academic freedom they have within the institution. They also concern minimum expectations for student involvement in an intellectual community, and the prominence of acculturation, personal skills, and values development as central qualities of higher education. To answer these questions requires a fresh look at both the design and control of the curriculum, and the core purposes of collegiate higher education in society.
In addition to posing these threshold questions, we offer the following specific recommendations for steps that need to be undertaken by the accreditation community:
- Establish reliable and valid performance measurements for distance learning;
- Require providers to substantiate evidence of contact between faculty and students;
- Require evidence of effective instructional techniques;
- Promote systematic efforts for selecting and training faculty;
- Assure the availability of learning resources;
- Promote ongoing monitoring and enhancement of the technology infrastructure of institutions;
- Focus attention on the development of courseware and the availability of information; and
- Examine alternatives to the traditional accreditation process.
Options for Federal Policy Development
There are a number of issues in the current federal standards that appear to be particularly problematic for distance learning, including: the definition of "sites," "branch campuses," and "locations;" standards for administrative capacity; the requirement that student credit is recorded either in credit hours or clock hours; the measurement of student financial need; and a host of other concerns.
It is easier to identify the regulatory barriers to distance learning than to know how they should be rewritten in a way that does not invite a new spate of fraud in the aid programs. If enough is not known about how to rework these provisions to accommodate distance learning providers, without inviting new opportunities for fraud, all of the aid programs could become vulnerable. An alternative that might solve the problem would be to create a new definition of a "distance learning" institution in the law, and amend the "experimental sites" provision in the law to extend Title IV eligibility to distance learning institutions that are accredited by a recognized accreditor but fail to meet federal institutional eligibility standards. The experimental sites provision also would allow monitoring and research to be done so that more appropriate standards can be written in the future.
Issues for Further Research and Analysis
Key questions requiring further research and analysis include:
- What are the demographic characteristics of students now being served by distance learning programs, and how does this compare to student characteristics for conventional programs?
- What can we generalize about the matriculation, enrollment, and patterns of learning progression for students in distance learning?
- Who designs the curriculum and course materials in distance learning programs?
- What tuition or fees do students pay for distance learning programs? What percentage of total institutional revenues come from tuition or fee sources?
- Can distance learning be provided at the same or lower cost than conventional education?
- Is there greater capacity to measure student learning outcomes in distance learning programs, and if so, how are learning goals set and measured?
© Copyright 1998
All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher.
Please print this form.
Complete copies of this study may be purchased from the Council for Higher Education Accreditation for $19.95. All orders must be prepaid money order or checks (made payable to the Council for Higher Education Accredtation) and sent to:
CHEA
Publications Department
One Dupont Circle NW, Suite 510
Washington DC, 20036-1135
| Name |
_____________________________________________________________ |
| Title |
_____________________________________________________________ |
| Organization |
_____________________________________________________________ |
| Address |
_____________________________________________________________ |
| City |
| __________________ |
|
State |
________ |
|
Zip Code |
_____________ |
|
| Telephone |
| __________________________ |
|
FAX |
__________________________ |
|
| E-mail |
_____________________________________________________________ |
| No. of copies |
_____________________________________________________________ |
|
