This page provides a detailed description of the evaluation plans for the Undergraduate Curriculum Development Testbed (UCDT). A shorter summary of UCDT program and evaluation activities may be found here.

 

 

Evaluation Plan for

Development, Deployment, and Evaluation of an
Interdisciplinary Undergraduate Curriculum Development Testbed

A project funded by the National Science Foundation program on
Institution-Wide Reform of Undergraduate Education
in Science, Mathematics, Engineering, and Technology

(Last update: 7/21/97)

Institutional Context

The report 1990, A Michigan Education -- sponsored by the dean of the College of Literature, Sciences, and Arts (LS&A)--conducted a careful and systematic review of the undergraduate program at the University of Michigan. This report found that teaching was undervalued at U of M and other research universities. As a result of the report, the dean of LS&A, the provost, and the president set out to increase the valuation of teaching, both in reality and perception, among the faculty and administrators. Through a combination of financial commitments and initiatives U of M has attempted to establish a new momentum and sense of purpose in undergraduate education.

The University of Michigan has embarked on a program of institution-wide reform in undergraduate education to creatively effect dramatic improvements in the integration of faculty research expertise in undergraduate education, through curricular and extra-curricular means. To accomplish this goal the University has partially funded several curriculum initiatives. The Undergraduate Curriculum Development Testbed is a University (UCDT--see Appendix A for more details) and NSF funded program initiative which was originally focused on the further development of interdisciplinary course sequences in global change (an area in which the University has significant strengths in both research and instruction). The limited initiative evolved into a much larger program designed to encourage systemic change at the University of Michigan. The ultimate goal of the Testbed is to develop, deploy, and evaluate new methodologies for the infusion of interdisciplinary undergraduate curricula into major research universities.

In 1990, the president of U of M, Dr. James J. Duderstadt, initiated the university's Project for the Interdisciplinary Study of Global Change (GCP). The main purpose of GCP was to facilitate cross-cutting interactions among faculty and students. A team of faculty from units scattered across the university immediately started to work on a novel introductory undergraduate sequence called Introduction to Global Change, Part I and II (U110 and U111), a sequence that was taught for the first time in 1992.

UCDT Development Group

The UCDT development group is composed of many individuals and units across the U of M. A team of professors from six schools and eight departments participated in the construct of the NSF proposal. The principal investigator for the project is Timothy Killeen, Director of the Space Physics Research Laboratory. The physical center of the UCDT project is located within the College of Engineer's Atmospheric, Oceanic, and Space Sciences Department. The UCDT development group supports the notion that the primary responsibility of the research university is to imbue the next generation with the benefits of the insights gained during such research.Unfortunately, differential support structures and rewards for "teaching" and "research" functions have, in many ways, impeded progress in undergraduate education. The development group argues that large research institutions, such as the U of M, must strive to return to an appropriate harmony between teaching and research.

The curriculum development group has adapted the "testbed" approach to use the Global Change curriculum as an experimental platform to test curricular innovation. The testbed provided an opportunity to learn and to share knowledge about interdisciplinary teaching, to demonstrate how research and teaching can be integrated, and to contribute to the advancement of scientific and technical literacy with the University general population. The major objectives of the UCDT project are listed below:

1. further develop interdisciplinary courses in global change;

2. develop and evaluate generalizable tools and templates and to establish improved methodologies for the infusion of interdisciplinary courses and materials into the undergraduate programs of major research universities

3. promote more widespread deployment of interdisciplinary courses on and off campus;

4. assess the viability of new teaching credit schemes for the interdisciplinary undergraduate teaching. The UCDT proposal indicated that a change in institutional culture is need to encourage and to sustain the development and deployment of excellent interdisciplinary undergraduate courses. NSF has agreed to fund the project to increase the pace of reform by facilitating the infusion of interdisciplinary innovations more broadly across campus.

Since 1990 the U of M has committed more than $1.4. M to the GCP. The Duderstad administration agreed to support the GCP from 1990 through 1999. This commitment has made it possible for GC faculty to raise substantial funding from external sources to fund additional interdisciplinary initiatives. For example, the U of M was a recipient of a major NSF grant through the Academic Research Infrastructure (ARI) program. This $1,000,000 grant, including 50% costing sharing from U of M, will enable the development of a unique teaching/research laboratory for the experimental study of global change. The new Global Change Laboratory (GCL) will be outfitted with instruments and equipment designed to make systematic measurements of environmental parameters in the entire atmospheric column from the surface to the upper atmosphere--over a ten-year time period.

Evaluation Team Philosophy

Formative Approach

The UCDT implementation plan called for the evaluation team to work closely with the curriculum development team to create a meaningful evaluation plan. The development of the evaluation plan was guided by four major principles. First, a formative prospective dominated evaluation activities and was used to identify and to develop relevant indicators for improving UCDT activities. Second, to ensure that these efforts are maximally useful information, evaluation became an integral part of UCDT activities rather than last-minute add-ons. Third, given the broad scope of UCDT goals and activities, the evaluation team used multiple approaches to understanding UCDT's impact on macro (institutional) and micro (individual) outcomes within a diverse educational community. Finally, every effort was made to use (adapt) existing sources of evaluation data (for example, the student-ratings of instructor effectiveness provided by U of M's Center for Research on Learning and Teaching) to collect data.

The evaluation team encouraged the curriculum development group to help create the evaluation plan. Nevertheless, to ensure an objective evaluation, the evaluation plan was carried out independently by the evaluation team. That is to say, the evaluation team deployed the instruments and analyzed the data.

The stakeholders of the evaluation include NSF, the curriculum development group, and the U of M's central administration. The evaluation team has incorporated what it perceives to be the primary interest of these stakeholders into the evaluation plan.

Participatory Evaluation

The evaluation team adopted a participatory evaluation approach. Cousins & Earl (1992) contend that participatory evaluation is marked by the intention of the evaluators to build evaluation capacity among the stakeholders (in this case, the curriculum developers). Evaluation capacity will help these stakeholders to develop operationalized objectives, evaluation plans, instruments, and data analysis skills. These skills will continue to serve the project by instilling in the stakeholders the values, knowledge, skills, and techniques needed to conduct evaluation even after the evaluators are no longer assigned to the project. Evaluation building activities were carried out informally. Through out the course of the project, the evaluation team made a point to explicate the evaluation activities and progress.

National Science Foundation Agenda

Program Goals

The National Science Foundation (NSF) has funded systemic change projects to stimulate comprehensive, innovative reform, that promote student learning. Projects are funded that have the potential for comprehensive and self-sustaining reform efforts in undergraduate education. An area of particular interest to NSF is the promulgation of interdisciplinary curricula. The NSF strategy is to identify and to remove traditional discipline-based barriers which detract from interdisciplinary curricula.

Project Goals

The evaluation team created an evaluation plan to capture information about the UCDT project. NSF supports the position that the project evaluation should document changes and progress towards NSF and institutional goals. These goals should include achievable objectives that address the needs of major stakeholders such as faculty, students, and administrators. NSF requires quantitative and qualitative information be collected to document that progress.

The UCDT project was designed to deploy an interdisciplinary curriculum as well as collect data about the interdisciplinary teaching environment at the University of Michigan. This project will suggest to faculty and administrators means of overcoming discipline-based barriers to encourage interdisciplinary teaching. The evaluation plan will be used to collect information and analyze the results.

Evaluation Plan for the Global Change Curriculum

The evaluation team encouraged the curriculum development group to articulate student outcomes. The student outcomes listed below were used to help guide evaluation activities to help the team evaluate the degree to which instructional activities achieved student outcomes. Due to the evolving nature of the curriculum, and the urgent need to create instructional activities and book lecturers, specific cognitive and affective outcomes were not delineated. Nevertheless, the outcomes developed by the curriculum development group were used to guide the evaluation plan. The evaluation team focused on measuring expectations, science preparation, experiences, behaviors, and attitudes. The evaluation team collected data about Global Change I and II by using surveys, interviews, observations, focus groups, Early Student Feedback, and classroom assessment techniques (see Appendix B for the Evaluation Work Plan). These constellation of techniques were chosen to ensure the data we collected had multiple reference points. This enabled the evaluation team to include information from a wide variety of stakeholders in their natural settings.

The course objectives for Global Change I included the following:

1. To improve understanding of the interdisciplinary scientific underpinnings for the study of Global Change.

2. To study the evolution of the physical world in order to better appreciate the temporal and spatial scales of change that have occurred in the past as well as those that might occur in the future.

3. To understand why Global Change studies require a system perspective in which many interacting components must be described.

4. To become better equipped to contribute to the important debates concerning global resource management, environment, environmental impact, and societal adaption strategies.

5. To learn how to use the vast resources of the web to find and use environmental information.

6. To learn how to develop simple dynamical models of earth system processes and to understand the importance of physical models of complex systems as well as their limitations.

Surveys

Surveys were delivered via the web to collect data about students' experiences with the lab, lecture, and Global Change web site. The web-based method of delivering the baseline, midterm, and final assessment instruments functioned as an efficient means of collect information about students. Because the data were already in electronic form, it was relatively easy to create ASCII files which could be brought into a word processor, spreadsheet or statistical program. The quantitative data was analyzed using SPSS and posted to the web.

Having the results on the web, allowed the evaluation team to present assessment findings to the curriculum development group during the weekly staff meeting. The curriculum development group used the assessment data and analysis on the web to present the findings to various groups on and off campus. During these presentations, members of the curriculum development group discussed the Global Change Project, UCDT, and the evaluation results using the web.

Baseline Instrument. The Baseline Instrument was designed to collect information about student demographics, the source of information students used to find out about the course, scientific preparation (i.e., science related course that the student have completed to date), expected benefits to be derived from taking the course, and career aspirations. The questions items were open-ended to allow students to respond in their own words.

Midterm Assessment. The midterm assessment instrument was developed to collect information about students' experience with the laboratory, lecture, and Global Change web page. Students were also asked to share their perception of the course instructors. This instrument employed closed-ended and open-ended questions. The questions that appeared on the midterm assessment instruction were approved by the curriculum development group. (A sample of the midterm assessment items and actual results for Global Change II can be found in the Appendix C.)

Final Assessment. The final assessment instrument was designed to parallel the midterm assessment instrument to measure the impact of the instructional experience. Additional questions were added to the final assessment instrument to gauge the effects of team teaching and to determine the level of interest in taking another Global Change course. Students were also asked to share their perceptions of the instructor's effectiveness, lab assignments, and educational technology. (A sample of the final assessment items and actual results for Global Change II can be found in the Appendix C.)

Interviews and Other Information-gathering Activities

The evaluation team conducted interviews with the faculty and staff to understand their experiences with the UCDT program. The interviews with faculty focused on their prior experiences with interdisciplinary teaching, perceptions of the climate for interdisciplinarity at the University, and improvements which would encourage interdisciplinary teaching. The graduate student instructors were interviewed to help the evaluation team to understand the structure and substance of the laboratory assignments. Interviews were also conducted with administrators to understand the interdisciplinary climate at the U of M and the level of institutional commitment to sustain interdisciplinary programs.

Faculty Interviews. An important goal of the UCDT is to help the university community better understand institutional and systemic barriers to a major expansion of interdisciplinary instruction and interdisciplinary curricula. Understanding the characteristics of the academic niche occupied by the Global Change curricula will help us understand the degree to which interdisciplinary programs can be integrated into the current university's academic structure. The quality and scope any academic niche is determined by a variety of factors such as available resources, administrative support, course offerings, and faculty and student participation. To define the academic niche occupied by the Global Change curricula, the evaluation team interviewed the faculty about their interdisciplinary experiences and perceptions.

During the fall 1997 semester three professors who were involved in teaching Global Change I were interviewed using a semi-structured interview format. These professors, all tenured and from different disciplines, were asked about their perceptions of previous and current interdisciplinary experiences; the influence of the institutional, disciplinary, and departmental context on interdisciplinary activity; the transferability of the UCDT experience to other programs at the university; and the resources and/or structural changes needed to support interdisciplinarity, in general, at the University of Michigan.

Observations. A member of the evaluation team attended most of the lectures and some of the labs during a substantial portion of the semester. During these observations the team members recorded notes about the instructional activities and student behavior. From these observations the team constructed instruments to collect data about student experiences. In addition, the team used the observations to present the curriculum development group with ideas for enhancing the opportunities for learning and/or mitigating pedagogical problems.

Focus Groups. The evaluation team conducted a student focus group during the 1997 winter semester to gather the collective experiences of students. Student who were taking the Global Change II course and had completed Global Change I were asked to participate. The evaluation team encouraged students to participate by offering pizza and soft drinks. This format allowed students to partake in discussions about the lab, lecture, and the web experience.

Early Student Feedback. During the 1996 fall semester the evaluation team asked the Center for Research on Learning and Teaching to conduct the Early Student Feedback. The Early Student Feedback assessment instrument was designed to help faculty get student feedback about their teaching early in the term. The technique allowed the use of small group discussions to gather information directly from students in a format that permits interaction and clarification. Students were asked to list the major strengths of the course and the changes which would improve the quality of the course.

The Early Student Feedback instrument was administered on Wednesday, October 16, 1996 (approximately six weeks into the term). In small groups, students were asked to reach consensus on the following questions and directions:

1. List the major strengths of the course. What is helping you learn?

2. List changes that could be made in the course to assist you in learning. Please explain how suggested changes could be made

The data collected from this instrument were compiled, summarized, and presented to the curriculum development team.

Classroom Assessment Techniques. The evaluation team introduced classroom assessment to the faculty and Graduate Student Instructors. These techniques, pioneered by Angelo (1993), are designed to (a) assess course-related knowledge and skills, (b) assess learner attitudes, values, and self-awareness, and (c) assess learner reactions to instruction. Three techniques were considered due to their low time and energy requirements: the One-Minute Paper, Muddiest Point, and One-Sentence Summary . Only the One-Minute Paper was used consistently throughout this course to assess the results of the lab.

To use the One-Minute Paper, an instructor stops class two or three minutes early and asks students to respond briefly to some variation on the following two questions: "What was the most important thing you learned during this class? and "What important question remains unanswered?"

The data collected from this instrument were presented during the weekly curriculum development meetings and posted to the web were students could read the material. On several occasions data from this instrument was used to adjust the pedagogy.

Evaluation of the Electronic Learning Environment

The Global Change curricula was designed to employ "state-of-the-art instructional tools" to help improve the quality of teaching and learning. The UCDT proposal discussed the integration of the Internet (i.e., WWW, e-mail), interactive modeling (i.e., STELLA program), video, CD ROM and multimedia tools to construct "computational and experimental modules that apply the principles of inquiry-based and open-ended learning to global change and earth system science studies." The implementation of these computer-based tools form the basis of the electronic learning environment (ELE).

The role the computer application plays as a learning tool is shaped by the characteristics of the hardware, software, students, instructor, and the learning task. The context, product, and process are the elements of the ELE. By examining the ELE it may be possible to evaluate the contribute it makes to learning.

World Wide Web

The World Wide Web contains an abundance of scientific information and analysis which can be used to used to study almost any subject. Scientific data and images collected on Earth and from remote sensing equipment are often available to the scientific community shortly after the data is collected. Government agencies have made their vast archives of data more accessible to stimulate scientific research and educational opportunities. Information about environmental, the economy, the distribution of flora and fauna can be used by both students and scientist to learn more about global change concepts and principle.

In order to utilize these vast resources students are required by the Global Change curricula to learn, use, and appreciate the web. Dr. Tim Killeen, the principle investigator for UCDT, decided that the World Wide Web would play a major role in the global change curricula to take advantage of its educational potential. Tim envisages that the Global Change web site (URL: www.sprl.umich.edu/GCL) will evolve into an electronic textbook. This electronic textbook will take advantage of hypertext, links, animation, and graphics. As the electronic web site evolves, the evaluation team will create instruments to document its capabilities and learning effects.

Other ELEs

The Global Change curricula incorporates the programs STELLA and ArcView 3.0 GIS into the pedagogy. STELLA is a software package designed to help students graphically build and control dynamic models. The STELLA program interface lets the user set up model elements (stocks and flows), to specify the relations between the elements, and then project how these elements will react over time. The program serves as a useful and flexible introduction to how computers may be used to model real-world problems and situations. STELLA is an important tool for understanding global change, modeling is the only way to predict the impact of global change. Current, all predictions which scientists use for estimating the impact of environmental change on the Earth's future are based on dynamic models, like STELLA. ArcView is also a powerful program used in the real world.

ArcView 3.0 GIS (see my review URL: www-personal.umich.edu/~jmfenty) is a computer mapping system designed by Environmental Systems Research Institute, Inc. This geographic information system is designed to help the user to analyze data in a spatial context. GIS technology integrates common database operations such as query and statistical analysis with unique visualization and geographic analysis. ArcView is most often used as a tool by GIS specialist to analyze street networks (traffic planning and maintenance), natural resources (natural resource management, habitat assessment), land parceling (zoning), and facilities management (utility planning and maintenance). ArcView's powerful visual and analytical capabilities have also been used to as a pedagogical tool.

Evaluation of ELE

Gagne (1992) posits a model of instructional design that may be used to evaluate ELEs. The outline below will describe the salient components of the model.

Instruction consist of a set of events external to the learner designed to support the internal processes of learning. The kind of events that constitute instruction should have a fairly precise relation to what is going on with the learner whenever learning is taking place. The overall purpose of instruction is to precipitate learning outcomes such as intellectual skills, verbal information, cognitive strategies, motor skills, and attitude.

Gagne's Outcomes of Instruction
Outcomes of Instruction Definition Examples of Performance
1. Intellectual skills ability to apply a rule learning how to identity a sonnet by its rhyme pattern
2. Cognitive strategy capabilities that govern the individual's own learning using a model to solve a novel problem
3. Verbal information declarative knowledge-the information we are able to recall stating the provisions of the fourth Amendment to the Constitution
4. Motor skills a skill that can be physically demonstrated planning the edge of a board; printing the letter E; keyboarding
5. Attitude establishing a specific attitude toward a thing. A persistent state that modifies the individual's choices of action choosing to read literature; choosing running as a regular form of exercise

Instructional events are specific external processes designed to support the internal process of learning

Gagne's Instructional Events
Instructional Events Relation to Learning Process
1. Gaining attention The goal is to commanding the learner's attention. The student can be prepared for learning through moving displays, music, or the teacher banging on the desk.
2. Informing the learner of the objective The leaner should be informed about the kind of performance that will be required to evidence learning outcomes that have been achieved.
3. Stimulating recall of prerequisite learning Learning is depend on previous knowledge and perceptions. The prompter should utilize some previous capabilities the learner has acquired through past experiences.
4. Presenting the stimulus material The prompter should present the learning materials such that the critical elements in the lesson are identified and highlighted. The event should be coordinated to emphasize feature for selective perception.
5. Providing the learning guidance Learning may be scaffolded through a series of communications in the form of hints or questions. The amount of hinting and prompting involved in the learning guidance event will vary with the kind of learner. Too much guidance may seem condescending to the quick learner, whereas too little can simply lead to frustration on the part of the slow learner.
6. Eliciting the performance provide the learner with the opportunity to demonstrate that the actual internal combining event of learning has taken place.
7. Providing feedback about performance correctness provide feedback that rewards good performance and encourages hard work
8. Assessing the performance assess the learning outcomes
9. Enhancing retention and transfer when information or knowledge is to be recalled, the existence of the meaningful context in which the material has been learned appears to offer the best assurance that the information can be reinstated

The instrument below was designed to help the evaluation team assess ELEs. The individual items were constructed using suggestion from Gagne's model and the team's ideas.

Dimension for Critiquing Software

Application Type (i.e., Tutorial, Simulation, Tools--Appendix D):

(Note: different scales may be used to judge the value of the dimensions under consideration. For example, it may be appropriate to use excellent =1; good=2; fair=3; poor=4; not applicable = 5.)
dimensions scale comments
1. Congruency of software goals/design with the course goals
2. Congruency of software goals/design with the needs of the intended learner
3. Congruency of software with the relevant instructional paradigm or theory (e.g., Gagne's Instructional Events Model)
4. Pace of instructional events
5. Learner control over the instructional events (e.g., level of difficulty; sequence)
6. Use of factors that may positively affect intrinsic motivation (e.g., challenge, fantasy, curiosity, fun, warmth)
7. Use of factors that may positively affect extrinsic motivation (e.g., cooperation, competition, recognition, reward)
8. Feedback about achievement (e.g., rewards, information)
9. Screen output (readability, clarity of screen layout, aesthetic of composition, appropriate use of graphics)
10. Audio output (e.g., incidental sounds, direct use of audio for instruction)
11. Animation used to contribute towards the achievement of learning outcomes
12. User interface (e.g., crash proof, simple instructions at first, suppression of extraneous items, continuous display of critical instructions, error handling, use of menus, use of input devices appropriate to varied needs/preferences or users)
13. Utilization of hardware (e.g., I/O, RAM, CPU, etc.)
14. Support materials (e.g., print, parsimonious instructions, trouble shooting guide)
15. Overall assessment of quality

Evaluation Plan for Systemic Reform Agenda

The University of Michigan is not unlike other research universities when it comes to interdisciplinarity. The University of Michigan has strong academic departments that exert considerable influence over faculty scholarship. The majority of the faculty confine themselves to disciplinary activities. As faculty mature in professional rank, some faculty experiment with inter-departmental research and teaching.

At the U of M their are many examples of faculty who have formed inter-departmental teaching and research teams. The evidence seems to indicate that this kind of grassroots organizing is the dominant form of extra-disciplinary activity. Interdisciplinarity appears in more structured formats within the research centers and institutes which ring the campus.

The Institute for Social Research and the Center for the Study of Women are examples of University of Michigan sponsored units which have faculty who conduct interdisciplinary research. These units are like magnets for faculty who want to do interdisciplinary research.

To accurately evaluate the current and future interdisciplinary context at the U of M would require a substantial protocol of surveys, interviews, archival analysis employed over many years. The National Science Foundation and central administration at the U of M has funded UCDT, in the current cycle, for two years. In those two years, UCDT is expected to create a revamped Global Change curriculum, and create tools and templates to change the institutional culture. Obviously, the likelihood of doing and measuring the latter goal is small. The U of M is like an ocean-liner, it can not turn on a dime. It may take many miles of ocean and much time before the effects of UCDT are recognizable.

Given this back ground, the evaluation team must make informed decisions about the UCDT products which have the potential to create systemic change. The evaluation team will evaluate the products' utility and document the perceptions of UCDT personnel regarding the systemic nature of the curricular products.

The evaluation team created a protocol to collect information about the climate for interdisciplinary teaching at the University of Michigan. Through purposive sampling we identified several individuals with knowledge and experience in this area. Three key faculty members of the UCDT curriculum development team and several deans and administrators were interviewed. The professors, all tenured and from different disciplines, were asked about their perceptions of previous and current interdisciplinary experiences; the influence of the institutional, disciplinary, and departmental context on interdisciplinary activity; the transferability of the UCDT experience to other programs at the university; and the resources and/or structural changes needed to support interdisciplinarity, in general, at the University of Michigan. In addition, the evaluation team reviewed university documents that addressed the issue of interdisciplinary teaching and/or research. This background information was reviewed to form general impressions of context of interdisciplinary the U of M..

Topology of Interdisciplinary. Latucca (1996), in her dissertation Envisioning Interdisciplinarity, investigated three aspects of interdisciplinary research and teaching: contexts, processes, and outcomes . She developed a topology that offers valuable distinctions among interdisciplinary programs. Included in the topology are informed disciplinary; synthetic interdisciplinarity; transdisciplinarity; conceptual interdisciplinarity (see Figure1). As evaluators, it is important that we consider the complexity of establishing the worth and merit of interdisciplinary programs. Figure 2 contains a number of questions that the evaluator should consider as they develop their plans, methods, and reports.

Figure 1. Topology of Interdisciplinarity (Lattuca, 1996; Hurst, 1992)
1. Multidisciplinarity (disciplinary courses that are informed by other disciplines) The disciplinary contributions may be mutual and cumulative, but they are not considered integrated. Communication among disciplines is considered to be minimal--thus it tends to be transitory and limited.
2. Informed Disciplinarity (disciplinary courses that are informed by other disciplines) Teaching issues and research questions of informed disciplinarity are essential disciplinary in nature, that is, they are motivated by a disciplinary question. When faculty use informed disciplinary teaching they make use of examples from other disciplines to help students make connections between disciplines. Nevertheless, these connections do not change the focus of the class from one discipline to another. This method borrows methods, theories, concepts, or other disciplinary components.
3. Synthetic Interdisciplinarity (courses that link disciplines) Example: A course that examines historical and legal perspectives on public education. Only when the borrowing described above is motivated by an interdisciplinary question or issue does scholarship qualify as interdisciplinary. Synthetic interdisciplinary occurs when teaching issues and research questions bridge disciplines. These bridging issues and questions are of two subtypes: (1) issues or questions that are found in the intersections of disciplines, and (2) issues and questions that are found in the gaps between disciplines. In the former case the issues questions belongs to both disciplines; in the latter, it belongs to neither.

4. Transdisciplinarity (focuses is on developing an overarching synthesis)

Example: Sociobiology--applies the principles of natural selection and evolutionary biology to the study of animal behavior.

Application of theories, concepts, or methods across disciplines with the intend of developing an overarching synthesis. The theories, concepts, or methods transcend disciplines and applicable to many fields. The disciplines do not contribute components, but rather provide settings in which to test the transdisciplinary concept, theory, or method. The disciplines become subordinate to the larger framework.

5. Conceptual Interdisciplinarity

(examines issues without a compelling disciplinary basis)

Example: Cultural studies, feminist theory, postmodernist, or critical theory.

Typically begins with a critique focusing on the limitations of disciplinary understandings of the issue or question. The interdisciplinarian then proceeds to create their own framework to answer the question or address the issue.

Figure 2. Issues to Consider When Evaluating Interdisciplinary Courses
1. What type of interdisciplinarity does the curriculum represent?

- understand the scope and intensity of conceptual interaction among the disciplines

- use the Latucca topology



2. In what interdisciplinary context did the curriculum emerged?

- articulate what brought the faculty together to participate in the project

- identify institutional factors affect the climate for interdisciplinarity

- identify the disciplinary antecedents

3. What process have faculty used to articulate the interdisciplinary curriculum? To what degree does the curriculum represent the shared vision of the collaborating faculty? To what degree are individual faculty committed to the curriculum? Which faculty will be around long enough to use evaluation data and analysis?

- identify what questions the curriculum is designed to address

- describe how faculty defined the course objectives, readings, and assignments

- describe conceptual threads that bind the curriculum together

- identify faculty and administrators that have a multi-year commitment

4. To what degree are the students prepared for innovative pedagogy? Are they prepared for the new learning experience? - describe the degree to students are prepared for the curricular experiences which have or have not prepared them for an interdisciplinary experience
5. To what degree does the curriculum exhibit structural integrity? To what extent do students experience "learning jet lag" due to abrupt shifts in pedagogy and expectations among faculty?

- describe the transitions involving instructors and topics

- describe the degree to which the faculty share pedagogical practices

- do students and/or faculty understand and use their curricular road maps

6. What are the student outcomes? Have the faculty collective articulated measurable student performance objectives?

-outline the student outcomes and their operationalized definitions.

- articulate cognitive and affective outcomes.

7. What is the quality of the reading materials (i.e., textbooks, course packs, handouts)? If reading materials have been developed for this curriculum to what degree do they facilitate learning? - develop a rubric to determine the utility of the reading materials
8. What examinations are available and viable? Will new examinations need to be created? -describe the validity and reliability of these instruments
9. How will the information be reported to the stakeholders? - identify how the stakeholders process information
10. What are the long-term prospects for continuation of the project? - determine the likelihood that the project will continue, and how long the current personnel intend to be associated with the project

 

A review of the literature about interdisciplinary research and teaching indicated that there are both impediments and catalyst. The impediments take the form of institutional, disciplinary, and individual factors which collectively function as potent disincentives. Early career faculty tend to be especially venerable to these influences. On the other hand, there are catalyst that enhance the likelihood that interdisciplinary programs will germinate and flourish. Below are ideas about interdisciplinary teaching gleamed from the literature (Latucca, 1996 & Hurst, 1995)

Interdisciplinary Teaching Impediments

Control of disciplinary communities. Disciplinary cultures exert powerful social and psychological pressures which shape the epistemology and ontology of faculty. The disciplinary community--through its disciplinary organizations, conferences, and journals--control the levers of professional reputation and career advancement. Academic cultures, by in large, determine the questions that will be investigated and the methods which will employed. For many faculty the disciplinary culture provides the conceptual framework from which reality is interpreted. Frameworks that emerge from different circumstances are typically dismissed.

Academic disciplines are often associated with specific jargon and research methodologies. These symbolic and epistemological hurdles make it difficult for faculty to nurture important interdisciplinary relationships. Interdisciplinarians must invest time and effort to build relationships and research programs. These start up cost tend to be much higher when compared to cost associated with disciplinary research.

Venues for Publication and Conference Presentations. Scholarly journals are often the means of evaluating the worth of a scholar. Journals that have disciplinary focus bestow the greatest prestige on the professors who publish under their auspices. Venues for publishing interdisciplinary research are fewer and are sometimes seen as purveyors of less than the best science In academia, reputations are often a function of publications in disciplinary journals

Evaluation. Traditional evaluation methods of examining research, teaching, and service focus on individual accomplishments in these areas. Evaluation of research and teaching are more problematic when a faculty member engages in interdisciplinary work. It is difficult to create evaluation criteria which unpack the individual faculty members work when it is packaged in a collective product. If interdisciplinary activity is to become the norm new models of personnel evaluation must be deployed.

Department Control. In a research university the academic departments are the primary agents for executing the institutional mission. Academic departments are often autonomous, loosely coupled units which are designed to minimize "communication cost" and to subsidize the "academic norms of professionalism (Hurst, 1992, p.13)." Academic departments and their allied academic communities exert tremendous pressure on faculty to conform to disciplinary norms.

Academic departments typically provide the basis for reward and resource. Funds are disbursed to academic departments to achieve excellence in specific spheres of intellectual influence. The boundaries between academic departments are well understood and respected by faculty and administrators. Faculty who venture outside their academic departments may be seen as siphoning off important resources in pursue of endeavors which have minimal impact on the prestige of the department.

Institutional Resource Allocation. At large research universities, academic units tend to be loosely coupled to each other financial and intellectually. Financial management systems, such as Responsibility Center Budgeting (RCB), encourage academic units to engage in revenue maximizing behavior to optimize student FTE and research revenues. Each unit is expected to be as self-supporting as possible. The notion of "each tub on its own bottom" is the prevailing style of central management.

Binkman and Morgan (1997) say this,

RCB is a system where by all of part of the tuition reflected in enrollments, along with sponsored research and other revenues, is returned to units in proportion to what the unit "earns." In theory, the units controls its own pricing policy and is responsible for all, or nearly all its cost (Berg, 1985). Surpluses and deficits are the responsibility of the unit. Additionally, in most RCB systems, the central administration of an institution oversees a taxing or subvention system where some activities such as libraries, judged to be valued but outside of proper market forces are subsided. Other central services, such as computing or printing, often operate on a charge-back system and must compete with outside vendors (p. 295).

This financial system discourages cross unit collaboration at the grassroots level. Individual faculty in different academic units wishing to collaborate must be conscious of the financial impact of their decisions on their home unit. Sharing student and faculty FTE between units may mean trimming revenues which might otherwise contribute to the bottom line for the home unit. To counteract the financial disincentives to participate in interdisciplinarity, funds acquired through the subvention system may be distributed to faculty initiating interdisciplinary programs.

Central administration, in order to sustain some level of interdisciplinarity, may use the subvention system to provide important seed money to help fledging programs. Nevertheless, the interdisciplinarians have a short window of opportunity to become rooted into the academic infrastructure. New interdisciplinary programs must acquire a stable financial and human capital. Because seed money will soon be exhausted, the interdisciplinarians must financially attach themselves to a department or external funder as well as acquire new professors who are willing and able to participate.

RCB, a system designed to maintain the "substantial independence of subsidiary budgetary units (Brinkman & Morgan, 1997, p. 295, " creates powerful disincentives for interdisciplinary activity. RCB system assigns decision-making about efficiency and effectiveness of programs at the program level. At this level it may be difficulty for deans and program coordinators to understand and to appreciate the contribution an interdisciplinary program makes to the institution as a whole. Even traditional courses, such as those that fall under general education, have been cannibalized by professional schools seeking to maximize financial returns. Some professional schools administer general education programs for their students. Thus, they endrun the traditional offerings of liberal arts colleges within their own institution. Given this competitive environment, it may be difficult for interdisciplinary programs to emerge.

Interdisciplinary Teaching Catalysts

1. Size: small group of disciplinary/interdisciplinary professors (3-7 members) who actively participate in the development of the curriculum

2. Deductively defined conceptual framework that emerges from a perceived gap in curricular offerings

3. Highly interactive team that meets regularly or a strong leader who can offer definitive answers

4. Excellent rapport among team members which emerges out of a strong affinity to a conceptual issue

5. Resources have been allocated toward team building, a communication infrastructure, and evaluation

6. A well defined user group conceptualized by the faculty

7. Adequate recognition and rewards for the curriculum development team from administrators

8. Multi-year commitments among tenured faculty

9. Interdisciplinary research centers (e.g., the Institute for Social Research) and other pre-established interdisciplinary communities (e.g., School of Natural Resources and the Environment)

References

Angelo, T. A. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass.

Brinkman, P. T. & Morgan, A. W. (1997). Planning and management for a changing environment (pp. 288-306). In M. W. Peterson (Ed.) Improving academic management. San Francisco: Jossey-Bass.

Cousins, J. B. & Earl, L. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14(4), 397-418.

Gagne, R. M., Briggs, L. J. & Wager, W. W. (1992). Principles of instructional design (4th. Edition). Forth Worth, Texas: Harcourt Brace Jovanovich College Publishers.

Hurst, P. J. (1992). The research university as an organizational context. Unpublished dissertation. Ann Arbor: University of Michigan.

Lattuca, L. (1996). Envisioning interdisciplinarity. Unpublished dissertation. Ann Arbor: University of Michigan.

 

Appendices

Appendix A - UCDT Project Components

Appendix B - Evaluation Work Plan

Appendix C - Comparison of Midterm and Final Assessment Results

Appendix D -Qualities of Good Software by Dr. Jerome Johnston

 

Last content update: 21 July 1997
Last html update: 23 November 1997