---

Instructional Software Evaluation Factors

Most of the factors and suggested qualities of software listed below apply to instructional software for computers and other media (laserdisc). Many of these factors will not apply to information manipulation software (i.e. word processors, spreadsheets, databases, or other specialty use programs) typically used at a middle or high school level. Although some of the factors listed below may assist in choice of software for these purposes, specialty factors inherent to the specific use would apply more directly in the selection of such software.

General

• Program is useful in an school setting.
• Program allows completion of lesson in one class period.
• Instruction is integrated with previous student experience.
• Program is likely to save time when compared to other means of presentation.
• An on-disk tutorial for the program's command structure is provided.

Content

• Content is appropriate for intended student population.
• Content is accurate.
• Content is current.
• Content breadth is reasonable.
• The processes and information learned are useful in domains other than the subject area of the program.
• Content is free of any bias or stereotyping.
• Content supports the school curriculum.
• Content is relevant to the subject field.
• Definitions are provided when necessary.
• There is continuity between the information presented and prerequisite skills required.
• There is a need for better than the standard treatment of this topic in the curriculum.

Appropriateness

• Application is well suited to computer use.
• The pedagogic approach is superior to what is available elsewhere.
• Readability level is appropriate for the intended student population.
• Tone of address is appropriate for the intended student population.
• The means of response is appropriate for the intended student population.
• Prerequisite skills required are appropriate for the intended population.
• Time required for typical use by a student does not exceed the attention span of that student.
• Multiple levels of instruction are available.
• Difficulty levels are based on discernible logic.
• Sufficient exposure and practice are provided to master skills.
• Sufficient information is presented for intended learning to occur.

Questioning Techniques

• Questions are appropriate to the content and effectively measure student mastery of the content.
• Questions incorrectly answered can be repeated later in the lesson/exercise.
• The number of trials are reasonable and appropriate.
• Calculation can be accomplished easily on-screen when appropriate.

Approach/Motivation

• Approach is appropriate for the intended student population.
• Format is varied.
• Overall tenor of interaction is helpful.
• Student is an active participant in the learning process.

Evaluator's Field Test Results

• Student understands on-screen presentation and is not confused.
• Student enjoys using the program.
• Student retains a positive attitude about using the program.
• Student retains the desire to use the program again.
• Student retains desire to pursue the topic in other ways.
• Program involved students in competition in a positive way.
• Program fosters cooperation among students.

Creativity

• Program challenges and stimulates creatively.
• Pedagogy is innovative.
• Program allows the student as many decisions as possible.
• Program provides opportunities to answer open-ended questions and provides evaluative criteria to assess responses.
• Program demonstrates a creative way of using knowledge.
• Program challenges the student to alter an underlying model or design an alternative model.

Learner Control

• Learner can alter program sequence and pace.
• Learner can review instructions and previous frames.
• Learner can end activity any time and return to main menu.
• Learner can enter program at different points.
• Learner can stop in the middle of an activity and at a later session begin at that stopping point with the previous record of progress intact.
• Help is available at likely points of need.

Learning Objectives, Goals, and Outcomes

• Learner objectives are stated and purpose is well defined.
• Steps taken to make learning generalizable to other situations.
• For programs requiring use over several days, learning outcomes are worth the time invested.

Feedback

• Feedback is positive.
• Feedback is appropriate to the intended student population.
• Feedback does not threaten or reward incorrect responses.
• Feedback is relevant to student responses.
• Feedback is timely.
• Feedback is informative.
• Feedback is corrective when appropriate.
• Feedback remedies and/or explains when appropriate.
• Feedback employs a variety of responses and avoids being boring.
• Feedback remains on the screen for the appropriate amount of time.
• Branching is used effectively to remediate.
• Program uses branching to adjust automatically difficulty levels or sequence according to student performance.

Simulations

• Simulation model is valid and neither too complex nor too simple for intended student population.
• Variables used in the simulation are the most relevant.
• Assumptions are adequately identified.
• Program simulates activities that can be too difficult to demonstrate in reality.
• The time needed to complete both a step and the entire simulation is reasonable and effective.
• Encourages decision making or calculation rather than guessing.

Teacher Modifiability

• Teacher can easily change or add content.
• Teacher can easily regulate parameters for each class.
• Teacher can easily regulate parameters for each student.
• Parameter setups can be bypassed.

Evaluation and Record Keeping

• Program provides an adequate means of evaluating mastery of the content.
• If tests are included, criteria for success are appropriate for the ability/skills of the intended student population.
• If tests are included, content accurately reflects the material presented.
• Score keeping and performance reports are provided when appropriate.
• Useful information about student performance is stored for future retrieval.
• Useful diagnostic pretest or placement test is provided, where appropriate.
• Useful diagnostic or prescriptive analysis of student performance is available to the teacher, when appropriate.
• Student performance information is easily accessible to the teacher.
• Management system includes adequate security.
• Program allows printout and screen display of student records.
• Program can hold multiple performance records of a single class.
• Program can hold multiple performance records of several classes.

Documentation and Support Material

• Quality of packaging is durable and appropriate for student use.
• Student, parent, or teacher guides and materials are clearly identified.
• Technical and operational explanations for implementation are clear and complete.
• If appropriate, "quick start-up" section is included.
• Useful reproducible student worksheets are provided.
• Other valuable support materials are provided.
• Sample screen-by-screen printouts of the program are provided.
• Teacher support materials can be separated from student materials.
• Useful suggestions are provided for introductory classroom activities.
• Useful suggestions are provided for classroom activities during the use of the program, where necessary or helpful.
• Useful suggestions are given for classroom logistics in a variety of hardware situations and student groupings.
• Useful suggestions are provided on how to integrate program with the regular curriculum.
• If the program is open-ended, subject-specific suggestions are included.
• Clear explanations of the differences between the various difficulty levels are provided.
• Prerequisite skills are clearly stated.
• Accurate and clear descriptions of content topics are provided.
• Accurate and clear descriptions of instructional activities are provided.
• Where appropriate, a description of how material correlates to standard textbook series is provided.
• Necessary information can be found quickly and easily.
• Quick reference card for program use is included, where appropriate.
• Printed text is clear and readable.
• Printed graphics are clear and readable.
• Printed text is free of errors in spelling, grammar, punctuation, and usage.

Technical Quality

• Audio output can be adjusted.
• Audio is clear and used effectively.
• Character sets used in text display are clear, appropriate, and visually interesting.
• Graphics are acceptable on a monochrome monitor, if color is not available.
• Graphics are clear and easily interpreted.
• Program is "crash-proof."
• Program runs without undue delays.
• Program runs consistently under all normal conditions.
• Transitions between screen displays are effective.
• Program guards against multiple key presses advancing the student past the next screen.
• Program avoids unnecessary or inappropriate moving back and forth between screens.
• Special features (i.e. flash, scrolling, split screen) are used effectively.
• Program requires a minimal amount of typing.
• Random generation is used when appropriate.
• Program judges responses accurately and accounts for minor variations in input format.
• Program allows user to correct answer before being accepted by the program.
• Program is capable of accepting partial answers when appropriate.
• Where students must input responses, inappropriate keys are disabled.
• Control keys are used consistently.
• Students require a minimum amount of teacher supervision while using the program.
• Computer operation does not interfere with concentration on activity.
• Program makes effective use of peripheral devices.
• Program considers a previously unexplored potential of the computer.
• Printing is easy and simple to accomplish.
• Procedural and instructional statements are clear.
• On-screen prompts clearly indicate where the user should focus attention.
• Frame formatting is clear, uncluttered, and consistent from screen to screen.
• Presentation of each discrete content topic is logical.
• Sequence of content topics and instruction is logical.
• Sequence of menu items is logical.
• Prompts and cues are clear and consistently applied.
• Hints are clear and not misleading.
• demonstrations and examples are clear and available when appropriate.
• Interface is simple enough to be used with little or no reading of the documentation.
• Program makes clear where the user is in the program.
• User-computer communication is consistent and logical.
• Prompts to save work are given when appropriate.

Start-Up and Implementation

• Graphics and audio are used to motivate.
• Graphics and audio are appropriate for the intended student population.
• Graphics, audio, and color enhance the instructional process.
• Graphics help focus attention to appropriate content without being distracting.

Probeware and Peripherals Included in the Software Package

• Probes or peripherals are durable.
• Probes or peripherals are sensitive.
• Audio and/or graphic quality are effective.
• Probes or peripherals are easy to install.
• Calibration is accurate and easy.
• Data displays are flexible.
• Data analysis is useful.

Hardware and Marketing Issues

• Potential usefulness of the program justifies its price.
• Peripherals that are difficult to acquire or inappropriately expensive are not required.
• Producer field test data are available.
• Field test data indicate that students learned more or better as a result of using the program.
• Preview copies are available.
• Backup copies are available.
• Adequate warranty is provided.
• Telephone support is available.
• If allowable, multiple loading is possible.
• Site license is available.
• Network versions are available.
• Multiple-copies discount is available.

Suggestions partially taken from Computers in the Schools: A Guide to Planning and Administration, by Anthony G. Picciano, New York: Macmillan Publishing, 1994