ReadySetPresent Articles can be purchased for $15.00 and are reproducible for a $1.00 royalty fee per copy. Once purchased, download instructions will be sent to you via email. (PC and MAC Compatible). You can purchase all additional royalty copies by clicking here or visiting the reprints page.
| Improving Classroom Instruction With Participant Feedback by Andrew E. Schwartz |
Instructor evaluation by means of participant feedback is neither new nor uncommon. Unfortunately, inordinate emphasis has been placed on this feedback as a source of instructor evaluation rather than as a source of information for improving instruction. Equally disturbing, educators have witnessed little if any significant experimentation or creative application of the feedback. This emphasis on evaluation as an end in itself has left many instructors fearful of feedback and unaware of its potential value and purpose. Hopefully, the following series of questions and the responses that follow them will shed some light on the benefits of participant feedback as a basis for self-improvement.
- What is the difference between evaluation and improvement of instruction? Few
professionals have differentiated between evaluation and improvement of instruction. Although
many of the reasons for this are unclear one fundamental difficulty is that people believe
instructional improvement to be the outcome of the evaluation process — a questionable
conclusion. All too often, evaluation is merely that — the process of gathering participant
opinions to arrive at a value judgment about the instructor. The results of the participant
feedback are then used to determine merit pay, salary increments, promotion, and/or tenure.
A more logical and direct utilization of participant feedback is possible when the primary goal is to make the instructor better at what he/she is expected to do. Viewed in this light, the fear of judgment does not need to be a factor inhibiting the instructor, who can collect information and then draw autonomous conclusions about his/her training performance from it. Equally appealing, the instructor is not dependent on a supervisor or administrator for guidance.
- What are the advantages and disadvantages of participant feedback? As a source of
information, the input participants can supply is broad in scope and therefore represents a
potential accurate sampling of the instructor’s performance. In most other forms of feedback,
one or two people alone are responsible for this information and opinion. Participants are
primary sources. They observe the instructor regularly, whereas other observers view the
instructor infrequently — and visits are usually short in duration. The process of gathering
participant input is also inexpensive and usually can be accomplished within the training
program’s schedule.
However, assuming that instructors can and do acquire valuable information from participant surveys, the instructor faces the potential danger of modifying his/her behavior according to participant belief or opinion, without an objective third party. In a sense, instructors may offer participants the opportunity to create the kind of instructor that they “like best,” not necessarily the one that teaches most effectively.
Whether participants will reveal their true feelings however, is additionally subject to question. Since the instructor holds the power to raise or lower their grade, participants may be reluctant to reveal their honest feelings unless the feedback is given anonymously.
- How do I choose a good method for participant feedback? Selecting a means of gathering
useful feedback can be a difficult task. An instructor can choose one of two routes;
commercially-prepared or self-made formats. Commercially-prepared instruments are usually
created by professionals in the field of evaluation. The Purdue Instructor Evaluation Scale
(PTES) and IDEA — Participant Reaction to Instructor and Course are two examples of
professionally - developed evaluations. The commercially-prepared forms have several
advantages:
- Most of these tools have been field-tested and validated.
- Commercially-prepared instruments usually yield data that lends itself to varied manipulation for analysis purposes.
- Professionally-developed tools tend to be comprehensive in nature.
- Professionally-developed tools usually contain a series of directions explaining how to administer the questionnaire as well as how to analyze data gathered through it.
There are several disadvantages:
- Commercially-prepared instruments can be complicated. The participants cannot always understand the questions being asked and the instructor can find the data difficult to analyze.
- Commercially-prepared participant feedback instruments often lack specificity. To be universally applicable, the questions must be so general that they can lack the substance which might allow an instructor to identify areas needing improvement.
- If several instructors in one school system use the same standardized questionnaire, participants quickly tire of repetitively completing the same form.
Self-made observation instruments are created by the instructor for his/her customized use. While this concept has tremendous potential, self-made feedback questionnaires remain relatively uncommon. They exhibit, however a number of clear advantages.
- The process of creating an individualized feedback tool allows the instructor to feel comfortable with the form and focuses attention in advance on the process of training.
- Self-made participant feedback tools allow the instructor to ask specific questions which he/she feels are important.
- Self-made participant feedback surveys can be very creative. They have the potential to gather data previously believed impossible to acquire.
- The time required to create a self-made participant feedback form fosters a commitment to use the tool on a regular basis.
Instructor-made participant feedback forms also reveal several disadvantages:
- The creation of self-made participant feedback forms is time-consuming and this can become prohibitive to an instructor with a heavy schedule.
- Instructor-made tools are often not field-tested and validated. Therefore, the data’s credibility is open to question.
- The bias of the creator becomes apparent in a self-made participant feedback form. This personal prejudice predetermines the items to be included or omitted in the instrument.
- The data derived from instructor-made participant feedback forms sometimes limit extensive analysis.
Neither the self-prepared nor commercially-prepared instrument need be used to the exclusion of the other. Each tool has merit depending on the needs and goals of the instructor. However, self-made participant feedback tools deserve greater attention and experimentation. Their potential for allowing instructors to engage in personalized, orderly self-improvement activities is substantial.
- What should a self-made questionnaire look like? Assuming that an instructor desires to
experiment, a decision must be made about the nature and set-up of the form’s response items.
If an instructor elects to create a written form, a variety of responses is possible.
While written-response items are most standard, under some circumstances other media can be utilized by the instructor, such as asking participants to draw or color pictures representing their opinion(s). Oral feedback sessions can also be an effective method of gathering information about the instructor’s professional capabilities, although face-to-face feedback is the method most susceptible to “sugar-coating” by participants uncomfortable with their true opinions.
- How specific should response items be? A current survey of both self-made and
commercially-made forms indicates that most participant feedback questionnaires deal in
general terms. The response items are broad, so they can be applicable to any instructor and
almost any course. While general statements can have considerable value, questions with
greater specificity can provide substantially more information to the instructor. This exactness
of the information can allow the instructor to make a decision whether to retain or remove
certain classroom techniques or personal quirks, whereas general forms can only give the
instructors global information on improving their behavior. Another advantage of specific
questions is that the information derived from the question minimizes the possibility of
instructor misinterpretation.
Much experimentation and research needs to be done to create quality response items in participant feedback forms. The instructor’s knowledge and ability to describe his/her own training behaviors and methods directly relates to the quality of feedback elicited from the form. Unfortunately, many instructors have not been trained to use a language that describes such instructional behavior. Additionally, not all participants can understand the terminology used to describe the training/learning act. Hence, the instructor’s ability to describe his/her training and the participant’s ability to understand the language are two critical factors in creating quality participant feedback forms.
- What kind of questions should be asked? No matter who generates the questionnaire,
information solicited from participants generally falls into one of three general areas of focus:
(1) questions about the instructor, (2) questions about the course, and (3) questions about the
participant. Framing questions around these three major areas allows the instructor to gather
comprehensive information which can be compared to ascertain consistency in participant
responses. Additionally, information related to the instructor can be contrasted with the data
about the course. This comparison may allow the instructor to determine how the participants
formed their opinions and what these attitudes were based on. How a participant feels about
himself/herself may also reveal some interesting information about the participant’s attitudes
toward the instructor, and vice versa.
While questions designed to gain information about the instructor are common to evaluation forms, those instruments which require the participant’s self-assessment are rarer. Yet this dimension of the forms has great potential in fostering the participant’s self-examination, and in potentially upgrading the quality of his/her responses.
- Are participants capable of providing accurate and objective information? While this
question is controversial, the fact remains that participants are a storehouse of information
concerning the instructor and course. In terms of professional contact hours, no other people
spend as much time with the instructor.
The greatest obstacle to accurate information lies in the level of the instructor’s ability to solicit information from the participant. Instructors need to create a positive atmosphere where the participant is encouraged to be honest and objective. It is important for the instructor to orient participants to the questionnaire and its purposes.
Among other things, orientation may be required to prevent any negative attitudes developing towards the questionnaire itself. Being told what to do without any explanation of why it’s being done is usually abhorrent to participants at any age level. If it is likely to happen, participants need to be informed that this same instrument may be administered by other instructors in the future. Without this explanation, participants will quite likely become bored or irritated with the process of filling out the same form time after time, and the quality and thoughtfulness of their responses will decrease.
Equally imperative, the participant needs to understand the language being used in the instrument. If the participant does not understand the word(s) being used by the instructor, the participant will be unable to provide accurate data. This is especially true for participants at the elementary level.
An explanation of whether the data is being used to evaluate the instructor and a guarantee that no retaliation will be involved for answers that are less than favorable are also necessary. Recrimination, whether intentional or unintentional, will have a detrimental effect on how future participants will respond to the form. A comprehensive orientation should prevent unnecessary tension which might cause participants to avoid filling out the form because of fear.
- What risks exist for the participant who provides feedback to the instructor? Under
proper conditions, there should not be any risk. To ensure these conditions, the instructor needs
to consider the following questions:
- Has the participant been offered or guaranteed anonymity?
- Will I be able to identify the respondents when the data is analyzed? If so, how will it affect my attitude toward them?
- When/how am I going to use the information? If the data are gathered before the participant’s grade is finalized, will this affect my decision?
If some honest forethought is not given to these considerations, there is no reason to expect equally honest participant responses.
- What variables must be considered when creating feedback questionnaires?
The quality of participant feedback is directly proportional to the quality of the feedback instrument that is created by the instructor. Care must be taken to tailor questionnaires to the group of people likely to be receiving them, even when attempting to elicit the same information form the various groups. A questionnaire for English as a second language students, for example, must be worded differently to elicit quality responses from one given to resume writing students being taught by the same instructor. When creating the feedback form, consider the following items:
- age level of the participants
- complexity of the form
- length of time needed to complete the form
- how often will the form be used
- reading level of the participants
- attention span of the participants
- input that participants had in creating the instrument, and
- date and time that the instrument is administered.
- Do instructors need to prepare themselves for participant feedback? Psychologically,
the instructor must be prepared to accept both negative and positive feedback. There will
always be a portion of the data which will not be complimentary, and the instructor must be
prepared to occasionally hear comments that he/she would rather ignore. In essence, the
instructor must strive for objectivity and openness.
Offensive data (opinions which seem unduly harsh) may not be uncommon when gathering participant feedback. This is especially true if participants have not had an opportunity to provide feedback to the instructor in the past. For some participants this may be the only way that they have of striking back at authority. With time, mutual trust, and frequency of utilization, participants should become accustomed to providing data and can be expected to do so in a reliable fashion.
It is vitally important that the instructor look at total responses (complete data) rather than single responses. It is quite unlikely that all participants will have the same opinion of the instructor. For this reason, single responses may not be indicative of the total group attitude. Instructors have a tendency to focus on the negative responses irrespective of how often the negative comment is made by participants. This natural tendency need to be minimized.
An extremely important aspect of gathering participant feedback is the commitment to gathering data over a long period of time. Surveying one out of four classes for one semester may not be an adequate sample of information. Likewise, gathering data one year and failing to do so for the next two years will not yield maximum information for the purpose of improving instruction. While no set amount of information needs to be gathered, there must be a sufficient amount to document trends in the instructor’s training behavior.
Equally important, the instructor needs to have a strong commitment to the improvements of instruction. All the detailed data gathering is worthless if the instructor reads and ignores it.
- The data is collected… how do I use it wisely? Collecting participant feedback is
relatively easy. Using it to improve instruction is more difficult. Three conditions need to be
present: 1) a commitment to improve instruction and 2) long-term collection of data. The third
condition may sound contradictory: 3) resist making any immediate changes. Improving
instruction needs to be a logical and systematic process. One way of accomplishing this task is
to use the following simple three-step model.
- Step One: Identify
- Step Two: Control
- Step Three: Maintain/Modify
This model insures that the instructor is well acquainted with what he or she is doing before tampering with the status quo. The first step is simple: unless instructors are able to pinpoint specifically what they are doing, it is unlikely that they will be able to help themselves.
The second step is to learn how to control these selected instructional behaviors which have a specific effect on the learners. Awareness of this relationship can allow instructors to better control their interactions with and within their groups.
The third step indicates that the instructor must make a decision about retaining behaviors which are desirable or effective and removing those behaviors which are undesirable or ineffective. It is important to recognize that this is the last step rather than the first step in autonomous self-improvement activities.
- What conclusion can be drawn about participant feedback? By freeing participant feedback from the tethers of narrow interpretation and limited use found in the past, the instructor can move beyond “feedback-as-judgment” to its use as a new, creative tool for self-improvement.
Andrew E. Schwartz, CEO, A.E. Schwartz & Associates of Boston, MA a comprehensive management training and professional development organization offering over 40 skills specific programs and practical solutions to today's business challenges.
Copyright, AE Schwartz & Associates. All rights reserved.
For additional presentation materials and resources: http://www.ReadySetPresent.com.
When ordering, if you need an extraction
program to unzip the file please visit one of these sites below.
These programs all have free trials that can be used to unzip our
files. Make sure you have your pop-up blockers disabled.
Winzip - http://download.winzip.com/winzip120.exe
Winrar - http://www.rarlab.com/rar/wrar380.exe
| Purchase
this article for $15.00. Articles are Reproducible for a $1.00 royalty fee per copy. |
