Return to Training Resource Center
EASTERN REGION TRAINING AND EVALUATION MODULE NO. 1
DECEMBER, 1996
THE EASTERN REGION TRAINING AND EVALUATION MODULE:
AN INTRODUCTION
Rosemary Auld Miller
Scientific Services Division
Eastern Region
NOAA/National Weather Service
Bohemia, New York
_______________________________________________________________________________________
This is the introduction of a new regional publication: a brief, concise TRAINING MODULE, designed for completion
within one hour. This module describes the motivation for its creation and its instructional
design, and includes suggstions of module topics for prospective authors.
_____________________________________________________________________________________
1. Objectives
Upon successful completion of this module, the trainee will meet the following objectives:
a. Given a list of the types of Eastern Region publications, describe at least three of the four motivating reasons for creating the Eastern Region Training and Evaluation Module (TEM) as a new regional publication.
b. Without reference to this TEM, list, and explain the importance of, at least seven of the eight major sections of a TEM.
c. Based on your observations and discussion with others, list two possible topics around which a TEM could be designed, that would benefit the reader's office training program (according to the Science and Operations Officer, or SOO/Development and Operations Hydrologist, or DOH).
d. Using this TEM as a reference, create an outline of a TEM to address a given training need at the office, from which (according to the SOO/DOH) a draft of a full TEM could be authored and submitted for publication. The outline must contain all TEM sections.
2. Introduction
Training, first and foremost, is a means to produce a change in performance (Mitchell 1993). As the National
Weather Service (NWS) continues to modernize, new technologies, hydrometeorological research, and forecast techniques
continue to be introduced and updated. The critical need for specific training (as well as ongoing professional
development in a broader sense) is a direct consequence of this unprecedented transition, and has assumed a high
priority in the agency (Spayd 1996).
There are many ways NWS personnel can obtain training. On-station training has recently become particularly important,
as changes in staffing, budgets, technology, data acquisition and interpretation, forecast techniques, and daily
operational responsibilities, all translate into a fundamental requirement for the most efficient use of
time. Access to brief, concise training materials thus constitutes one of the greatest needs of on-station training
programs.
Reading, or even writing, a paper for publication is an effective means of learning about new forecast techniques
and technologies, climatology, numerical guidance, etc. In the Eastern Region, these publications range from the
nontechnical and limited-in-scope (General Interest Paper); to the more detailed technical treatment (Technical
Attachment, WSR-88D Operations Note); to the in-depth analysis, which may include a significant amount of data,
and have wider distribution outside of the region (Technical Memorandum). The Training and Applications
Note is a little different in that it is a medium for the presentation of training techniques, topics, and
applications, e.g. "ADAP Manual" (Czepyha 1993), "A Forecaster's Guide to MicroART" (Wooldridge
1991), and "A Met Intern handbook" (Kapela and Eddy 1992).
Each of these publications offers the reader operational insights, experiences, and other information in a broad
educational sense. The TEM will complement these publications in that it will be purely a training medium: the
objective of the module is to improve actual performance or knowledge, and will include a means of practice as
well as an evaluation of the reader's progress (usually questions or exercises within the body of the TEM and at
its completion).
One motivation for creating a TEM? It is hoped that, as a formally reviewed Eastern Region publication,
the TEM will serve as an alternative to designing and undertaking a local research project to motivated potential
authors; rather, enthusiasm and keen interest in hydrometeorological topics could be expressed in the creation
of quality, concise training materials such as a TEM (and there is no better way to learn a subject than by teaching
it!). One motivation for completing a TEM? It is concise training designed to improve performance and understanding,
requiring less than one hour for completion. It is training covering a very narrow topic, with clear
objectives and relevance to operations, and can be traditional or computer-based in presentation.
The remainder of this paper will discuss the format and design of the TEM, and the procedure for submitting it
to Eastern Region Headquarters for publication. Questions will be included, as a standard component of the TEM.
3. Development of a TEM
a. Defining a need for training
Since each module is designed to improve performance or skill in a specific way, the first step in creating a TEM
is to determine what forecasting or other operationally oriented activities actually need improvement. For
example, could you (or your station's staff) perform better as forecasters if you (they) could:
TEM topics need not be limited to those of interest to meteorologists and hydrologists, but could include
those of interest to hydrometeorological technicians and other staff. Other topics might include, for example,
aspects of hydrology; the use of Q-Vectors; recognizing the potential for conditional symmetric instability (CSI);
techniques in mesoscale analysis and forecasting; numerical model guidance and biases; and frontogenesis. Note
that the TEM would be ideal for taking the modular, or "partitioned," approach to some of the more involved
training topics.
b. Format and length
Now that you've decided what operational skills or knowledge could be improved with training that could be provided
in a brief, modular format, you need to choose the platform that would provide the most effective training. Could
the training be successful in a standard paper form, or would it require an interactive quality or display capability
that a computer could provide? Good computer-based training requires time and effort, but
can be very effective. However, paper-based training modules can also be quite good; the quality of learning is
not solely a function of the medium, but is in large part a matter of the "readability" and composition
of the material (which is why there exist so many mediocre training materials out there). The format to be used
for a TEM is what you see in this document, and is not restricted to paper presentation. Its basic components,
outlined below, must also be included in computer-based TEMs.
One particularly important aspect of "readability" is the length of what's being read. Everyone knows
how motivation to learn (or even start training) is affected by how "imposing" the training material
appears. It's hard to be enthusiastic about sitting down to learn something useful if you know it's going to take
12 hours to complete--over a period of six sessions, spaced weeks apart. This is the reason the TEM is designed
to take less than one hour to complete.
c. The components of the TEM
Header
The one or two sentences that comprise the Header serve as the very first motivator for the potential reader. It
briefly describes the essence of the TEM, thus allowing him or her to decide quickly whether or not to continue.
Objectives
Simply put, the objectives of a training module clearly indicate what the reader should be able to do upon completion
of the module, and the accomplishment of those objectives should be measurable or observable under clearly
stated conditions (Mager 1988). For example, as recommended by Mager (1988), this TEM's Objective a describes
a desired performance ("describe the...motivating reasons for creating the Eastern Region TEM..."),
the conditions under which the action is performed ("Given a list of the types of Eastern Region publications..."),
and criterion for its evaluation ("describe at least three of the four motivating reasons...").
All of these elements of an objective are vital, in that they give direction and focus to both the reader and the
designer of the training, and the prospective reader can quickly determine whether he or she should invest time
in this training.
Note that the performance described in each of this module's objectives state an action that the reader
will be able to perform upon completion. It is clear what this module will enable the reader to do, more so than
if more abstract or vague generalities had been used. For example, consider this objective: "The reader will
understand, or know, how to create an Eastern Region TEM." What's actually meant by understand
and know could certainly be open to interpretation. Contrast that with this TEM's Objective d: "Create
an outline of a TEM..." Mitchell (1993) recommends a list of "evaluative" verbs for writing
training objectives that includes, but is not limited to: compare, create, defend, state, demonstrate, describe,
discriminate, explain, identify, list, make, perform, present, program, show, solve, use, and utilize.
Introduction
The introduction of any presentation provides the motivation for the reader to continue, answering
for him or her the most basic question: How do I benefit from this; why should I care? Thus, the introduction
should not only provide a background and context for the training, but also state clearly why it's important and
how the reader will benefit from it. Without the WHY, there can be little or no satisfaction, motivation, and especially
learning.
Training
This section comprises the actual content of the training, and must match the objectives stated at the beginning;
if it doesn't, then obviously the desired objectives or the actual content of the training must be changed. For
this very important section to be successful, it must be clear, readable, logically sequenced, interesting, and
relevant. The process of training is as important as the content itself (Mitchell 1993). Recall effective
training you may have had in the past; what was it that enabled you to learn? Always design the actual training
with the reader's perspective in mind.
Remember that this is a training module: one brief, self-contained lesson, or one block of a series
of short lessons. Drawing from the example in Section 3.1, one TEM could lay out the basics and forcing terms of
the QG Omega equation; reinforcement within the module could be provided with a brief example (and subsequent practice)
of operational evaluation of vertical motion via those terms (using PC-GRIDDS, for example). The exercises or questions
at the end should resemble the practice section as well. People learn by DOING, so some sort of practice exercises
are essential within the TEM, and should carefully relate to the skill being taught.
Conclusion
One rule of thumb in teaching is, "Tell them what you're going to teach them, teach them, then tell them what
you taught them" (Mitchell 1993). The conclusion provides for the student a concise summary of the training.
Evaluation
Recall that the module format begins with the all-important objectives, describing reader performance at module
completion, and its evaluation (Mager 1988). The Evaluation section, then, is about providing essential
feedback to the trainee to help him or her measure progress--it is NOT about grading (other than as
a vehicle for obtaining training credit within an office training plan, for example). This section will be composed
of (most often) questions or exercises that allow the reader to apply what's learned in the module training section.
To be effective, these exercises must relate directly to the objectives laid out at the beginning.
References
Only items that have been cited in the module should be included in this section. Please refer to Glahn and Carter
(1993) for proper formatting.
Answers to Exercises
These must be included for the reader's benefit, as well as for the SOO/DOH (or other designated training coordinator).
Note that the answers to the practice exercises appear on the back of the Practice and Evaluation Exercises
sheet, while the answers to the evaluation exercises appear on a separate sheet, to be kept by the SOO/DOH.
+++ See Practice & Evaluation Page for Practice questions 1-3.
4. The Review Process
Any completed draft version of a TEM should first be reviewed by the SOO/DOH. Once this internal review is complete,
the TEM is ready for regional review. Reynolds (1993) outlines this procedure. The final version of the TEM will
be distributed to each Eastern Region office, and all regional headquarters.
5. Conclusion
The Training and Evaluation Module (TEM) is introduced, in its proper format, as a new Eastern Region publication.
It is designed to be a brief and specific training medium (i.e., includes practice and evaluation sections)
that adds a new component to field office training programs, in an era when time for training has become a significant
problem in the field. The development of the TEM and its individual components is described, and practice questions
are included. An evaluation section is also included, which provides guidance for the reader's training coordinator.
+++ See Practice and Evaluation Page for the final exercises.
Acknowledgments. The author is grateful to many people in and outside the NWS Eastern Region for their opinions
and contributions to this paper, including the SOO/DOH Training Team and other SOOs and DOHs. Particular thanks
go to Gary Carter, Ken Johnson, Mark Wool, Steve Brueske, Gail Hartfield, and Lans Rothfusz, for their time, editorial
assistance, and valuable insights.
PRACTICE
1. Name one training need you think your office has, and why (What operational skills or knowledge could be improved
with one or more TEMs?).
2. Review the part of Section 3.3 that discusses the objectives of a TEM. Compose one objective of a TEM that would
be based on the training need you listed in question 1. Include the essential elements of performance, conditions,
and criterion.
3. Review the part of Section 3.3 that discusses the introduction of a TEM. Compose a brief introduction for a
TEM that would be based on the training need you listed in question 1.
EVALUATION
1. The other Eastern Region publications are the General Interest Paper, Technical Attachment, WSR-88D
Operations Note, Technical Memorandum, and Training and Applications Note. Describe at least
three of the four motivating reasons for creating the TEM as a new Eastern Region publication.
2. Without reference to this TEM, list and explain the importance of at least seven of the eight major sections
of the TEM.
3. Based on your observations and discussion with others, list two possible topics around which a TEM could
be designed (other than those suggested in this module) that would benefit your office training program.
For one of these two training needs, a) compose one complete objective of this TEM, and b) create
a complete outline of a TEM designed to improve performance in the subject defined above. Be sure it incorporates
all components of the TEM (not the actual training itself, of course, or specific references, exercise
answers, etc.), and includes ideas for both practice and evaluation exercises (note that the questions themselves
are not required).
REFERENCES
Czepyha, L. J., 1993: ADAP manual. Eastern Region Training and Applications Notes, ER TAN-6,
National Weather Service, NOAA, U.S. Department of Commerce, 53 pp.
Glahn, H. R., and G. M. Carter, 1993: Eastern Region Publications Handbook. National Weather Service, NOAA,
U.S. Department of Commerce, 56 pp.
Kapela, A. F., and S. G. Eddy, 1992: A met intern handbook. Eastern Region Training and Applications Notes,
ER TAN-5, National Weather Service, NOAA, U.S. Department of Commerce, 19 pp.
Mager, R. F., 1988: Making Instruction Work. The New Mager Six-Pack, Lake Publishing
Company, 200 pp.
Mitchell, G., 1993: The Trainer's Handbook. AMACOM, American Management Association,
423 pp.
Reynolds, S. D., 1993: Procedures for conducting local research projects. Eastern Region Training
and Applications Notes, ER TAN-7, National Weather Service, NOAA, U.S. Department of Commerce, 27 pp.
Spayd, L. E., 1996: Education and training in the NWS: 1995 - 2005. Preprints Fifth Symposium on Education,
Atlanta, Amer. Meteor. Soc., 63-66.
Wooldridge, M. E., 1991: A forecasters guide to MicroART. Eastern Region Training and
Applications Notes, ER TAN-3, National Weather Service, NOAA, U.S. Department of Commerce, 18 pp.
ANSWERS TO PRACTICE EXERCISES
1. The question is really "answered" this way: Is this training need at your office something that you
feel can improve performance (operations) in some specific way with this type of training approach (TEM)?
2. For that same training need listed in No. 1, does your objective:
a) state clearly and simply what the reader should be able to DO (i.e., a performance), under given
conditions, upon successful completion of the module?
b) include actions that are stated as "evaluative verbs"--demonstrate, describe, identify,
etc. --as opposed to vague generalities that would be very difficult to measure (understand, know,
etc.)?
c) include actions that are in some way measurable or observable (criteria)?
3. Does your introduction:
a) provide to the reader the motivation (rationale) for undertaking this training?
b) provide some background and context for the training, including any references?
ANSWERS TO EVALUATION EXERCISES
Describe three of the four...
1. Motivation to create a TEM as a new Eastern Region publication:
a) As a consequence of the NWS modernization, there is now a critical need for training.
b) On-station training is particularly important, as time for training is short.
c) The TEM is on-station training that requires one hour or less for completion.
d) The TEM complements the other Eastern Region publications, in that (unlike the latter) it is purely a training
medium. This means that a) it is designed to improve actual performance, and b) it provides important feedback
about progress (in the form of practice and evaluation exercises) to the reader.
List and explain the importance of at least seven of the eight major sections...
2. The major sections of the TEM:
a) Header
- briefly describes the essence of the TEM to allow reader to consider its relevance to him/her.
b) Objectives
- indicate clearly and simply what the reader should be able to DO upon module completion.
- include specific actions that are measurable or observed, under stated conditions.
- include "evaluative verbs" rather than vague, immeasurable verbs.
c) Introduction
- provides the motivation (rationale) for the reader, i.e., why is this important?
- provides the background and context for the training.
d) Training
- matches the objectives.
- provides the actual content of the training, including practice exercises.
- is relevant to the operational environment of the reader.
- is clear, readable, logically sequenced, and interesting.
- can be completed (including the rest of the TEM) in one hour or less.
e) Conclusion
- effectively summarizes the training (reviews the rationale, relevance, and training).
f) Evaluation
- provides questions or exercises at the end of the TEM, to provide final feedback about progress to the reader. This may also be used by the SOO, DOH, DAPM, etc., as a means for granting on-station training credit.
g) References
- provides information to the reader that would allow him or her to obtain further details on sources of related interest cited in the TEM.
h) Evaluation Exercises and Answers
- allows for direct feedback to the reader, giving him or her a measure of progress.
3. These are, of course, office training needs as perceived by the reader. As stated in the Answers to
the Practice Exercises, the training needs listed should be those that could be improved by this training method
(TEM).
a) As in No. 2 of Answers to Practice Exercises, does this objective:
- state clearly and simply what the reader should be able to DO (i.e., a performance) upon successful completion
of the module?
- include actions that are stated as "evaluative verbs"--demonstrate, describe, identify, etc. --as opposed to vague generalities that would be very difficult to measure (understand, know, etc.)?
- include actions that are in some way measurable or observable, under stated conditions (i.e., performance, conditions, and criterion)?
b) As in No. 2 of Answers to Practice Exercises, does this OUTLINE include all the following sections
(and accomplish the essence of those sections), including only ideas for possible exercises?
a) Header
- describe briefly the essence of the TEM to allow the reader to determine its relevance?
b) Objectives
- indicate clearly and simply what the reader should be able to DO upon module completion.
- include specific actions that are measurable or observed, under stated conditions.
- include "evaluative verbs" rather than vague, immeasurable verbs.
c) Introduction
- provides the motivation (rationale) for the reader--i.e., why is this important?
- provides background and context for the training.
d) Training
- matches the objective
- provides actual content of training, including practice exercises.
- is relevant to the operational environment of the reader.
- is clear, readable, logically sequenced, and interesting.
- can be (including the rest of the TEM), completed in one hour or less.
e) Conclusion
- effectively summarizes the training (reviews the rationale, relevance, and training).
f) Evaluation
- provides questions or exercises at the end of the TEM, to provide final feedback about progress to the reader. This may also be used by the SOO, DOH, DAPM, etc., as a means for assigning on-station training credit.
g) References
- provides information to the reader that would enable him or her to obtain more information on sources of related interest cited in the TEM.
h) Practice and Evaluation Answers
- allows for direct feedback to the reader, giving him or her a measure of progress.
Return to Training Resource Center