THE PEER REVIEW PROCESS AS A STEP TOWARD PROGRAM ACCREDITATION AND AS AN ALTERNATIVE TO LEADER CERTIFICATION
by Simon Priest, Ph.D.

Certification of outdoor leaders is currently a "dead" issue. The topic was ignored at a recent summit conference on outdoor leadership (Phipps & Cash, 1991) and the British preparation schemes, which began the entire movement toward certification some thirty years ago, did away with this requirement well over a decade ago (Priest, 1987). The failure of certification to solve the problems of increasing accidents and environmental impact inherent to the field of adventure programming is likely due to several shortcomings of the training and assessment process (March, 1980). The most notable drawback is that certain key components of leadership do not easily lend themselves to valid and reliable evaluation: you just can't certify meta skills such as judgment (Rollins, 1983).

Wade (1983) has recommended accreditation of programs as a viable alternative to certification of people, and recent workshops at AEE conferences have tended to confirm this option. The profession appears to support accreditation and sees certification as something that programs do in-house for their own staff. A recent study for the AEE (Bassin, Breault, Fleming, Foell, Neufeld & Priest, 1991) found that two thirds of all organizational members surveyed would prefer program accreditation, provided the scheme was regionally administered (to reduce costs), voluntary in nature, and evaluated by external peer review (rather than by internal staff). To this end, the AEE has begun testing a program accreditation model in the northeastern United States based on the peer review process (Gray, 1990).

Regardless of whether accreditation becomes a panacea for the ills of our field or follows the certification path toward extinction, the underlying process of peer evaluation is of far more importance and usefulness to adventure programs interested in reviewing operational safety. The remaining focus of this article is on safety reviews conducted by external experts drawn from a program's peer group. The content serves as a layperson's guide to conducting such reviews and is excerpted from the AEE's new book on Safety Practices in Adventure Programming (Priest & Dixon, 1990, pp. 19-22). This book states that, in keeping with commonly accepted practice, "programs must review the safety of their entire program on a regular basis," suggests that programs document internal safety reviews once a year, and recommends a thorough peer review by outside experts "once every two years" (p. 19).

Safety reviews check the safety operations of programs in a proactive, rather than a reactive, manner. Programs ought "to establish some form of safety review system, preferably one involving external peers who can provide truthful and critical advice without threatening the program's collective ego. At best, a program will have two forms of safety review: internal (from ongoing efforts of [an] in-house safety committee) and external (periodic involvement of outside experts or peers). The two forms of review attempt to compare the program's performance on safety with known state-of-the-art safety guidelines and if all is trustworthy, valid, and reliable, the findings of the two reviews [ought to] agree" (p. 19).

"In a safety review, a small team of people visit and observe the program in action. During the visit they collect data concerning the safety practices and procedures of the program, based on their observations. From these data, and from their experience-based judgment, they infer whether the program is safe and what changes might improve the overall level of safety. They discuss their findings with the appropriate program representative, and/or the safety committee, which provides a chance for feedback and checking of impressions. In conclusion, they write and present a summary report of the review methods and results, along with their recommendations for changes" (p. 19).

The following list, adapted from Wade and Fischesser (1988), outlines the format and content of a safety review. Any adventure program considering such an approach may find this list beneficial.

Before the review

• Establish guidelines for conduct of the review: who, what, where, when, why, how much?
• Appoint a representative liaison between program or safety committee and safety review team.
• Select mutually acceptable team members (avoid controversial or political appointments).
• Examine existing paperwork (brochures, reports, reviews, forms, manuals, lists, and logs).
• Inform staff of pending interviews and encourage open and honest sharing of concerns.

During the review

• At the introduction, discuss particular areas of safety concern and attempt to ease anxieties.
• Maintain professional, diplomatic, and tactful behavior throughout the safety review.
• Explain the review to participants who ask; avoid public criticisms which may be overheard.
• The review content may be patterned after items on this list or other items may be included.
• The team may wish to divide responsibilities for observing, interviewing, reading, etc.

After the review

• Orally present findings (in private) to key program personnel (include safety committee).
• Statements about safety practices or observed behaviors should be factual, not opinionated.
• Exchange new ideas for improving safety through transactional (two-way) communication.
• Balance negatives with positives; permit personnel the chance to explain any disagreements.
• Submit written report giving requirements, recommendations, suggestions, and observations.

Some items worth considering during a safety review!

• Does the program clearly state philosophy, mission, goals, and objectives in a safety policy?
• Does that safety policy explain the educational rationale behind putting people at risk?
• Do program guidelines apply to staff and participants alike or is there a double standard?
• Are safety (risk) management plans completed for each adventure activity or field trip?
• Do these plans contain the necessary safety preparations and emergency information?
• Are accident response kits (including first aid materials) present for all activities?
• Are there emergency communication links from field staff to outside medical help?
• Are drivers appropriately licensed to drive vehicles and transport numbers of passengers?
• Are safety practices such as seatbelts and equipment checks in place for transportation?
• Is equipment tested and used properly in the program? Is a repair and maintenance log kept?
• Are participants warned and informed of the risks and responsibilities prior to acceptance?
• Are the pre-enrollment orientation materials (sent to participants) accurate and adequate?
• Do participants complete an appropriate health/medical disclosure form and waiver/release?
• Does the program carry adequate/appropriate insurance coverage for accident and liability?
• Are staff competent in dealing with participants' typical and emergency health care issues?
• Is there a policy statement concerning the use and abuse of alcohol and other drugs?
• Are participants screened for health concerns or do they take a pre-enrollment medical exam?
• Are activities or sites chosen for potential to meet program goals and educational objectives?
• Are participants properly briefed and progressively prepared for each activity or site?
• Are participants coerced or forced into a particular activity or are they challenged by choice?
• Does the program conform to local resource regulations and correctly apply for use permits?
• Do on-site and local facilities adhere to the public laws governing their proper operation?
• Does the program have a safety committee? Does it monitor and sanction safety practices?
• Are accident and incident data reported, compiled, analyzed, and acted upon accordingly?
• Do job contracts include procedures for training and assessing staff? Are these followed?
• Do reasonable or acceptable staff/participant ratios exist for all adventure activities and sites?
• Are groups with designated staff in the field restricted to environmentally sensitive sizes?
• Are appropriate minimal impact environmental procedures taught and ensured by staff?
• Do staff have the necessary combination of hard, soft, and meta skills required for their jobs?
• Are staff competent at the adventure activities they run and familiar with the sites they use?
• Do staff anticipate dangers and are they capable of responding to accidents while in the field?
• Are staff appropriately qualified or certified in first aid and CPR? Is such evidence on file?
• Do staff files contain up-to-date records of training and assessment and/or recent certificates?
• Are staff overworked or burned out? Do they appear to work well together as a group?
• Are (written) procedures in place to handle (most commonly expected) emergency accidents?

With certification of individual leaders failing to address today's concerns of accidents and environmental damage to the outdoors, adventure program accreditation seems like the next best thing because it considers program safety as a whole rather than looking at the one small aspect of staffing. As this trend gains momentum, we may discover it too has shortcomings or we may find that the new scheme becomes unmanageable or restrictive. However, the key to success lies in avoiding the urge to toss out the baby with the bath water. Let us keep the component of safety reviews - they will go along way to helping us avoid government intervention and remain a self-policing profession.

REFERENCES
Bassin, Z., Breault, M., Fleming, J., Foell, S., Neufeld J. & Priest, S. (1991). An AEE Organizational Membership Preference for Program Accreditation. Unpublished report.

Gray, D. (1990). A pilot model for a New England peer review program. Unpublished manuscript, Association for Experiential Education.

March, W. (1980). Assessing outdoor leaders: The catch-22 of wilderness leadership certification. Foothills Wilderness Journal, 7(2), 16-17.

Phipps, M. L. & Cash, R. G. (1991). National Conference for Outdoor Leadership: Employee Preparation toward 2001. Gunnison, CO: Western State College of Colorado.

Priest, S. (1987). Outdoor Leadership Certification: always an issue, but no longer a trend. Bradford Papers Annual, 2, 37-44.

Priest, S. & Dixon, T. (1990). Safety Practices in Adventure Programs. Boulder, CO: AEE

Rollins, R. (1983). Leadership Certification Revisited. CAHPER Journal, 50(1), 8-9.

Wade, I. R. (1983). An alternative to certification programs. Unpublished manuscript, Outward Bound USA.

Wade, I. R. & Fischesser, M. (1988). The Safety Review Manual: A guide to conducting safety reviews for assessing and upgrading safety in outdoor adventure programs. Unpublished manuscript, Outward Bound USA.