By Dr. Calli Theisen Sanders

Calli Theisen Sanders, EdD, is Senior Associate Director of Athletics at Iowa State University, where she oversees 15 sport teams. She has also been an Associate Athletic Director at the University of Alabama-Birmingham and Montana State University, and Associate Dean of Students at the University of Maine. She has served on many national committees, including the NCAA Division I Women's Basketball Committee. She can be reached at: csanders@iastate.edu.

On most college campuses, it's common for students to evaluate their professors at the end of the term. I recently asked a faculty member here at Iowa State University about the importance of these evaluations in her academic department, and her answer was straightforward: "How could you legitimize what you're doing in the classroom without direct feedback from your students?"

But the truth is, the same principle isn't usually followed in athletics. I've traditionally based my coaching evaluations on observations made during practices and games and what I witness when I travel with teams. We also use NCAA-mandated exit interviews to help identify problem areas. But the faculty member's answer strongly suggested that we needed another piece.

In response, our athletic department implemented a process for our student-athletes to evaluate their coaches after every season. We decided that if coaches are teachers and the playing arena is their classroom, then we could really enhance our department by using evaluation techniques that have stood the test of time in the academic world.

We took one full year to implement the idea and have now completed the first year of the student-athlete evaluations. The feedback we've received has been very valuable, and we're confident that this process will help us improve our programs for many years to come.

GETTING STARTED
We began this endeavor by discussing it with our head coaches. They were clearly hesitant about the proposition and had many questions: What form would the evaluation take? Who would administer it? Who would see the results, and how would they be used?

The coaches' anxiety was relieved by assurances that this would be used as a tool to help them develop in their jobs, not as ammunition to fire them. We emphasized that student-athletes at our institution do not have the power to hire or fire their coaches, but that feedback about their experiences would be incredibly helpful to their "instructors."

Coaches also had the opportunity to offer input. Some asked whether, in certain sports, an assistant coach should be evaluated instead of the head coach. For example, event-specific coaches in track and field work much more closely with certain athletes than the head coach does. We ultimately decided each student-athlete should focus their evaluation on the head coach--since the head coach has ultimate responsibility for the sport, he or she should be the one evaluated. However, based on this discussion, we decided to allow student-athletes to evaluate their event coach along with the head coach through the questionnaire.

Next, we developed the evaluation instrument. We talked to several athletic departments with student-athlete evaluation systems in place and used their surveys as a starting point.

While it was tempting to include lots of in-depth questions in the survey, we decided it was important to keep the document short and simple. Because the validity of the results is key, we kept all questions straightforward--brief and to the point, with no room for misinterpretation. Direct questions also help ensure that the reviews are consistent between head coaches and that a baseline is established for year-to-year comparisons. And, like any good evaluation instrument, the components cover standards based on a current, accurate job description of the person being evaluated.

However, we also wanted some open-ended feedback from our student-athletes. Therefore, our form consists of a series of short questions on a Likert scale, with each question followed by a comment section.

We begin with a request for an overall rating of the head coach, and then offer a series of more specific questions. Those areas include:

• Coach's strengths and weaknesses
• Coach's knowledge of sport techniques
• Whether the athlete respects the coach
• How the head coach utilizes assistant coaches
• Coach's communication of the athlete's role
• Coach's focus on academic success
• Coach's ability to prepare athlete for competition
• Whether the coach treats the athlete with respect.

A few questions allow the athlete to comment on assistant coaches and other program support staff. And we conclude with two very general questions where athletes can add comments about any aspect of the program. (See "Scoring Your Coach" for a look at our complete survey.)

We also ask for some demographic information in the survey, such as the athlete's year in the sport and role on the team. This assists us in putting each respondent's comments into context. It also allows us to identify a particular athlete if some major red flag shows up in his or her responses. This personal information, however, is not shown to the coach to ensure the athlete's anonymity with them.

Our last step before going forward was to present the instrument to our student-athlete advisory committee. As we had predicted, they really liked the idea. In fact, they said they would feel empowered by the process. When talking with them, we were sure to emphasize the same message we told the head coaches--that the questionnaire would be a part of the overall evaluation process and would never be used as the sole basis for a firing decision. We certainly didn't want our student-athletes to have any misperception about who the decision-makers are in this regard.

IMPLEMENTATION
Some administrators conduct all their performance reviews of head coaches during the summer months, when they're least pressed for time. However, we feel it's important for the season's highlights and challenges to still be fresh in the participants' minds, so we evaluate each head coach immediately after their sport season. We administer the student-athlete evaluations at the same time, and they are included in our overall employee evaluation process.

Some may argue that the student-athletes' opinions of their coaching staff will be too directly influenced by the team's wins and losses if the evaluation occurs directly after the season. But we find those biases are easily detected, and as long as we remember to look for them, we believe the positives of this timing far outweigh the negatives.

To initiate the procedure, the athletic administrator in charge of the sport attends the first team meeting after the traditional season ends. For spring sports that conclude after final exam week, it is sometimes necessary to have this session just before season's end so all student-athletes are still available to complete the evaluation.

The administrator distributes the forms and reviews the questions with the student-athletes, without any of the coaches present. The student-athletes are assured that their coaches will not see the individual completed forms, nor will they be able to identify a student-athlete who provided any particular comment. This assurance is essential to getting open, honest feedback, particularly from underclass team members. Graduating student-athletes are always included in this process, and if possible, we also secure data from individuals who are leaving the program prior to completion of their eligibility.

Some athletes have suggested making the process electronic, so they can fill out the evaluations at their leisure, but at this point, we intend to continue with our existing method. Even though it takes more effort to type their written comments into a word processor manually, I think it ensures a better response rate. I also like that an athletic administrator is available to answer any questions athletes have as they complete the form.

Compiling the data is the most arduous part of the process, but also the most productive. After sifting through the responses, I prepare an evaluation summary with a chart that counts every rating the coach received. Each question is restated, followed by the number of student-athletes who gave the coach each particular rating. I compute the average of these ratings for each question and assign a score. An average of all these scores for all questions is then calculated, which we call the summary evaluation score. Finally, the report includes each open-ended question, followed by a series of bullet points containing every comment from the student-athletes.

VALUABLE INFORMATION
The reports are shared with each head coach during his or her annual performance evaluation meeting. After many years of conducting these reviews, I have learned that a coach's highly competitive nature makes it a challenge to present information that might be considered negative. Therefore, I use a very delicate approach in presenting the survey results.

I make sure to provide a good balance of positive and negative feedback. I remind them to take the scores and comments seriously and to use it to their advantage in working with student-athletes in the future, but not to dwell on the negative.

To begin, coaches are given the opportunity to respond to any single comment from a student-athlete that might raise administrative concern. This may include allegations of using harsh or demeaning language, or lack of support in dealing with a student-athlete's academic-athletic conflicts. But we don't spend a lot of time on isolated comments.

Rather, we focus on the common responses and trends that need to be identified and discuss them at length. These might include statements about the coach's ability to motivate players prior to competition, or a coach's use of assistants in practice sessions. The goal is for the responses to foster a discussion that helps the coach increase his or her effectiveness.

Coaches are often surprised by how their particular words and style can be interpreted so differently by individual athletes. What is viewed as encouraging and motivational by one might be interpreted as abrasive and insensitive by another. Being aware of these varying reactions can help the coach be more effective in communicating with athletes in the future.

Some coaches are also surprised by how their athletes feel about them, both positively and negatively. For example, one coach thought he had great communication with his athletes. But through this process, he learned that the athletes were uncertain about their status and roles on the team. Another coach, who uses strict discipline, anticipated negative reactions from the student-athletes, but was surprised by how much they supported and appreciated their coach's style.

One coach was surprised to find extremely positive comments from top to bottom. This coach anticipated some negativity from those who didn't get as much playing time or whom the coach put more pressure on to work hard and be successful. I think this positive reinforcement for the coach was really beneficial for motivating this individual.

In nearly every case, I utilized student-athlete feedback to establish goals for the coach in the coming year. Some of these were general, such as "use assistant coaches more effectively." Others were more specific, such as "work on better strategies for motivating athletes before competition." Sometimes I set goals for maintaining certain behavior, such as, "continue to provide excellent support for your student-athletes' academic endeavors."

For the one coach who was particularly surprised by comments concerning a lack of communication with student-athletes, we worked on goals to improve in this area. He is scheduling more meetings with athletes, and creating much more one-on-one time for information exchange.

Several coaches indicated they intend to address some of the athletes' issues head on during their first team meeting of the new year. We talked about how information is power--and if the coach can anticipate how student-athletes might perceive or interpret something, he or she can address it early and minimize problems later on.

Most of what I learned from the evaluations reinforced what I had already observed directly or knew from my daily interactions with the coaches and team members. In one instance, though, the evaluation helped identify a particular weakness in a coach's style. We kept this in mind during our search for an assistant coach for the program, looking for someone with a personality trait that would compensate for the head coach's weakness.

After each coach has the opportunity to review the evaluation summary, a chart is compiled that includes every head coach's score for every question asked, and each coach's summary score. The names of the coaches are removed and the chart is shared with all head coaches. This gives them the opportunity to see their scores in context--and to compare themselves with their peers at our institution.

In addition, the scores will be tracked on a year-to-year basis so that overall trends, or trends in one particular area, can be identified. This provides an additional component in the coaches' annual reviews to discuss areas where progress is being made or sustained, as well as areas that might not be moving in a positive direction or may need improvement.

THE ULTIMATE GOAL
A system that provides student-athletes with the opportunity to evaluate their head coach produces a lot of winners. The coaches win by learning more about student-athletes' perceptions of them. The student-athletes win by having a chance to be heard and to feel they are a valued part of a very important process. And athletic administrators win by gaining additional feedback about their coaches that helps improve the entire program.

But what may be the greatest benefit of these evaluations is that they lead to healthy, productive conversations between coach and administrator. Topics not previously on the table arise and in-depth discussion of student-athlete welfare can occur.

There is also great potential for these surveys to be used for other benefits in the future. For example, additional questions could be linked to specific department goals. Or questions could become specific to the head coach based on his or her strengths and weaknesses.

Tremendous learning takes place in the coach's "classroom," and as administrators, we have an obligation to assist in making the experience positive, valuable, and memorable for both instructor and student. At Iowa State, student-athlete evaluations of coaches are helping us with this ultimate goal.