It can be tough to gauge your success in the educational world. Are your programs really making a difference to students? Do classes target academics as well as real-world issues? Could they be improved upon?
As you evaluate your school’s programs, consider asking former students how they fared after high school. One Colorado school did just that.
In the fall of 2014, the International Baccalaureate (IB) program at Poudre High School (PHS) in Fort Collins, CO, was challenged to rethink the operational, instructional, and administrative policies and procedures of the program.
To begin the process of improvement, the IB faculty was asked for their input. Upon review, the comments included conflicting views about existing IB policies and procedures, and it became apparent that to fully assess the program, staff needed to hear from additional stakeholders. One idea was to survey current students—the rationale being that since students are the ones who have to adhere to the policies, they would provide valuable information as to what constitutes realistic terms. The faculty also considered looking at existing data on IB exit exams to see whether areas for improvement were highlighted.
Calling All Former Students
We needed a better approach for program assessment, and we surmised that alumni feedback could provide valuable perspective for improvement. Student-driven policy can be problematic because students are not necessarily knowledgeable about what the end goal of their course of study should be.
We found the best approach for measuring program effectiveness was surveying former students, because alumni feedback is based on experiences influenced by a wider context of life, including participation in postsecondary experiences and careers. Indeed, if the goal of the PHS IB program is to prepare students for life beyond high school, then it is only logical to ask those students how well the program met those goals.
Although the process of surveying alumni was daunting at first, the PHS IB program embarked upon a concerted social media effort in the fall of 2014 to recruit participants. Fortunately, one of the teachers in the program had maintained contact with numerous graduates, so a general call went out on social media seeking volunteers from the graduating classes of 1998–2013 to take the survey. Over the course of the next three weeks, 175 graduates (from 2003 to 2014) indicated their interest in participating in the study and providing email addresses for further communications. Then, all 175 students were randomly assigned to one of two groups: One group took a quantitative survey; the other group took a qualitative survey. The questions, designed by the IB director with input from other IB staff, included demographic, reflective, and evaluative questions addressing which aspects of the program were most or least memorable or useful in life after high school. In the end, 150 surveys were returned. Although both quantitative and qualitative surveys were distributed, this article highlights the results of the qualitative surveys.
After analyzing and coding the results, several themes emerged in response to the question: “To what extent did participation in the IB program prepare you for postsecondary pursuits?” We received overwhelmingly positive responses from graduates. Many cited specific skills as beneficial in college courses, including writing, critical thinking, and source analysis. Approximately 30 percent of the responses specifically addressed the fact that they felt more confident entering college courses, and 80 percent reported feeling more prepared in college classes because of their IB courses and exams. The writing skills learned in IB were most often cited as beneficial in all college courses; one student noted that writing research papers in college was simple after IB.
Research and study skills were also seen as beneficial by the majority of students. One student enthusiastically stated, “I was ideally prepared for college.” And another indicated that, “Overall it seems like the breadth and depth of the IB program, combined with a variety of different experiences … are what really help in college.”
Informing Program Instruction
These results are highly useful in informing program instruction, policies, and practices because the recommendations come from alumni with experiences that provide a rich description of what is truly beneficial in college. Alumni surveys gave the IB program a stronger sense of the job it did in preparing students for life beyond high school, and provided a sense of the areas where it could perform better.
When it came to what the program could improve upon, most students referenced either the perceived “elitist” mentality of the program itself, or the lack of a specific course offering during their time in the program. While these are important points for improvement, they do not necessarily suggest that students were not prepared for life beyond high school. Instead, they point to individual preferences and feelings that may not be indicative of college preparation.
Going through this process provided some profound systemic implications. For instance, reflections from middle/junior high students on their elementary school experiences and how well they were prepared for the next level might provide insight for curricular decisions. Likewise, high school students likely have thoughts on their experiences in middle/junior high and how prepared they felt for high school. In both cases, the input from alumni could facilitate discussions around academic, personal, and social skills and strategies to ensure student success in grades K–12.
Additionally, data from alumni surveys has an advantage over surveys of current students, because current students may correlate their anticipated grade in a course with feedback they provide for the course and/or instructor—whereas alumni have had space and time to reflect. For example, experiences in high school with an easy-but-popular teacher may later be tempered by the realization that those experiences did not prepare students well for the next phase of life. On the other hand, experiences in high school that were frustrating and may have resulted in negative feedback on a student survey at the time may be more deeply appreciated when considered later, as students realize such experiences may have helped to develop academic and social skills that were used later in life.
Beyond Standardized Tests
Finally, feedback from former students provides additional data for districts that rely on standardized test scores to evaluate teacher and student performance. While test scores are one point of data, they do not necessarily take into account extenuating circumstances such as student transiency and student effort, thus creating a potential issue of validity. Reflections from alumni about their level of effort and attendance in school may mitigate the issue.
One of the greatest implications of this survey process is that forward-thinking principals and innovative school districts can improve student engagement, strengthen vertical articulation, and address the Every Student Succeeds Act (ESSA) indicator of “student engagement.” Alumni input serves to strengthen the school’s programs and provide valuable insight into systemic policies and procedures.
Cori Hixon, MEd, is an assistant principal and International Baccalaureate director, and Russell C. Brown, MEd, is a social studies teacher. Both serve Poudre High School in Fort Collins, CO.