|
|
|
The Use of the Graduating Senior Exit Interview as an Effective Assessment Tool
Richard J. Puerzer
Department of Engineering Hofstra University
David M. Rooney
Department of Engineering Hofstra University
I. Introduction The exit interview is a tool that has long been used in a variety of industries to assess important measures, such as an employee’s experiences during his or her tenure with a company or a patient’s perception of the quality of care received while in a hospital emergency department. Likewise, in assessing an engineering program, the exit interview can be a valuable tool in determining the program's success in imparting the abilities enumerated in the ABET 2000 guidelines. Information obtained from exit interviews can be especially useful for smaller engineering programs for the interviews can provide timely feedback regarding the changes the program is undertaking in order to refine their goals and methods in the educational enterprise. Also, exit interviews can be used to quickly determine market trends regarding hiring, salary, and graduate school acceptance of new graduates. In conjunction with other assessment measures such as an alumni survey, a school or program can compare how well it prepares its graduates regarding the eleven key outcomes as defined in the ABET 2000 guidelines. Finally, the interview may be the final personal contact between a graduating senior and the educational institution, and thus may be the final opportunity to elicit information pertinent to its achievement of its goals. II. Background Hofstra University, located in Hempstead, N.Y., is primarily a liberal arts college which has housed a Department of Engineering (offering ABET-accredited degrees since 1971) within a College of Liberal Arts and Sciences. A characteristic of the engineering programs at Hofstra is their small size. During a given academic year, the population averages about 250 undergraduate majors, divided among programs in Electrical Engineering, Industrial Engineering, Mechanical Engineering, and Engineering Science. The last named degree consists of Biomedical, Civil, and Environmental options. In 1999, the engineering department at Hofstra elected to be accredited under the ABET 2000 guidelines. In preparation for the accreditation visit, the engineering department began the use of several tools for the determination of outcomes regarding the educational objectives of the department. One such tool introduced was the interview of all graduating seniors for the purpose of evaluating their perceptions as they left Hofstra and moved on to employment as engineers and/or graduate school. It was believed that graduating seniors would provide interesting and meaningful information on many aspects of their engineering education, ranging from what they found most useful to what they disliked and found ineffectual.
III. The Use of Exit Interviews Several engineering programs have documented their use of exit interviews in the assessment process in accordance with ABET 2000. Likewise, the methods used for the elicitation of information from graduating students vary from program to program. Several engineering programs, including the Virginia Military Institute, the University of Maryland, and the University of Pittsburgh have reported the use of an exit survey to be completed by graduating students. Other programs have reported the use of an exit interview, including the University of Scranton and Manhattan College. Still other programs, including Marquette University, Trinity College, and Western Washington University have reported the integration of an exit interview and an exit survey. This was also the approach taken by the Department of Engineering at Hofstra University. The goal of the exit interview at Hofstra University is somewhat different from the goal of an alumni survey, since the graduating senior has typically not gained the perspective to evaluate a program’s effectiveness in respect to all the ABET objectives and outcomes which alumni with several years’ worth of professional experience could more intelligently address. However, it can probe several of those objectives, and can focus on certain areas which other assessment tools indicate are high priority areas demanding attention. IV. The Exit Interview Process at Hofstra The exit interview process begins about midway into each semester with the mailing of notification letters to all students anticipating graduation at the end of the semester. These letters explain to the students that they are required to schedule a brief meeting with a specified faculty member in the weeks before the end of the semester. Enclosed with the letter is a brief questionnaire for the students to fill out before the meeting. A concerted effort is made to interview all graduating seniors each semester. In the past three years, approximately 80%-90% of all graduating seniors have been interviewed. Of course 100% participation is always the goal, but several factors militate against a perfect coverage, especially because of the difficulty in arranging exit interviews with students graduating at the end of the summer (rather than in December or May) and with part-time students completing a program in a semester when they are taking one evening course, often not within the department. While the letter informing them of the process can suggest the great importance of participation, it cannot currently imply that it is mandatory, since it is not a published graduation requirement. The core of the process at Hofstra consists of a private interview with a faculty member. The faculty member will have reviewed the written questionnaire in preparation for the face to face meeting. It is used to establish some basic demographic information on the graduating student and their immediate employment prospects. It also contains a series of questions asking students to give opinions the strengths and weaknesses of their degree programs. Three faculty members have conducted the interviews over the last three years. These faculty members represent the electrical engineering, mechanical engineering, and industrial engineering programs. In an attempt to avoid bias in the interview data collected, students are matched with faculty who do not represent their major. Although the students may have had some classroom contact with their interviewer, it is thought that their opinions will be stronger concerning their major. The three faculty members selected as interviewers were chosen because of their non-confrontational nature in dealing with the students in order to elicit a frank response in the interview. The faculty member begins the interview by explaining the purpose of the interview as part of a process to improve the effectiveness of the degree program, and not in any way as an evaluation of the student. This tends to motivate most students to go into detail in their discussion of their engineering education. The faculty member also assures the student that the information discussed in the exit interview will be held in confidence, and that all information taken from the interview will be combined with that of other interviews in the creation of the exit interview report. This is especially important to emphasize, because the effectiveness of specific faculty in teaching particular courses may arise in the course of the interview, and some students will be more naturally reticent about voicing opinions about an interviewer’s professional colleagues if they are not assured at the outset of the anonymity of their comments. Then, using the questionnaire as the basis for many of the questions, the interviewer asks the student to elaborate on the written responses. Specifically, students are asked about their future plans, their criticisms of their educational experience, and their suggestions for the betterment of the engineering department and the university. Often times, the faculty member may need to draw information out of the student by asking questions such as "what was your favorite class and why did you enjoy it," or "was their anything that you wished that you had been taught, but were not." Sometimes the department may be considering structural changes in a curriculum, and here too the graduating senior can provide valuable input. The faculty member may also ask the student how prepared they feel to be entering industry. Other times, the faculty may not need to draw any information out of the student, as he or she is more than willing to offer criticisms and suggestions. The interview is then closed by thanking the student for his or her information and congratulating the student on his or her achievement. The student is also asked to maintain contact with the department in subsequent years by forwarding any address change directly to the departmental secretarial staff. The interviewer alerts the graduating senior that a more comprehensive survey will be mailed to him or her within a few years as part of an alumni survey, which the department undertakes every three years. As this may be the last personal communication between the student and the university, it behooves the faculty member to leave a positive caring impression on the student. V. Using the Results The information culled from the exit interviews and surveys can be used for a variety of assessment purposes. The three faculty members who conduct the interviews meet annually to discuss the year’s accumulation of interview data. The files are then divided largely according to interviewer, but some redistribution of files takes place to guarantee that final reports group all graduates of a given degree program in the same report. Then, they individually write reports on these results, concentrating on several key areas. These key areas include the immediate future of the graduates, the general perception of the graduates regarding the Engineering Department and the University as a whole, specific criticisms of the graduates, and suggestions by the graduates for the improvement of the Department. In discussing the immediate future of the graduates, the report includes the count of those already holding job offers, those accepted into graduate school, and those still waiting; the average starting salary and title of those holding job offers; and general observations including perhaps a comparison of this year’s graduates to those of previous years regarding this area. Specific criticisms of the graduates regarding classes, professors, laboratories, as well as concerns outside of the department (the job placement center, for example) are all discussed. The suggestions run the gamut from the revision of a curriculum to concentrate on certain areas, to suggesting that a specific professor to change his methods of presentation. The three exit interview reports are then presented to the faculty of the Department of Engineering each October as a part of the regular faculty meeting. The faculty discusses the reports and determines if any action is warranted. Examples of action in the past that have been precipitated by the results of the exit interviews include: forging a stronger relationship between the job placement office and the department in an effort to educate those in the job placement office to the needs of the engineering students, replacing and updating laboratory equipment which the students perceived as antiquated, and making AutoCAD the standard tool for the computer aided design class and applications in the department. After discussion by the faculty pertaining to the three exit interview reports, those reports are then distributed to those stakeholders found outside of the department, specifically the Department’s Industrial Advisory Board. An oral summary is presented by one of the interviewers and the information is discussed at one of the five annual Industrial Advisory Board meetings. Finally, these reports are filed for future ABET evaluations and for maintaining a documentary record giving a historical perspective on graduating senior evaluations of the programs. This exit interview report is useful for a small engineering program in a number of ways. First, the exit surveys provide timely information on student perceptions. This is especially useful in a small program because it would require several years of data collection to achieve a meaningful sample size for use in evaluating student opinions in a tool such as a survey. Second, the surveys provide a means for evaluating the impact of change in the engineering department. Although the perceptions of individual students must be considered anecdotal, still, trends within groups of students can easily be identified and tracked from year to year. Lastly, the reports can be important input in the continuing deliberations over how to allocate departmental resources on items such as software and laboratory tools. If it can be shown that students, and not just faculty, perceive the need for new tools in order to be better educated, the department may be more easily moved to make such purchases. The information gained from exit interviews can also be triangulated with data drawn from other outcomes assessment tools, such as alumni surveys and industrial advisory board opinions, and can then be weighted accordingly. VI. Conclusions The exit interview can be an effective tool for the measure of outcomes assessment. Students are of course the primary stakeholders in the educational enterprise, and their opinions must be solicited. The exit interview collects student opinion at an important juncture in their lives, a point at which they are still a part of the department but also at which they may have developed some perspective concerning their education. The information culled from the exit interviews can be considered as timely feedback upon which to base departmental decisions. It can also be examined over a longer period of time to track the effectiveness of changes made in the department to improve the education of the students. Finally it can also be combined with information gained from other outcomes assessment tools in order to triangulate the effectiveness of the educational enterprise.
References
|
|
ASEE 
