Editorial Note: This article has been reproduced for syndication with permission from Dr. Jessica Egbert, Vice President of Institutional Effectiveness and Community Engagement at Rocky Mountain University of Health Professions, and the Association for Institutional Research (AIR). The original article can be found here.
Measuring the holistic student experience is complicated. To ensure meaningful and actionable data, strategies for evaluating and demonstrating learning must go beyond grades; similarly, strategies for evaluating student perception of the experience must go beyond cursory survey data.
At my institution, we are not tied to standardized surveys (many are not relevant as we offer only graduate-level education). The disadvantage is that national benchmarking without standardized surveys is more cumbersome, yet the advantages are the flexibility of integrating best practices, analyzing higher education trends, and evaluating mission fulfillment meets our changing data needs.
The scouring of literature led to an institutional survey structure change in 2017 that introduced focused appreciative inquiry. “AI is a qualitative approach focusing on potential strengths, and is an important approach in the current economic climate of austerity” (Collington and Fook, 2016). Utilizing AI adjusts the survey approach from a deficiency model to an abundance model. It is not that we do not seek constructive criticism in our surveys.
However, as Collington and Fook (2016) note the problem with deficiency models “…is that there may be some inherent advantages of an existing situation which may then be ignored, and so a full or complex picture is not gained.”
When I consider an AI approach to assessing a holistic student experience, the following words come to mind: storytelling, conversational, context, and affect. These are not terms I’ve traditionally associated with institutional research. However, institutional research does not exist for data mining and reporting alone; rather, it tells institutional stories in meaningful and actionable ways. Contextual competency informs meaning and action; AI is powerful strategy to add context.
In addition to the value AI provides in adding context, we may also consider the value of AI to our survey participants and related constituents. Historically, extensive effort had to be put forth for prospective students, employees, donors, or others to discover the sentiment surrounding an institution. Today, readily available for our digital natives are objective and subjective reviews of any product or service. Additionally, social media creates opportunities to instantaneously share reviews to one person or to millions of people. Often, it is our own educational programs, facilities, athletics, compliance, and culture of student-centeredness that are the subject of emotion-laden commentary.
A quick search of online dialogue in higher education reveals conversation topics on reputation, enrollment, innovation, litigiousness, governance, influence, marketing, safety, diversity, cost, alumni, and fundraising. Layers of diverse and often divergent sentiment accompany each of these topics. By failing to address the emotional nature of humanity, we are overlooking critical data in our reporting, continuous improvement processes, and in the authenticity of our storytelling.
So, is “feeling” important to institutional research? Absolutely.
I recently inquired of academic personnel regarding the recent integration of AI survey items. A master’s degree program director noted an increase in student comments, a doctoral degree program director noted that the positive addition of AI items to student surveys provides her an opportunity to recognize faculty in new ways, and an additional doctoral faculty member noted the contribution of AI towards demonstrating compliance with specialized accreditation. Additionally, the Provost provide a favorable response to collecting AI data, including probing questions to drive continuous improvement initiatives. This feedback validates the literature in demonstrating how the qualitative AI questions drive measurable outcomes (e.g., increased quantities of responses, expanded recognition programs, and continuous improvement initiatives).
The simplest way to integrate an AI philosophy into practice is to redirect or expand existing strategies to include strength-based approaches. For example, ask the respondent to describe what is going well or to describe the most positive, memorable response. A mixed-methods approach to holistic assessment is the Net Promoter Score (NPS).
NPS is a best practice measurement utilized most often in the private sector and which is quickly gaining traction in higher education. As described by Net Promoter (2017), “Net Promoter Score®, or NPS®, measures customer experience and predicts business growth.” Utilizing the NPS 0-10 point scale in conjunction with an AI-oriented open-ended question inquiring for the respondent’s rationale for the rating, an institution main gain unique, actionable insight of the holistic student experience. The NPS, as a measure of loyalty, expands our view of what makes for a loyal enthusiast who will refer others (e.g., to a major, a campus service, or to the institution).
Numbers alone may reveal a lot, but the absence of context increases the likelihood of misinterpretation and missed opportunity. “At its heart, AI is about the search for the best in people, their organizations, and the strengths-filled, opportunity-rich world around them. AI is not so much a shift in the methods and models of organizational change, but AI is a fundamental shift in the overall perspective taken throughout the entire change process to ‘see’ the wholeness of the human system and to “inquire” into that system’s strengths, possibilities, and successes” (Stavros, Godwin, and Cooperrider, 2015). As you consider exploring the holistic student experience and its impact on your institution, I recommend exploring recent literature accessible through your campus library or Google Scholar and integrating AI principles into your data collection and storytelling.
- Collington, V., & Fook, J. (2016). Instigating Change through Appreciative Inquiry: A Case Study. International Journal of Higher Education Management, 3(1).
- Net Promoter. (2017). What is Net Promoter? Retrieved November 14, 2017, from https://www.netpromoter.com/know/.
- Stavros, J. M., Godwin, L. N., & Cooperrider, D. L. (2015). Appreciative Inquiry in Practicing Organization Development: Leading Transformation and Change (Fourth ed.). Hoboken: John Wiley & Sons, Inc.
The post Appreciative Inquiry: Measuring the Holistic Student Experience appeared first on Watermark.