Case Study

Leveraging Technology for General Education Learning Outcome Assessment with Hawaii Pacific University

About Hawaii Pacific University

Founded in 1965, Hawaii Pacific University (HPU) is now the state’s largest private nonprofit university with 32 undergraduate and 17 graduate programs serving 7,000 students, including students from every state and 65 countries. HPU’s programs strive to anticipate the changing needs of the community and prepare graduates to live, work, and learn as active members of a global society.

The Challenge

In the fall of 2015, HPU launched its current general education program, which takes advantage of Hawaii’s position as the American gateway to Asia and the Pacific to introduce students to ideas, perspectives, and experiences relevant to their lives. “We cultivate the skills, knowledge, and values expected of all educated persons: critical thinking, information literacy, oral communication, quantitative reasoning and written communication,” said Valentina Abordonado, assistant dean of general education.

HPU has adopted the Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics launched in 2007 by the American Association of Colleges and Universities (AAC&U). “Importantly, we have also adapted them. For instance, our English department took their big red pens and wrote all over the written communication rubric and in so doing, established ownership over that rubric. It became ours,” Abordonado said. “We love these rubrics because they privilege the role and importance of faculty as authors of assignments and arbiters of quality.” HPU elected to evaluate student work with assignments already embedded in their courses.

But HPU needed technology to streamline the process of assessing HPU’s 13 general education learning outcomes.

The Solution

To support all aspects of assessment, HPU adopted Taskstream by Watermark for planning and assessing findings and Aqua by Watermark for evaluating student artifacts. “We were so fortunate to work with Watermark. They provided us with the technology tools to guide us, and I must say that we have leaped forward much faster than we might have done left to our own devices,”Abordonado said.

Taskstream provides a digital workspace for each program where they can:

  • Articulate a mission statement
  • Identify program learning outcomes and align them to institutional learning outcomes
  • Map courses to program learning outcomes and identify measures to assess outcomes
  • Examine and reflect on assessment findings, plan actions based on findings, assess the impact of those actions, and monitor activity to track progress
  • Demonstrate success

The process began with selecting the assignments to be assessed and having students submit them via the LMS. HPU also identified faculty scorers in each college and facilitated rubric norming.

“Although it sounds like a mind-numbing event, rubric norming was really the the most exciting part of this assessment project, because it gathered faculty in a single room and sparked conversations about learning and our criteria for learning and student performance,” Abordonado said. “I’ve been kind of a fly on the wall in conducting these rubric norming sessions, and it’s just been wonderful to observe the conversations that faculty have.”

They also worked with the academic assessment and program review committee to ensure they could aggregate assessment results, disaggregate results by degree program, and analyze and reflect on results as part of annual assessment planning.

HPU created annual assessment cycles for each year, including an assessment plan, assessment findings, an action plan and a status report so they could look back on their plan and see how they were doing.

The Win

With the stage set in Taskstream’s assessment planning system, HPU set up its first assessment measurement project in Aqua, declaring 2017-2018 the Year of Written Communication. “My takeaway from this is just how easy it is. I was able to literally set up this project in under a half an hour,” Abordonado said.

When students had submitted their work through the LMS, faculty received an evaluation queue of artifacts. “We discovered that there was a really simple scoring process for faculty. It was easy, quick, and convenient,” Abordonado said. “It was intended for busy faculty who want to know exactly what to do and how to do it.”

Even more than ease of use, HPU appreciated the system’s results. “Aqua provides us with a very powerful reporting tool. It’s clear, clean, and focused on the results that we want to know about,” Abordonado said.

The results showed that while students met the defined “acceptable” target for most criteria, there was work to be done on style, syntax, and grammar. Similarly, students needed more emphasis on academic conventions and synthesis to reach the defined “ideal” target.

“This really was a conversation sparker for us. And we knew that the final step in this project was to close the loop,” Abordonado said. “So we asked ourselves, ‘How we should use these findings? Are we satisfied with these results? If not, what are we going to do about it?’”

After discussing their findings at HPU’s Assessment Day and in relevant committees, deans asked each program to respond to the results and make an action plan. “It gave faculty a way to begin the conversation about achieving the desired outcome for written communication in general education and capstone courses,” Abordonado said. The project also allowed them to answer an accreditation standard for WSCUC. “We were able to take the work we did in Aqua and include it as a piece of our student success website. We have links to the rubrics and our results on the website, so Aqua has helped us move towards achieving this particular standard.”

Looking Ahead

Now that HPU has strong processes and supporting software in place, it has begun assessing two to three learning outcomes per academic year. This puts them on schedule to assess all learning outcomes in the General Education program within the five-year program review cycle.