Improving the Quality of Assessment with University of Arizona
Founded in 1885 as a public land grant institution, the University of Arizona (UA) now serves more than 45,000 students across 19 colleges and schools. UA’s mission is to improve the prospects and enrich the lives of the people of Arizona and the world through education, research, and creative expression. The university strives to expand the student experience through engagement, advance knowledge through innovations in creative inquiry and collaboration, and forge novel partnerships to positively impact its community.
In 2009, interest in assessment was gaining momentum across the country, and accreditors began looking more closely at outcomes. In response, the University of Arizona formed its Assessment Coordinating Council. The council, comprised of associate academic deans or their designee, was charged with bringing assessment to the forefront for the colleges and getting them to engage. In response to the council, the University of Arizona designed and built a website to help track assessment. The site included a page for every department and major. Each department updated its page annually, reporting on student learning outcomes, how the outcomes were measured, what the findings were, and changes to be made in response to the findings.
Over the course of seven years, each program at the University of Arizona undergoes an academic program review, with departments updating their assessment webpage and reporting their program assessment and review efforts. Assessment plans are scored against a rubric and the department and the Office of Instruction and Assessment (OIA) reviewer develop an improvement plan.
By the time work with cohort six began, it was clear that the university’s academic program review website had a number of flaws, including poorly organized information and loss of previous findings, which were deleted when departments posted new findings. Because previous findings were lost, OIA had to go back to earlier versions and compare them to find the changes. In addition, there was no consistency from page to page, so it was impossible to run analytics across web pages. This made finding the needed information for accreditation reports, as well as progress reports that go out to the colleges and the provost for review, time consuming.
“It was so hard to review department assessment websites that we became discouraged,” said Dr. Elaine Marchello, OIA’s Assistant Director of Assessment. The website was cumbersome and unintuitive, leaving users frustrated and unmotivated. People needed to be continually reminded and prodded to put in the necessary assessment-related information.
Marchello and the team knew they needed a more intuitive, easily trainable assessment solution that reflected how the university reported assessment information. After reviewing a couple of options, the University of Arizona selected Taskstream by Watermark. For Marchello, the choice was simple. “Once I saw their Accountability Management System [Taskstream], I wanted to stop looking,” Marchello said. “I called my colleague and told her, ‘You have to see this.'”
What so impressed Marchello? The system was easy to use, provided better reporting, and had useful analytics.
And then there was the Watermark team. “Customer service is important to us,” Marchello said. “We wanted a company that was easy to work with, had a good help desk, and offered a helpful tutorial and Taskstream by Watermark checked all of those boxes.”
In December of 2016, the University of Arizona began using Taskstream by Watermark.The OIA spent the rest of that calendar year working with a volunteer department to pilot the program, helping users get accustomed to the software, and designing a workshop to bring the rest of the university departments onto the new system. To reach the department members, OIA did one-on-one orientations.
The full rollout of the program began in early 2018. From January to May of 2018, the OIA conducted two training workshops each month with two to three departments attending each session.
OIA was “blown away” by attendee response. Based on survey results, nearly 76 percent of attendees described their workshop experience as either good or awesome; just under 80 percent said they were comfortable using the system or believe they were but had not yet used it. Dr. Ingrid Novodvorsky, Director of Teaching, Learning, and Assessment, knew the workshops went well before seeing the results of the survey, “A common comment was, ‘Oh this is so much easier than reporting on the website,'” Novodorsky said.
The positive response to the workshop has driven system use. With more than 50 percent of the users registered, the OIA estimates that they already have double the information they had with the previous system. In addition, the information is clear, uniform, and aligned.
Using Taskstream allows OIA to more efficiently handle their workload. “It’s so much easier to get my work done since we got Taskstream because everything is now in one place,” said Marchello.
Novodvorsky concurred, “Before we had to hunt around to find the necessary information but now with uniform reporting in Taskstream, I know exactly where to look for things.”
Getting the work done in a more efficient manner has given OIA the time to do more outcomes mapping and analytics, making it easier to report on progress.
Implementing the new system has also given the University of Arizona an opportunity to renew their focus on assessment. People gained a refresher of the process and a greater understanding of the assessment cycle.
The university now uses Taskstream for tasks beyond assessment, including aligning program learning outcomes with institutional learning outcomes, setting up templates within Taskstream for reporting to program accreditors, and creating Higher Learning Commission reports.
With the successful launch of Taskstream by Watermark, the University of Arizona is striving to bring on more users and is encouraging departments to include program review materials in their system “workspaces.” They are also conducting one-to one-training with new users. With a more intuitive system and a renewed sense of urgency on assessment, the OIA expects an increase in departments’ system use. This will lead to more information and analytics being collected, ultimately improving the quality of assessment.
In addition, the University of Arizona is considering using the system for strategic planning and academic program review self-study reports.