Hello Outcomes Assessment Projects (formerly Aqua) users,
We now offer far more flexibility in setting up and managing multiple scores, a “N/A” option for scoring, and an enhanced rubric editor. Read on.
You now have much more flexibility and control over how and when you collect multiple scores.
With the release of Outcomes Assessment Projects (formerly Aqua) late last year, we provided a very simple option for institutions that wanted to get more than one score on each submission of student work. You could select anywhere from one to four scores during project setup. We knew the approach was limiting, but we didn’t consider a single student submission as “complete” or include its scores in the outcome performance reports until it had received the requested number of scores. Well, we listened and learned from those of you who have run your assessment projects in Outcomes Assessment Projects (formerly Aqua) and sought to improve this capability with maximum flexibility and minimum complexity.
Now, each project is created with a default setting of one score for each student submission to be evaluated. But the “Scoring Settings” tab has three powerful options for users with the Assessment Coordinator role:
- You can add a second (or third, or fourth) score on your submissions whenever you want. Really, whenever you want. It could be before scoring opens, while scoring is taking place, or four months later.
- You can start and stop scoring on a specific scoring round whenever you need to.
- You can set a random sample (0-100%) as the target for completion for the second and subsequent scores.
Collectively, these settings open up some nice options to allow you to respond to shifting resources or adjust scoring goals as you go along. If your scorers loved scoring their first set of papers so much they are willing to do more, you can add a second or third score mid-project.
If you just need a random sample of 35% scored a second time so you can check inter-rater reliability, that is easy to do as well. And with the ability to add and remove evaluators once a project is open, you can limit the second or third score to a select group of evaluators.
Now that you can start and stop scoring when you need to, you can pause scoring to re-calibrate your evaluators. You can also stop scoring new artifacts entirely and focus on getting a second score on what has been completed so far.
Reports have been updated throughout to provide clearer visibility into the scoring progress.
N/A is … Now/Available!
You can now enable an “N/A” option to let evaluators mark specific criteria as “not applicable” to the student work. This is a project-level setting that will apply to any of the rubrics in that project.
When enabled, N/A will show up as a scoring option for each criterion. When an evaluator selects “N/A” for a criterion, Outcomes Assessment Projects (formerly Aqua) captures that “N/A” but excludes it from any calculations of outcome performance. We updated all applicable reports to show usage of N/A by evaluators.
The Outcomes Assessment Projects (formerly Aqua) rubric editor is much improved!
The updates to the rubric editor in Outcomes Assessment Projects (formerly Aqua) make it much easier to customize the rubric and get it live quickly with your desired format. Begin by choosing a rubric format that optimizes the setup for more detailed rubrics with performance descriptors for each cell, or a minimal format that is ideal for simple “meets/does not meet” type rating scales.
The Outcomes Assessment Projects (formerly Aqua) rubric editor and supported scoring formats still have fewer options than our LAT & AMS rubrics, but you can start the scale at zero or one, add up to five performance levels (before adding N/A), and change the order.
We look forward to hearing your feedback on these updates and bringing you more exciting changes this fall!
– The Taskstream Product Team
The post "New Scoring Settings in Outcomes Assessment Projects (formerly Aqua) Provide Even More Flexibility" appeared first on Watermark.
