At Southern Connecticut State University, we try to think of data in terms of educational intelligence…meaning, taking a more holistic view of data collection so that we can connect data points that will provide meaningful insights into how we can improve institutional effectiveness and student outcomes.
How Do We Measure Student Success?
Metrics reflect our cognitive biases. If student success is seen through the lens of future employment, then the most suitable metrics that measure whether students are prepared to enter the creative class in their future places of employment. And whether they are able to work autonomously and handle cognitive complexity.
If student success is seen through the lens of young adult development, then the ability to navigate in a complex society with many choices is an appropriate metric. So, too, is the extent to which students encounter failure and persevere and/or seek guidance before the issue escalates beyond recovery.
If earned income is equated with success, then wages in first job and alumni default on student loans are appropriate metrics. Consider, for example, the Wall Street Journal’s method of ranking colleges. Melissa Korn (September 29, 2016) explains this method: “Outcome scores are derived from graduation rates and academic reputation, as well as measures of loan-repayment rates and graduate salaries.”
If student success is considered through the lens of liberal education, then human flourishing, as evidenced by the vigor of students’ problem solving and competency in dealing with complexity, is an essential metric. The competency to deal with complexity rests upon both learning and development. This competency emerges when students have had developmental experiences that demonstrated to them that they are, indeed, able to deal with complicated social situations and constant change.
Ben-Avie et al. (2003) describe a study that uncovered links between social competence in tough situations and achievement in math. The strength of the relationship between students’ social knowledge of themselves and others and their achievement in mathematics was found to be strong (r = .80; p = .04). This makes sense. In both math and social interactions, success depends on awareness of the challenge and of an ideal outcome, skills to map out a strategy to solve the problem, and willingness and skill in persisting at and refining the strategy until a positive outcome is achieved.
The competency to handle cognitive complexity and persist until a positive outcome is achieved is more likely among students who have:
- been exposed to many kinds of experiences
- been encouraged to see the world metaphorically as well as concretely
- the ability to create flexible internal images
- the linguistic skills to share them
Longitudinal Studies to Collect Data
To examine these ideas, all first-time, full-time undergraduate students at our university are included in longitudinal, cohort studies. The students are followed from New Student Orientation through graduation from the university, or subsequent enrollment in other colleges and universities. As each incoming class enters the university, a cohort dataset is established. A cohort dataset initially contains such demographic information from Banner as high school rank, high school GPA, SAT scores, gender, ethnicity, residential status, registration with Veterans Services and the Disability Resource Center.
Each year, new data are added, including earned credits, cumulative GPA, registration status, and scores on surveys and direct assessments. Students’ ID numbers are used to link their demographic characteristics with their scores from surveys and assessments to create comprehensive cohort datasets. The university uses Watermark with all the certification programs on campus and participates in the Multi-State Collaborative to Advance Quality Student Learning, a nation-wide initiative that uses Watermark as its technical partner.
Our Findings
The ten-year study demonstrated that psychological-educational factors that were important predictors of first-year students’ retention and academic performance were also important when they were upperclassmen. Measures of students’ crystallized knowledge prior to college enrollment were not found to predict students’ subsequent performance in college and graduation outcomes.
The study also found that GPA and sense of belonging were the main predictors of persistence—regardless of ethnicity or gender. While psychological-educational factors are harder to measure than alumni default on student loans, for example, these factors are directly related to teaching and learning in the classroom. In particular, longitudinal studies that focus on that which is amenable to change yield more useful and actionable data than studies that focus on students’ demographic characteristics and crystallized learning, which cannot be changed.
I see assessment as a scholarly endeavor that aims to improve the underlying operating system of the university so that it more effectively promotes students’ learning and development. In well-functioning universities, the community comes together to resolve long-term problems and engage in global planning to improve learning and developmental outcomes among the students. A culture of shared accountability permeates the community; thus faculty and staff feel safe to take risks trying out new ideas.
The post How SCSU Uses Data to Improve Student Outcomes appeared first on Watermark.
