Calibrating Performance through Benchmarking (7.5 minutes)

Transcript

Thomas is the provost. Recently, he was reviewing his institution’s National Student Clearinghouse’s PDP dashboards with Sandra, the Director of Institutional Research.

He asks Sandra how their institution’s transfer-out rate compares to their peer institutions.

Sandra says, “Let’s talk about benchmarking.”  Put simply, benchmarking is the act of comparing.

If you have been in the market to buy a car and compare vehicles based on an important metric like price or gas mileage, then you’ve conducted a benchmarking project.

In the context of higher education, it is challenging to know how their experience and/or results compare to other institutions.

But if they compare their institution’s performance to their peer institutions, then they have a better understanding of their performance. In other words, benchmarking calibrates performance.

So, why should an institution benchmark their data against other institutions? There are four main reasons.

First, benchmarking can help identify their strengths and weaknesses. It helps identify areas in which they excel and areas that they don’t.

Second, benchmarking helps to set improvement goals. For example, they could set an improvement goal to increase their completion rates to 48% which is the average rate of their peer institutions.

Third, benchmarking provides evidence that improvement is possible. In this example, their peer institutions have higher completion rates than their institution, which means it is possible to perform at a higher level. So, instead of debating if higher completion rates are possible, they have evidence to share and learn from.

And fourth, benchmarking informs decision making. Higher education institutions are complex organizations, and it is challenging to know where to allocate their resources. In the case where their retention rates are high compared to their peers, but their completion rates are low, they have evidence of where they should focus their long-term attention.

The National Student Clearinghouse offers five benchmarking dashboards that allow your institution to benchmark against other PDP institutions.

• The Credit Accumulation Rate Benchmarking Dashboard provides a view of how successful students are at completing enough credits in their first academic year of enrollment to progress toward on-time completion as compared to benchmarking institutions.
• The Credit Completion Ratio Benchmarking dashboard provides a view of how successful students are at completing the credits they attempt in their first year of enrollment compared to benchmarking institutions.
• The Outcomes Benchmarking Dashboard provides a view of completion rates and other outcomes for students by cohort year as compared to benchmarking institutions.
• The Retention and Persistence Benchmarking dashboard provides a view of first-to-second-year retention and persistence rates for students who attended your institution as first-time or transfer-in students by cohort year as compared to benchmarking institutions. The dashboard displays first-to-second-year retention and persistence rates for up to eight consecutive student cohorts.
• And the Transfer benchmarking dashboard provides a view of transfer student rates and other outcomes for students, such as credentials earned before or after transfer by cohort year.

Now, let’s go back to Thomas’ question which was, “How does our transfer-out rate compare to our peers?” To answer his question, Sandra clicks on the Transfer benchmarking dashboard.

First, Sandra needs to choose the institutions they’d like to benchmark against. To do that, she clicks on “Edit” in Data Sources. Next, she clicks on “Institution Type” to open the drop-down menu. Sandra deselects “All”  then selects “Four-year” institutions. Next, Sandra clicks the double arrows to move over all four-year institutions into the “Benchmarking Institutions” box.Then she clicks “Apply Settings”.

Next, Sandra reviews the Dashboard Settings. The “Earned Credit Milestone” allows the user to filter to students with varying earned amounts of credits prior to transfer. They choose to leave it set to all.

And the “Transferred Within” filter determines the timeframe when the student transfers out. Thomas asks Sandra to leave it set at “zero to two years” which represents students who entered their institution but transferred out after less than two years.

Now, they’re ready to review the results. Scrolling down to the line chart, they note two things. First, their institution has a lower transfer out rate in comparison to their benchmarking institutions for the cohort years represented. Second, the overall transfer-out rate for theirs, and their benchmarking institutions, has declined approximately eight percentage points from 2015-16 to 2020-21.

The next chart provides information on the destination institution type. Reviewing the 2014-15 cohort, they found that 26% of their institution’s students transferred to 4-year institutions and 74% transferred to 2-year institutions within two years of entering their institution. As they scan across the cohorts, they notice that the 2020-21 cohort had a much higher percentage of students transferred to 4-year institutions which is interesting.

Looking at their benchmarking institutions across cohorts, they see that a higher percentage of students transferred to 4-year institutions while a smaller percentage transferred to 2-year institutions compared to their institution and had the same increase in the percentage of students transferring to 4-year institutions in 2020-21.

The last chart provides information on the credential earned by the students after they transferred.  Reviewing the 2014-15 cohort, they found that 73% of their institution’s students who transferred out within two years earned a credential post-transfer compared to 69% for their benchmarking institutions. But, by 2018-19, that gap had nearly closed. For that cohort, 63% earned a credential compared to 62% at their benchmarking institutions.

In summary, benchmarking provides valuable context to understand an institution’s performance. Benchmarking against peer institutions, or benchmarking an institution’s performance over time, provides evidence that their institution is improving and gives insight into areas where Improvement is needed.

Thank you for joining us.