How to Measure DEI Progress in Higher Ed

DEI training for Higher Ed

Building DEI in higher education can improve your institution’s reputation, and it’s critical to recruiting and retaining students, faculty, and staff. But it’s difficult to measure the impact of DEI initiatives, leading to inaction, focusing on the wrong efforts, or an inability to provide the right resources to those who need them most.

So, how can you measure the success of DEI efforts at your institution?

This article will show you:

Start by gathering the right data

DEI-related data comes from different sources. Some data, like student demographics, likely exist, while other data will need to be gathered.

We’ll summarize common types of data to collect for:

Student Data
Student demographics

Use admissions information to gather student demographic data such as gender, race, ethnicity, and age. And collect data about student disabilities and conditions, which can impact how online education is delivered and identifies differing needs for assistive devices.

Academic performance data

Academic performance data, correlated with demographic data, can shed light on the students or groups that are more or less likely to succeed. This information can help identify opportunities to implement new DEI initiatives that support student success.

Example:

You find that 20% of your total student population and 35% of your online students have disabilities that impact their ability to engage, interact, and learn.

After analyzing their performance, you find that online students with disabilities achieve lower grades than those who attend classes on campus.

The lower grade could be due to course curriculum, but it could also be tied to accessibility issues online students encounter that can negatively impact their performance.

DEI & institutional experience data

Without asking for it, most students don’t share the impact of DEI efforts with institutions. Conduct regular student surveys to understand experiences, opinions, ideas, and awareness of DEI initiatives.

Surveys help understand why students choose a particular university or why they decided to leave – both could be related to DEI.

Faculty & staff data
Compensation data

Compensation data should be readily available within HR and accounting systems. Gathering this information can highlight potential gaps or biases and should be part of this initial audit.

However, it’s important not to jump to conclusions based on compensation alone. When evaluating compensation differences, leaders must take into account any outlying factors tied to individual compensation packages.

Example:

A compensation analysis shows that some female instructors in the business

department aren’t compensated as well as male instructors with similar years of employment at the institution. Additionally, after digging in further, you find that Black females in particular are compensated less than White females.

Is this gap due to biases, or were other factors taken into consideration when assigning compensation?

Other factors that can impact compensation include performance, the number of classes each instructor teaches, departmental research grants and donations, and fundraising, among other factors.

Understanding this information helps determine whether gaps in compensation are due to potential bias or not. In this scenario, start by looking at individual performance reviews for the department’s female and male instructors with similar years of experience and who teach the same number of classes. Look into the other factors listed above. If all factors are equal except salary, this may point to a bias.

 

Scenario A: 12% of the female performance review scores were below average, while only 10% of male performance reviews were below average.

In this scenario, the difference between the percentage of females and males who received below-average scores is relatively small. While these scores may not be the sole reason behind the compensation gap, they could be a contributing factor.

Consider the individuals or teams behind the reviews. Are they judging individuals differently based on biased beliefs that cause lower scores for one segment?

Scenario B: 45% of White females received above-average performance reviews, and 37% of Black females received the same.

The percentages in Scenario B is significant, which could indicate potential bias against Black female instructors. Dig into this information to determine what factors may be contributing to the compensation gap. True performance is acceptable, but inherent biases against capability related to race are not.

Demographic data

Most institutions have faculty and staff demographic data available within the systems used for HR purposes.

Understanding demographic information at every level is important because an institution may have a diverse workforce as a whole but still struggle in key areas, like diversity in positions of leadership. An institution may be diverse in terms of race or ethnicity, but there could also be gaps in the number of male and female employees.

Leadership diversity data

It’s important to understand if an institution’s leadership team is truly diverse. Gather data around employee demographics and positions of leadership to identify any gaps that exist in leadership roles.

DEI training data

This information could be tricky to gather, depending on the level of past DEI training. Some institutions use the LMS to test knowledge based on information covered during training, but others may consider watching a video as training.

Regardless of type, gather any available data about DEI training, such as:
Employee experience data

Like with students, conducting surveys regularly with faculty and staff helps understand opinions, expectations, awareness, and understanding of DEI efforts. It uncovers information that might not otherwise be available and gives employees a voice in developing DEI initiatives that meet their needs.

A few ways to segment employees for surveys:
Institutional data

Aside from the data for students, faculty and staff, leaders should also look to gather institutional information that can be used to improve DEI efforts.

Budget & financial impact

Data from operational and institutional efforts can help teams understand the financial impact of DEI initiatives, the gaps that exist, and the allocated budget.

Spending for DEI initiatives

Gather data about the average spend on DEI initiatives each year.

DEI’s impact on recruiting and retention

Example: DEI may improve employee engagement, which can lead to better retention rates. How much does this save in reduced turnover and new employee training? Evaluate and measure every aspect of improvement DEI creates.

Recruitment & retention data

Look at the percentage of diverse candidates and how they progress through different levels in the hiring process.

Next, determine which levels need to improve and which are in good standing. Set realistic goals to maintain and improve these levels, starting with the lower levels.

Employee retention

The examples below are a brief look at data to gather about the retention of diverse employees and leaders. Are there any areas of employee retention that are weaker than others? If yes, dig into why.

Number/percent of diversity:

Find strengths & weaknesses

After gathering the data you need data, look at your strengths and weaknesses.

Set goals

Once a bigger picture of DEI initiatives emerges, the next step is to set goals that outline where the institution should be and the potential impact of achieving this desired state. It’s important to be specific and realistic about goals, milestones, and timelines.

Prioritize steps to maintain & improve

After identifying what’s working and what needs improvement, focus on the areas of opportunity. How can these be addressed in order to meet the DEI goals?

Data example for online students

For this example, assume that the data indicates two things about online students:

  1. 85% feel welcomed and included because of the student orientation

  2. 35% believe that the online course content doesn’t isn’t accessible
What are the next steps?

Overall, the student orientation is in good standing, but real priority is improving the accessibility of online course content. You’ll see in the steps below that efforts for each can benefit the other.

Prioritize the accessibility of course content

1. Make quick accessibility updates while planning longer-term efforts. LMS admins can make changes to global LMS settings that impact the online course experience for a large number of students, such as:

2. Define longer-term efforts

While the quicker changes are being implemented, prepare longer-term efforts, such as:

3. Use student surveys about the orientation to serve a dual purpose

While 85% of students indicated that the orientation process made them feel welcomed and included, there are likely still some ways to improve. Adding questions about accessibility to the surveys can help students see that their well-being and needs are important, right from the beginning of their relationship with the institution.

Surveys serve two purposes:

DEI measurement in action

Measurement goes beyond just analyzing the impact of DEI initiatives. Collecting and analyzing the right data provides valuable insight to help refine and improve initiatives that directly impact reputation, recruitment, retention, resources, and revenue – the 5 Rs.

Ultimately, allowing institutions to deliver an exceptional educational experience for students and an elevated teaching experience for instructors.

Read our recent blog if you’re wondering what to expect from educational technology and online learning in the near future.

2023 Higher Ed Online Learning Trends

Sign up below to receive more resources for tips, best practices, white papers, and industry trends
Want to see Honorlock in action? Schedule a demo

How to Adopt DEI Best Practices in Online Learning | Honorlock session | 2023 ICAI Annual Conference

A recent survey of chief online officers published by Quality Matters predicted that by 2025, programs that blended on-campus and online learning will become the norm for students. As institutions rapidly adapt to meet these demands, they are met with a challenging question: how will they ensure that they are creating diverse, equitable, and inclusive learning environments for their students?

Hear from Elisabeth Greenwood, Instructional Technology Specialist and Chair of the Academic Affairs DEI task force at the University of Central Florida, and Jordan Adair, VP of Product at Honorlock, as they discuss:

  • How to overcome the challenges of establishing and using DEI in online learning
  • Ways to use DEI in online courses to improve student engagement
  • Steps to create equitable learning environments

Speakers

Elisabeth Greenwood

Instructional Technology Specialist and Chair of the Academic Affairs DEI task force at the University of Central Florida

Jordan Adair

VP of Product at Honorlock

Webinar: Overcoming Biases & Building Inclusion

A group of diverse people sitting in chairs in a circle, having conversation.

Whether we realize it or not, biases exist that can lead to unfair assumptions about others that have tangible negative consequences.

Join the webinar to learn how higher ed institutions and organizations can address and overcome implicit bias to develop inclusion.

This webinar will show you how to:

  • Recognize the types of biases that exist
  • Realize the negative impact biases can have on culture, reputation, and members of the institution
  • Acknowledge biases, create awareness, and take steps to address them
  • Understand the true value and importance of inclusion in DEI efforts

Speakers

Bethany Adams from Villanova

Bethany Adams

MA, SHRM-SCP, Villanova University

Bethany Adams has more than 15 years of experience in HR and education. She began her career working in training & development and HR in the restaurant & retail industries. She has experience in both small, privately-owned businesses and large, Fortune 500 organizations. Bethany holds a BA in Psychology, MA in Organizational Management, SHRM-SCP certification, and is a Certified Genos International Emotional Intelligence Practitioner.

Bethany teaches at Villanova University in the Graduate Programs in Human Resource Development. She has spoken at a wide variety of HR & education conferences throughout the country and leads corporate trainings on topics including strategic HR, design thinking, DEI, and emotional intelligence. Bethany also hosts an HR podcast called HR Tea.

Kyle Ali, instructor at Villanova and Google employee

Kyle Ali

Education for Social Impact Senior Program Manager, Google Adjunct Professor, Villanova University

Kyle leads initiatives to build a diverse pipeline of future computer science professionals through unique educational programs at Google. In previous roles at Google, Kyle played a significant role shaping team cultures and strategy and led large scale initiatives, including a global effort to redefine organizational values and other projects focused on identifying and engaging diverse talent.

Kyle is also an Adjunct Professor at Villanova University and currently teaches “Diversity in a Global Economy” to graduate students.

Kyle earned undergraduate degrees in Journalism and Political Science from the University of Missouri, an MBA from Baylor University, and a Professional Certificate in Negotiation from Northwestern University.

Online proctoring improves the testing experience

Using online proctoring to improve the online assessment experience

Online exams can be stressful for. And online proctoring can amplify that stress if it isn’t used correctly. Instead of focusing on the exam itself, the wrong approach to proctoring can lead to more questions and concerns that can negatively impact their performance.

Catching cheating vs improving the test-taking experience

Online proctoring is just a way to catch cheating, right? It shouldn’t be. 

Catching cheating is the focus for many online proctoring companies, but Honorlock goes far beyond that. Honorlock takes a different approach to online proctoring that focuses on improving the testing experience while still protecting academic integrity. 

Simply put, Honorlock makes things easy for test takers and it just works. 

How does Honorlock improve the testing experience?

Quick summary of how Honorlock works:

  • Honorlock’s blended proctoring solution combines AI with human proctors.
  • AI monitors exams and alerts a human proctor if it detects potentially problematic behavior.
  • The proctor can review test taker’s behavior in an analysis window and decide whether to intervene and redirect the them or not.

Quick summary of how Honorlock works:

Honorlock allows proctored exams to be taken 24/7/365 and there’s no fee for canceling the exam or for exams that take longer to complete. 

Unfortunately, some companies charge exam takers if they cancel an exam within a certain time frame before it starts. Even worse, in some cases, exam takers have to pay for exams by the hour. This means paying more money for longer exams.

No online proctoring software to download

Honorlock’s online proctoring software is browser-based, which means there’s no software to download to take the exam. 

Easy to set up & no extra logins or passwords required

Honorlock’s online proctoring platform integrates directly with your LMS, such as Blackboard (and Blackboard Ultra), Canvas, and D2L, which is important because test takers won’t have to create extra profiles that require extra logins and passwords, and they can get support with without leaving the exam window

Some online proctoring platforms have clunky LMS integrations, which creates a poor experience and they’ll probably have to create additional profiles on different platforms.

Works with poor internet connection

Test takers won’t need to worry about their internet connection during a proctored exam with Honorlock. If the internet connection is slow or if there’s an outage, our software works normally except it captures still images of the webcam and the screen instead of a video recording. 

On the other hand, some other proctoring services charge a fee if a test taker has an issue with their internet connection, 

Detects voices instead of unimportant sounds

The terms “sound detection” and “voice detection” are often used interchangeably, but they’re different.

Sound detection triggers for basically any sounds during an exam, like sneezing or a doorbell ringing. 

Honorlock’s Voice Detection listens for specific keywords and phrases, such as “Hey Siri” or “OK Google.” This helps reduce the number of false flags that exam administrators have to review, and test takers won’t be interrupted for sneezing or talking.

Provides accommodations

Accommodations can mean many different things, but here are some that are commonly used:

Honorlock proctors are trained to provide support during times of stress

A nationally certified counselor and educator trained Honorlock’s online proctoring team to identify signs of stress and provide support during those moments.

This human-centric communication helps test takers gain confidence in the process and with our test proctors.

Honorlock is proven to reduce test anxiety

The University of North Alabama and Honorlock conducted a test anxiety survey to better understand the causes of test anxiety and how online proctoring impacts it.

The test takers were surveyed before and after taking assessments proctored by Honorlock. The survey found that every test taker who interacted with an Honorlock proctor had a significant decrease in anxiety for future exams. 

“The proctor popping in was different than I expected – in a positive way. I imagined them being more strict. I felt that the proctor was helpful and a lot less intimidating than I thought.” – quote from a post-exam survey interview

Honorlock is proof that when online proctoring is done the right way, it’s a resource that supports test takers, promotes academic integrity, and protects the reputation of your programs.

Schedule a Demo