Are your organization’s exams harder than the ones people pass to become an attorney, a CPA, or a licensed physician? Maybe they are.
But AI chatbots already passed those exams. Older versions of ChatGPT passed the bar exam, the CPA exam, and the US medical licensing exam. That doesn’t mean ChatGPT is a lawyer or a doctor, but it does show that AI can answer difficult questions.
Passing needs proof
AI’s ability to answer difficult questions creates serious challenges for organizations offering online credentialing exams and pre-hire assessments. A passing score may show that the candidate passed, but not whether they passed without AI.
Without the right technology, it’s difficult to confidently confirm whether a candidate passed an exam on their own or used AI.
So, the real question is whether your exams assess the candidate or the candidate using AI?
Bonus question: If a candidate did use AI, could you prove it?
A little help from AI can go a long way
A candidate with no subject knowledge could try to use AI on every question. The more realistic risk is a candidate who already knows enough to answer most of the exam but uses AI on a small set of questions. Those questions may be the difference between passing and failing.
AI doesn’t need to answer every question to compromise the results. If it helps on the most difficult items, the candidate’s own answers on the rest of the exam can make their attempt look normal. They may even miss questions that someone using AI for everything would get right, which can potentially make their score look less suspicious.
That makes exam misconduct harder to catch, prove, and separate from the candidate’s actual ability. The score may look valid, even though the candidate got help on the parts the exam was designed to test.
The more evidence, the merrier
Exam psychometrics gives credentialing programs a valuable way to identify scores that need review, while remote proctoring provides tangible evidence of what happened during the exam, such as device activity, video monitoring, cell phone detection, and controls that restrict unauthorized AI tools.
Can AI help with performance-based or skills assessments?
Performance-based assessments, or hands-on skills assessments, are one of the best ways to see whether candidates can actually perform the work. They’re more realistic and they make it harder to cheat because candidates need to produce something.
They’re as close as it gets to assessing whether the candidate can actually do the task. But AI tools can help with all of those. This is why we can’t have nice things.
Here are a few examples of performance-based exam tasks and how AI can help candidates through them:
- Preparing an income statement in Excel and fixing a misclassified expense.
- AI can read the trial balance, place accounts, and point to the likely error.
- Debugging a checkout error in an e-commerce app.
- AI can review the code, explain the error, and suggest what to edit and test.
- Setting up user access in a live cloud environment.
- AI can read the instructions and provide commands for permissions, roles, and access settings.
- Completing a HACCP log with prep times, temperatures, and corrective actions.
- AI can review the data and suggest the correct corrective action.
So even in a performance-based assessment, AI doesn’t have to do everything. It just has to help candidates enough to help them pass.
What AI tools are candidates using?
Candidates can use AI tools to answer questions, write responses, generate code, explain steps, and get real-time help during exams, interviews, and skills assessments.
- AI chatbots like ChatGPT, Claude, and Gemini can answer exam questions, write essays, solve problems, explain formulas, and generate code.
- Invisible AI assistants can “see” what’s on the candidate’s screen, “hear” questions they’re asked, and provide answers within a transparent overlay only the candidate can see.
- Sometimes these are called AI coding assistants or interview assistants, depending on their use case.
Interactive tutorial of how invisible AI assistants work
Honorlock's online proctoring helps organizations secure high-stakes assessments
Honorlock helps organizations make online assessments easier to trust for hiring, credentialing, internal training, and compliance testing. Using a hybrid proctoring solution that combines AI with live proctors, Honorlock blocks unauthorized AI, detects proxy candidates, protects proprietary exam content, and verifies that candidates legitimately earned the results.
Here are a few specific exam proctoring features that can help detect and prevent the use of unauthorized AI:
- Block unauthorized AI and applications, including chatbots and invisible AI assistants, while allowing approved software, such as Excel, coding environments, and industry-specific platforms.
- Detect cell phones, tablets, smartwatches, and other secondary devices.
- Verify identity and scan the workspace before the proctored exam starts.
- Monitor and record behavior and desktop activity during the exam.
- Identify keywords and phrases candidates may say aloud to activate virtual assistants, such as “Hey Siri” or “OK Google.”
- Automatically find leaked exam content on the internet and send one-click takedown requests.
