Generative AI (GenAI) has been making waves across various functions. In a world where assessments play a pivotal role in shaping HR decisions, the rise of GenAI introduces a new dynamic: a double-edged sword of innovation and challenge. 

As businesses harness the power of GenAI to enhance hiring efficiency and evaluate employee potential, the risk of compromising assessment objectivity looms large. According to research conducted by Bryq, approximately 1/3 of applicants have started using GenAI in the hiring process. With this in mind, the ease of using a trusted 'AI mentor' could open the door to cheating, raising concerns about the reliability of evaluations. 

Detecting cheating with GenAI in assessments is challenging due to its ability to generate human-like responses, adapt to detection methods, understand natural language, exhibit contextual awareness, and behave stealthily. As we navigate this evolving landscape, it's crucial to strike a balance between the benefits of GenAI and maintaining the integrity of assessment processes.

The Complexity of Cheating Detection

Identifying instances of cheating in online assessments is a complex task, given the unique challenges posed by the digital environment. In the age of rapidly evolving technologies, such as those associated with GenAI, traditional detection methods increasingly struggle to keep pace with such adaptability and innovation.

Moreover, the prevalence of a diverse array of devices, including smartphones and smartwatches, provides individuals with easy access to potentially prohibited communication channels. 

While one approach to mitigate the risk of compromising result integrity might involve enhancing proctoring measures, this solution is not without its shortcomings. The prospect of stricter proctoring raises valid concerns related to privacy and ethics for the candidates involved.

The Value of Soft Skills in Combating AI Challenges

The challenges that we mentioned mainly come from tests that measure knowledge, the kind where you need to give exact answers to specific problems—what we call hard skills assessments. These kinds of tests can be a bit tricky because they open the door for tools like GenAI to provide accurate responses, possibly messing with the fairness of scores.

On the flip side, recent research tells us that it's not all about hard skills anymore. Soft skills, like being adaptable or communicating well, are turning out to be super important as job roles get broader. Think qualities like open-mindedness, flexibility, and understanding others – they really matter in today's work landscape.

Soft skills go beyond technical abilities; they're about how people relate to themselves and others. Here's the twist: unlike hard skills, soft skills don't have a clear right or wrong answer. That makes it tricky for tools like GenAI because they're not so good at figuring out the best behavioral tendencies for a particular role. 

As Manto Papagianni, Head of Product at Bryq, recently explained in an interview conducted by AMS, “Assessments incorporating personality questions inherently resist manipulation since personality traits lack absolute right or wrong answers, and the alignment between one's personality and job prerequisites can't be directed by GenAI.”

Therefore, when evaluating candidates for diverse roles, especially those involving interactions with people, it's evident that soft skills are crucial.

Bryq's Methodology and Approach

Unveiling the true extent of AI utilization poses a challenge, even for top GenAI companies. Rather than relying on easily manipulated datasets like resumes, a more effective approach involves implementing a meticulously structured assessment and adopting proactive measures to prevent cheating. This is how the Bryq Assessment is safeguarded:

Proctoring and Timed Questions 

Webcam monitoring and timed questions discourage external assistance, as candidates face skill-level challenges within set time constraints.

Calculator Use

Candidates are allowed calculators, promoting the use of helpful tools for optimal performance.

Controlled Mode of Administration 

Registration is required for assessment access, ensuring a controlled and monitored environment.

Content Interchangeability 

Diverse question types, including photos, videos, images, puzzles, and equations, add complexity, making cheating more challenging.

The Importance of a Scientifically Informed Approach

Industrial-Organizational (I-O) Psychologists play a pivotal role in optimizing the design and interpretation of AI-powered assessments. I-O Psychology, a discipline that applies psychological principles to address organizational behavioral issues, proves instrumental in ensuring the efficacy of these assessments.

In the realm of AI-powered assessments, I-O Psychologists offer essential expertise in the psychometric principles governing the AI tools. This expertise is harnessed to guarantee the reliability and validity of the assessments. Additionally, their input extends to the design phase, where considerations such as the clarity of instructions and overall user experience are paramount. By prioritizing these factors, I-O’s enhance user acceptance and engagement in the assessment process.

A distinctive strength of I-O Psychologists lies in their proficiency in delivering constructive feedback. This skill is leveraged to shape the design of AI-powered assessments by incorporating effective feedback mechanisms. These mechanisms not only aid individuals in comprehending their strengths and weaknesses but also align with broader organizational and individual development goals.

Furthermore, they contribute to the ongoing enhancement of AI assessments by analyzing data on assessment performance and recommending adjustments. This adaptability ensures that assessments remain relevant and effective amid the evolving needs and contexts of organizations. Ultimately, the multifaceted contributions of I-O’s are integral to optimizing the utility and impact of AI-powered assessments in organizational settings.

Building on this framework, Bryq incorporates the expertise of its in-house team of I-O Psychologists, who are integral to the development and refinement of its assessments. These professionals apply their deep understanding of psychometric principles and organizational behavior to enhance the efficacy and user experience of Bryq's tools.

Conclusion

In wrapping up, it's clear that GenAI is reshaping HR assessments, offering exciting innovations but also posing unique challenges, especially with a significant portion of applicants now using it in hiring. The real test lies in skillfully detecting and managing GenAI-assisted cheating, which demands a well-thought-out approach that values both the advantages of GenAI and the integrity of the assessment process.

The emerging emphasis on soft skills in response to GenAI's impact on hard skills assessments is noteworthy. Bryq's methodology, which concentrates on personality and cognitive abilities, adeptly navigates this landscape, incorporating strong anti-cheating measures. Crucially, the role of I-O Psychologists can't be overstated in this context. Their insights ensure that AI-powered assessments are not only reliable but also continuously adapted to the evolving HR environment.

Ultimately, this calls for a nuanced, flexible approach to integrating GenAI into HR practices, striking a balance that is both ethical and effective.