Can multimedia assessments improve the quality of selection?

Can multimedia assessments improve the quality of selection?
Promoted by Can multimedia assessments improve the quality of selection?

Ali Shalfrooshan

Ali Shalfrooshan

Managing R&D Consultant

In 2016, the Government Recruitment Service (GRS) made the decision to improve the provision of psychometric tests for the Civil Service. The goal was to improve the quality of selection decisions across the Civil Service through more efficient sifting of applications, and to support their ambition to become the most inclusive employer in the UK by 2020.

Following a Civil Service wide consultation, GRS identified that a completely bespoke item-banked multimedia assessment, applicable to all departments across six grades of seniority, would be needed to drive higher quality recruitment. To help them with this ambitious goal they identified a supplier, PSI, to support them. The following article covers the main challenges of the project, what we did to deliver the assessment and some of the outcomes.

The challenges

  1. Deliver a new engaging assessment and platform in a short timescale: Due to the complex and unique nature of the technological and accessibility requirements, an entirely new online assessment platform needed to be developed within 4 to 5 months.

  2. Inclusive and accessible to all candidates: The assessment platform needed to have a mobile-first design, a practice site to help reduce candidate anxiety, and to be fair to all candidates. The Civil Service formally committed itself to the Disability Confident Scheme, therefore it was critical that the platform met best practice accessibility standards.

  3. Enhance efficiencies and reduce costs: A key goal and challenge of the project was to enhance existing processes to make them more efficient and more cost effective.

The solution

For the test to be accurate across departments and levels of seniority, the relevant situations, tasks and context needed to be identified. To achieve this, 60 deep-dive interviews with stakeholders were carried out and over 200 job descriptions were reviewed. From this, a set of 839 tasks and 364 critical incidents were identified. Based on the job analysis, PSI consultants generated over 3,078 questions for the new assessment.

To ensure fairness and relevance, the test content was reviewed by 74 internal experts, 42% of which were from diversity networks or units. Based on this review, certain questions were either not used or updated based on this feedback. In addition, to ensure that the software was accessible to all users, the Digital Accessibility Centre (DAC) was commissioned to test the software and confirm it met best practice standards.

The test also needed to be engaging. Therefore, a significant effort was made in the production phase to ensure that all of the videos produced were realistic and immersive. Over 1,500 applications were received from a diverse range of actors, from which 57 of the best were cast covering 77 roles. To ensure the production was as cost effective and efficient as possible, it was filmed at a studio specialising in green screen with 67 videos depicting work-related scenarios. Draft versions of the test were trialled with over 4,000 Civil Servants, resulting in a total of 6,371 test completions. An additional 10,000 triallists took part to ensure the tests created were as accurate, fair and robust as possible.

The results

The mobile-first multimedia assessment was launched in February 2017. Since the launch it has been completed by over 200,000 applicants and the practice site by over 100,000 job seekers. The test has helped fill over 1,200 job role vacancies from administrative to leadership roles.

  1. Deliver a new engaging assessment and platform: The test is an innovative step forward in assessment, providing an engaging applicant experience. Every activity was delivered in a short timescale and the success is reflected in the significant adoption of the assessment by 41 departments across the Civil Service.

  2. Inclusive and accessible to all candidates: Based on the analysis of a sample of 58,295 candidates the test has been deemed fair at the test’s pass mark and does not cause adverse impact for applicants with disabilities, BAME and LGBT status. Since the test’s deployment, the number of requests for adjustments by disabled test takers has reduced by an incredible 80%.

  3. Enhance efficiencies and reduce costs: Sir Jeremy Heywood commented “The Civil Service is undergoing big changes in the way it works, to become more efficient and effective in the delivery of services, while responding to some of the biggest challenges in its history”. This project is a perfect example of delivering against these goals. In just over a year of usage, the Civil Service estimated that the implementation of the test has saved over half a million pounds in resources needed to manually sift job applications.

The project involved significant stakeholder involvement with contributors from 27 professions and 98 departments/agencies. Arguably this collaborative approach between GRS and the test designers was the reason the project was a success. Overall, being able to have such a positive impact on such an important institution was a true honour, especially during such an important time for the UK.


Be the first to comment.

Related Content

Lessons from lastminute.com on getting through disruption with comms & wellbeing
HR Grapevine
HR Grapevine | Executive Grapevine International Ltd

'Better prepared' | Lessons from lastminute.com on getting through disruption with comms & wellbeing

Feature
7 mins read
How Monzo is using shared documents to personalise work for staff
HR Grapevine
HR Grapevine | Executive Grapevine International Ltd

‘Working with me’ | How Monzo is using shared documents to personalise work for staff

News
4 mins read
How HR can support working parents as school returns
HR Grapevine
HR Grapevine | Executive Grapevine International Ltd

PODCAST | How HR can support working parents as school returns

Podcast
11 mins
How to trial flexibility and 'employee autonomy' in your business
HR Grapevine
HR Grapevine | Executive Grapevine International Ltd

Gousto's Head of People | How to trial flexibility and 'employee autonomy' in your business

Video
Watch Now
Reflections on the Great Resignation: Where do we go next?
Clear Review
HR Grapevine
HR Grapevine | Executive Grapevine International Ltd

Reflections on the Great Resignation: Where do we go next?

Webinar
Register Now
2021-2022 Executive Survey: Winning the war for talent
Ceridian
HR Grapevine
HR Grapevine | Executive Grapevine International Ltd

2021-2022 Executive Survey: Winning the war for talent

Learning Resource
Download Now