A load off the University of Helsinki’s mind
This spring, the Faculty of Educational Sciences at the University of Helsinki was considering holding their entrance examination interviews online. They knew the possible pitfalls that could come with creating a new mass exam system without proper testing.
In order to be able to use the online entrance examination system, they needed to be absolutely certain that their system had the capacity to allow all 5000 prospective students to login and record their video responses during a single 30-minute window. There was no room for error – their system simply had to succeed.
If the University of Helsinki’s entrance exam system failed, any number of students would have missed out on a possible study place in their chosen programme which not only would have been unfair for the applicant but also might have had legal implications.
Naturally the system was tested with traditional methods where the expected load is simulated in different parts of the system. The results that this simulation gives like HTTP requests per second translates poorly to how many actual users a system is able to cope with. As modern systems are complex and consist of different components, you need to test systems as a whole. So the University was seeking a solution that would help them verify their entrance exam system to ensure it could handle all the applicants on exam date.
Supervisor became the obvious choice as it was the only solution that was able to interact with the platform just like real users. It also gave a definite answer whether all the applicants would be able to complete the exam.
“The methodology behind Supervisor is truly one of a kind. It’s scalable and real and it’s the only load testing tool that also measures the user experience” says Jaakko Kurhila, CDO, University of Helsinki
The testing process itself was simple, safe and required no setup.
In each test run Supervisor sent 5000 users over a period of 30 minutes, exactly as the users are entering the system at the exam day – each using an unique email address – to login to the university’s entrance exam system.
Each user interacted with the examination platform like a real user would - first testing that their webcam and audio were functioning, then clicking through the questions and recording a response to each question and submitting their answers.
After each run the results were shared with the team which in turn worked on fixing issues on scaling essentially enabling the University of Helsinki was able to rehearse the entrance exam interview three weeks prior to the actual date, allowing them to decide weather to go with the online examination system or not and avert any crises on the actual exam date.