I looked at 7 ways to quickly test the competencies of IT professionals that can be applied before conducting a large, lengthy and time-consuming technical interview. Then I expressed my sympathy for time-limited tests. In this article, I will cover the topic of tests in more detail.
Time-limited tests are a universal tool that is well suited for testing the knowledge and practical skills of any specialist in any profession.
So, the task - we have a stream of applicants for a vacancy, we need to quickly and easily get additional information about the skills of the candidates and their compliance with the requirements of our vacancy. We want such testing of candidates' competencies to be quick, reliable to a high degree and convenient for candidates so that they agree to be tested by us.
A good solution for this task are short tests, limited in time. It is not the moment of the start of the test that is limited, but the time during which the candidate must answer the questions. A typical example of such a test is the knowledge of the rules of the road test, which is the first stage of the test for obtaining a driver's license. You have 20 questions to answer in 20 minutes.
Some theory
I talked about the hybrid Homo sapiens decision-making model proposed by Daniel Kahneman and his colleagues. According to this concept, human behavior is controlled by two interacting decision-making systems. System 1 is fast and automatic, ensures the safety of the body and does not require significant effort to form a solution. This system learns on the basis of experience that a person receives during his life. The accuracy of the decisions of this system depends on personal experience and training, and the speed depends on the characteristics of the nervous system of the individual. System 2 is slow, requires effort and concentration. She provides us with complex reasoning and logical conclusion, her work reveals the potential of human intelligence. However, the operation of this system intensively consumes resources - energy and attention. Therefore, most of the decisions are made by System 1 - this way human behavior becomes much more effective. System 1 learns for a long time due to the efforts made by System 2, but then it gives quick automatic reactions. System 2 is a general purpose problem solver, but is slow and tiresome quickly. It is possible to βpumpβ System 2, but the limits of possible improvements are very modest and it takes a long time and requires hard efforts. System 1 βpumpingβ is in great demand in human society. When we look for a person who is experienced in something, this means that his System 1 is trained to quickly solve the problems we need.
The best way to assess the capabilities of System 1 of a particular person in a certain area of ββββknowledge, I think tests are limited in time. Once compiled, the test allows you to quickly evaluate and compare a large number of candidates. This is a tool for digitizing the control of knowledge and skills.
How to make a good test?
The purpose of a well-designed test is to determine the degree to which the candidate System 1 is trained in the knowledge and skills you need. To create such a test, you first need to decide on the topics and necessary skills, and then write questions and answer options.
So, here are my criteria for preparing a test that accurately and accurately assesses the knowledge and skills of a candidate:
- Questions and answer options should be simple. Either you know the correct answer or you don't. You should not lay in the test the need for complex reasoning and calculations.
- The test should be limited in time. You can even limit the time to think about each answer. If a candidate cannot decide on an answer within, say, 30 seconds, then much thought is unlikely to help him. Googling the right answer in 30 seconds should be hard too.
- Questions should be about practices that are really needed in the work - not abstract and theoretical, but purely practical.
- It is desirable that there are several questions for each small topic. These questions may change for different candidates (this is similar to different versions of tests at school) or all be present in a longer version of the test.
- The number of questions and the time to take the test should be tightly linked. Measure how long it takes to read the questions and answer options. Add to this time 10-20 seconds for each question - this is the time to think and choose an answer.
- It is advisable to try out the test on your employees and track their passing time in order to determine the sufficient time for candidates to pass the test.
- The scope of the test depends on the purpose of its application. For the initial assessment of competencies, in my opinion, 10-30 questions are enough with a time limit of 5-15 minutes. For a more detailed diagnosis of skills, tests for 30-45 minutes containing 50-100 questions are suitable.
For example, here is a test that I developed and used recently when selecting candidates for the position of an IT recruiter. The test was given 6 minutes, the time was controlled manually and on parole. All of the tested candidates fit within this time frame. It took me 30 minutes to complete the test.
You can take the test and at the end you can see where you made a mistake. When candidates passed this test, no errors were shown to them, we analyzed the errors later at interviews with candidates who made no more than 3 errors.
Tools
Now I create tests and surveys using Google Forms - it is a simple, convenient, versatile and free tool. However, I lack some functionality to call Google Forms a good tool for creating tests. My main complaints about Google Forms:
- There is no accounting and control of the time spent both on the entire test and on each question. This gives more information about the candidate's behavior during the test.
- Since Google Forms is not designed for tests by default, many options that are important for tests (for example, "question answer is required" and "shuffle answers") have to be "clicked" for each question - which requires time and attention. In order for each question to be asked on a separate screen, you need to create separate sections for each question, and this also leads to a large number of additional clicks.
- If you need to make a new test as a combination of fragments from several existing tests (for example, a test for a fullstack developer to collect from a part of the questions for the frontend and backend in a certain language), then you have to duplicate the questions by hand. There is no way to select and copy multiple sections or questions to another form.
Colleagues, if you know the best solutions for creating tests, please write about them in the comments.
Source: habr.com