Hiring IT specialists is a challenging task. Firstly, there is currently a shortage of experienced professionals in the market, and they are aware of it. Candidates often are not willing to spend much time on an employer's 'selection activities' unless they are intrigued beforehand. The previously popular practice of 'giving a test lasting 8+ hours' no longer works. For an initial assessment of knowledge and filtering candidates before conducting a full-scale technical interview, faster methods must be utilized. Secondly, to effectively evaluate knowledge and skills, one must either possess such skills themselves or involve a colleague who has them. These difficulties can be addressed using methods that I will discuss in this article. I personally apply these methods and have created a sort of ranking for myself.
So, here are my top 7 quick ways to assess the competencies of IT specialists before the interview:
7. Review the candidate's portfolio, code samples, and open repositories.
6. A short timed test task (completed in 30-60 minutes).
5. A brief express interview about skills via phone/Skype (like a questionnaire, just online and verbal).
4. Live Doing (Coding) – solving a simple task in real time with a shared screen.
3. Questionnaires with open-ended questions about experience.
2. Short tests with multiple-choice answers, limited by time to complete.
1. A multi-stage test task, the first stage completed before the interview.
Next, I will detail these methods, their advantages and disadvantages, and the situations in which I use each method for quick competency assessment of programmers.
In the previous article on the hiring funnel I conducted a survey among readers about quick ways to assess the skills of IT specialists. In this article, I share the methods that I personally like, why I like them, and how I apply them. I start with the first place and finish with the seventh.
1. A multi-stage test task, the first stage completed before the interview.
I believe this method of assessing developers' competencies is the best. Unlike traditional test tasks where you say, 'Here’s the task and go do it,' in my version, the process of completing the test task is broken down into stages – discussing and understanding the task, designing the solution and estimating required resources, several stages of implementing the solution, documentation, and acceptance of the solution. This approach is closer to modern software development technology than just 'take it and do it.' Details follow.
In what cases do I use this method?
For my projects, I usually hire remote employees who develop a separate, autonomous, and relatively independent part of the project. This reduces the need for communication between employees, often to zero. Employees communicate not with each other but with the project manager. Therefore, it is essential for me to immediately assess a person's ability to quickly understand the task, ask clarifying questions, independently devise an action plan for solving the task, estimate the necessary resources and time. The multi-stage test task helps me well in this.
How to implement it
We identify and formulate an independent and original task related to the project that the developer will work on. I usually describe a simplified prototype of the main task or future product as the assignment, which will require the developer to face the main problems and technologies of the project.
The first stage of the test task is familiarization with the task, clarifying any unclear points, designing the solution, planning steps for solving the task, and estimating the time for completing individual steps and the entire test task. As a result, I expect a document of 1-2 pages outlining the developer's action plan and time estimates. I also ask candidates to specify which stage they would like to fully implement to demonstrate their skills in practice. No programming is required at this stage.
This task (the same one) is given to several candidates. It is expected that responses from the candidates will arrive the following day. Then, within 2-3 days, after all responses are received, we analyze what the candidates have sent us and the clarifying questions they asked before starting the task. Based on this information, you can invite any number of candidates needed to the next stage.
The next stage is a short interview. We already have something to talk about. The candidate has a rough idea of the subject area of the project they will be working on. The main goal of this interview is to answer the candidate's technical questions and motivate them to complete the main test assignment – programming the part of the task they have chosen themselves. Or the part whose implementation you want to see.
It is always very interesting to observe which part of the assignment the developer wants to implement. Some prefer to lay out the project structure, decomposing the solution into modules and classes, that is, they move from top to bottom. Others isolate a separate subtask that they consider most important without outlining the overall solution. In other words, they go from bottom to top – from the most complex subtask to the complete solution.
Benefits
We can see the candidate's erudition, the applicability of their knowledge to our project, and the development of their communication skills. It is also easy for us to compare candidates with each other. Usually, I discard those candidates who gave overly optimistic and overly pessimistic estimates of the time required to complete the task. Of course, I have my own time estimate. A low estimate from a candidate likely indicates that the person has not fully grasped the task and performed this test superficially. An excessively high time estimate usually indicates that the candidate has a poor understanding of the subject area and lacks experience in the necessary topics. I do not immediately disqualify candidates based on their estimates; rather, I ask them to justify their estimate if it has not already been sufficiently motivated.
Some may find this method complicated and time-consuming. My estimate for the effort involved in using this method is as follows: it takes about 30-60 minutes to describe the test task, and then 15-20 minutes to check each candidate's response. Candidates usually spend no more than 1-2 hours on such a test task, immersing themselves in the essence of the problems they will need to solve later. At this stage, a candidate may lose interest and withdraw from communicating with you, having spent a little time.
Disadvantages
First, you need to come up with an original, isolated, and concise test task, which is not always possible. Second, not all candidates immediately understand that programming is not required at the first stage. Some jump straight into programming and disappear for several days, then submit a fully completed test task. Formally, they did not complete this test task correctly because they did not do what was required of them. But at the same time, they did manage to provide an adequate solution to the entire test task. To avoid such situations, I usually call all candidates who received the task two days after it was issued to find out how they are doing.
2. Short tests with answer options, limited by time
I don't often use this method, although I really like it and consider it one of the best ways to quickly check competencies. I will write a separate article about this method soon. Such tests are widely used in various fields of knowledge. The most vivid and typical example is the theoretical exam for obtaining a driver's license. In Russia, this exam consists of 20 questions that must be answered in 20 minutes. One mistake is allowed. With two mistakes, you need to correctly answer 10 additional questions without error. This method can be easily automated.
Unfortunately, I have not encountered good implementations of such tests for programmers. If you know of good ready-made implementations of such tests for programmers, please write in the comments.
How to implement it
I have worked with self-implemented tests by employers when fulfilling orders as a recruiting outsourcer. It is quite possible to implement such a test, for example, using Google Forms. The main problem lies in formulating questions and answer options. Typically, employers' creativity lasts for about 10 questions. Unfortunately, it is not possible to implement question rotation from a pool and time restrictions in Google Forms. If you know a good online tool for creating your tests where you can limit the test duration and organize the selection of different questions for different candidates, please share such services in the comments.
In what cases do I use this method?
Currently, I use this method at the request of employers if they have ready-made tests that can be given to candidates. It is also possible to combine such tests with the fourth method from my ranking – asking the candidate to share their screen and take the test. This way, you can discuss questions and answer options with them.
Benefits
With a good implementation, this method is autonomous. The candidate can choose a convenient time to take the test, and you do not need to spend much of your time.
Disadvantages
High-quality implementation of this method is quite costly, and it is not very convenient for a small company that occasionally hires new employees.
3. Questionnaires with open-ended questions about experience
This is a set of open-ended questions that encourage the candidate to reflect based on their experience. At the same time, we do not offer answer options. Open-ended questions are those that cannot be answered simply and unambiguously. For example, recall the hardest task you solved with the framework of a certain type? What was the main difficulty for you? Such questions cannot be answered simply. In fact, the only simple answer is – I don't have such experience; I haven't worked with this tool.
How to implement it
Easily implemented using Google Forms. The main thing is to come up with questions. I use several standard constructs.
Tell us about the last project you worked on using XXX, what was the most difficult aspect of this project for you?
What are the main advantages of the XXX technology for you? Please provide examples from your experience.
Having chosen the XXX technology, what other alternatives did you consider and why did you specifically choose XXX?In what situations would you prefer technology AAA over BBB?
Tell us about the most complex task you solved using XXX, what was the main difficulty?
Accordingly, these constructs can be applied to many technologies in your tech stack. Answering such questions isn't easy with template phrases from the internet, as they are personal and based on individual experiences. When responding to these questions, candidates usually keep in mind that any of their answers during the interview may lead to follow-up questions. Therefore, if they lack experience, candidates often remove themselves from consideration, realizing that further discussion may be pointless.
In what cases do I use this method?
When working with orders for selecting specialists, if the client hasn't proposed their own method for initial competence verification, I use this method. I already have prepared questionnaires on several topics, and it's easy for me to apply this method for a new client.
Benefits
It's not difficult to implement using Google Forms. Moreover, a new survey can be created based on the previous one by simply changing the names of technologies and tools. For instance, a survey on experience with React will be quite similar to a survey on experience with Angular.
Creating such a questionnaire takes about 15-20 minutes, and candidates usually spend 15-30 minutes answering. The time investment is minimal, but we obtain information about the candidate's personal experience, which allows us to make each interview unique and more interesting. Typically, the duration of the interview after such a questionnaire is shorter, as we don't have to ask simple, repetitive questions.
Disadvantages
To distinguish a candidate's own answer from a 'googled' one, you need to have a grasp of the subject matter. But this understanding quickly develops with experience. After reviewing 10-20 responses, you will learn to differentiate between candidates' original answers and those found online.
4. Live-Doing (Coding) – we solve a simple task in real time with a shared screen.
The essence of this method is to ask the candidate to solve a simple task while observing the process. The candidate is free to use anything and there are no restrictions on searching for information online. Candidates may feel stressed by being watched during their work. Not all candidates agree to this form of skill assessment. However, this approach allows you to see what knowledge a person has in their mind, how they can utilize it even under stress, and what information they will seek online. The candidate's level becomes apparent almost immediately. Beginners often use the most basic, even primitive features of the language, and tend to manually implement the functionalities of basic libraries. More experienced candidates are well-versed in basic classes, methods, and functions and can solve simple tasks 2-3 times faster than beginners by applying familiar functionalities from the language's basic library. Even more experienced candidates usually begin by discussing various approaches to solving the task and present several solution options, asking which implementation of the options I would like to see. Everything the candidate does can be discussed. Even with the same task, interviews can yield very different outcomes, as can the candidates' solutions.
As a variation of this method, you can ask the candidate to take a professional competency test, justifying their choice of one answer option over another. Unlike standard testing, you will learn how well-founded their answer choices were. You can create your own variations of this method, taking into account the specifics of your vacancy.
How to implement it
This method can be easily implemented using Skype or another similar video-calling system that allows screen sharing. You can create tasks yourself or utilize platforms like Code Wars, among various ready-made tests.
In what cases do I use this method?
When selecting programmers and the resume doesn't clearly indicate the candidate's skill level, I propose interviews in this format. In my experience, about 90% of developers don't mind. They appreciate that the conversation about programming starts right from the first interview, rather than answering silly questions like 'Where do you see yourself in five years?'
Benefits
Despite the stress and anxiety of the candidate, their overall skill level becomes evident immediately. The candidate's communication skills are also clearly noticeable – how they reason, explain, and justify their decisions. If further discussion with colleagues is needed, it's easy to record the screen and show the interview to others.
Disadvantages
Connection may drop. Due to nerves, the candidate might start to struggle. In this situation, it's possible to pause and give them time to think about the task alone, call back in about 10 minutes, and continue. If the candidate still behaves strangely after that, it may be worth trying a different way to assess their skills.
5. A short express interview about skills via phone/Skype
This is simply a voice conversation over the phone, Skype, or another voice communication system. During this, we can assess the candidate's communication skills, their erudition, and breadth of knowledge. A questionnaire can be used as a conversation guide. Alternatively, one can discuss the candidate's responses to your questionnaire in more detail.
How to implement it
We agree with the candidate on a conversation and call them. We ask questions and note their responses.
In what cases do I use this method?
I usually apply this method alongside a questionnaire when the candidate's answers seem original or insufficiently convincing. I discuss questions from the questionnaire with the candidate to explore their opinions in more depth. I consider such conversations essential when the candidate's communication skills are important, as well as their ability to clearly and understandably articulate their thoughts.
Benefits
Without a voice conversation on professional topics, it's usually impossible to determine how well a candidate can express their thoughts.
Disadvantages
The main drawback is the additional time costs. Therefore, I use this method in conjunction with others when necessary. Additionally, there are candidates who articulate professional topics well but have limited practical skills. If you need a programmer who can reliably and effectively solve problems, it's better to choose another method for initial competency assessment. However, if you're looking for a manager or analyst—someone who translates from layman's language to 'programmer language' and back—this competency assessment method will be quite useful.
6. A short timed test assignment (to be completed in 30-60 minutes)
For several professions, it's important for specialists to quickly find solutions to problems. Typically, the problems are straightforward, but the time taken to resolve them is crucial.
How to implement it
We agree with the candidate on the time frame for completing the test assignment. At the agreed time, we send the candidate the task conditions and check if they understand what is required of them. We time how long the candidate spends solving the problem. Then we analyze the solution and the time taken.
In what cases do I use this method?
In my experience, this method has been used to assess the competencies of technical support specialists, SQL programmers, and testers (QA). The tasks included types like 'find the problematic areas and propose a solution' or 'optimize the SQL query to run three times faster,' etc. Of course, you can come up with your own tasks. This method can be applied even for beginner developers.
Benefits
We only spend our time on preparing and reviewing the assignment. The candidate can choose a time that is convenient for them to complete the assignment.
Disadvantages
The main drawback is that solutions to your tasks or similar might be available online, so it's necessary to have a variety of options and to periodically create new assignments. If you need to test reaction speed and breadth of knowledge, personally, I prefer timed tests (method #2).
7. Review the candidate's portfolio, code samples, and public repositories
This is arguably the clearest method of competency assessment, provided that your candidates have portfolios and your recruitment team includes specialists capable of evaluating them.
How to implement it
We review candidates' resumes. If we find links to portfolios, we examine them. If there are no mentions of a portfolio in the resume, we request it from the candidate.
In what cases do I use this method?
In my experience, this method has been used very rarely. It's not common to find relevant work in candidates' portfolios. Experienced candidates often prefer this approach over standard and uninteresting test assignments. They say, 'Check out my repo, there are dozens of examples of my solutions to various problems; you'll see how I write code.'
Benefits
This saves candidates' time. If there are professionals in your team with time available, it’s possible to quickly filter out unsuitable candidates without communicating with them. While the recruiter is searching for candidates, a colleague is evaluating the portfolio. This allows for relatively fast and parallel work.
Disadvantages
This method can't be applied to all IT professions. To evaluate a portfolio, one must possess advanced skills. If you are not a specialist, you won't be able to assess the portfolio effectively.
Colleagues, I invite you to discuss what you've read in the comments. What other quick methods for competency assessment do you use?
Source: habr.com
