{"id":95924,"date":"2020-10-05T01:42:16","date_gmt":"2020-10-04T23:42:16","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/progulka-po-grablyam-10-kriticheskih-oshibok-razrabotki-testa-dlya-proverki-znanij"},"modified":"2020-10-05T01:42:16","modified_gmt":"2020-10-04T23:42:16","slug":"progulka-po-grablyam-10-kriticheskih-oshibok-razrabotki-testa-dlya-proverki-znanij","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/progulka-po-grablyam-10-kriticheskih-oshibok-razrabotki-testa-dlya-proverki-znanij","title":{"rendered":"A Walk through the Traps: 10 Critical Test Development Mistakes for Knowledge Assessment","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"A<\/a><\/noindex>
\nBefore enrolling in the new Machine Learning Advanced course, we assess prospective students to determine their readiness level and understand what they need to prepare for the course. However, a dilemma arises: on one hand, we must check knowledge of Data Science, and on the other hand, we cannot conduct a full four-hour exam.<\/p>\n

To solve this issue, we established a TestDev team directly within the Data Science course development team (and it seems this is just the beginning). Here\u2019s a list of 10 pitfalls that one encounters when developing tests for knowledge assessment. We hope that the world of online education will improve a bit because of this.
\n<\/a><\/noindex><\/p>\n

Pitfall 1: Failing to clearly define the testing objectives<\/h2>\n

\nTo properly define objectives and create a test that considers them, during the planning phase, we must answer several questions:<\/p>\n

    \n
  1. What exactly are we checking?\u00a0<\/li>\n
  2. In what environment will the testing take place, and what mechanics will be used? What limitations exist in this environment? This point will also help us understand the technical requirements for the device on which the testing will occur, as well as the content (if tests are being taken on phones, images must be readable even on small screens, and there should be a possibility to zoom in, etc.).<\/li>\n
  3. How long will the testing last? We need to think about the conditions under which the user will take the test. Is there a chance that they will need to interrupt the testing process and then continue later?<\/li>\n
  4. Will there be feedback? How is it formed and delivered? What is needed to receive it? Is there a time gap between completing the test and getting feedback?<\/li>\n<\/ol>\n

    \nIn our case, by answering these questions, we determined the following list of objectives for the test:<\/p>\n

      \n
    1. The test should show whether prospective students are ready for the course and if they have enough knowledge and skills.<\/li>\n
    2. The test should provide us with material for feedback, indicating the topic where students made mistakes, so they can strengthen their knowledge. We will explain how to create it further.<\/li>\n<\/ol>\n

      <\/p>\n

      Pitfall 2: Not creating specifications for the expert \u2014 the test compiler.<\/h2>\n

      \nTo create test assignments, it is crucial to involve an expert in the relevant field of knowledge being assessed. Additionally, the expert needs a well-crafted specification (description) that includes the test topics, the knowledge\/skills being evaluated, and their level. <\/p>\n

      An expert will not create such a specification for themselves, as their job is to devise assignments, not the test structure. Moreover, few people are developing tests professionally, even in the teaching process. This is taught as a separate specialty\u2014psychometrics.<\/p>\n

      If you want to quickly get acquainted with psychometrics, there is a summer school<\/a><\/noindex> for everyone interested. For a deeper study, there is a master's program<\/a><\/noindex> and a graduate school at the Institute of Education.<\/i><\/p>\n

      When preparing the specification, we gather a detailed description of the test for the expert (or better yet, together with them): the topics of the assignments, the type of assignments, and their quantity.<\/p>\n

      How to choose the type of assignments: once the topics are determined, we decide how best to assess them with assignments. Classic options include open-ended questions, multiple-choice or single-choice questions, matching, etc. (don't forget about the technical limitations of the environment in which the testing takes place!). After defining and specifying the types of assignments, we have a ready specification for the expert. We can call it the test specification.<\/p>\n

      Pitfall 3: Not Involving the Expert in Test Development<\/h2>\n

      \nWhen involving the expert in the test development process, it is important not just to outline the 'scope of work' but to engage them in the actual development procedure.<\/p>\n

      How to make working with the expert as effective as possible:<\/p>\n