Pedagogy interested me for a very long time and, for many years, I, being a student, brought up, but at the same time, harassed and harassed by the existing organization of education, thought about how to improve it. Lately I've been getting more and more opportunities to test some of the ideas in practice. In particular, this spring, at the Polytechnic University (SPBPU), I had the opportunity to read the course “Signal Processing”. Its organization, especially the organization of reporting, is the first experiment, the results of which seem to me to be somewhat successful, and in this article I want to talk about the organization of this course.
I still don’t have a clear understanding of what should be read in a course with such a name, but in general this is a course about what and how you can automatically do with images, sound, text, video and other examples of natural and artificially generated signals . According to what was read before and would be most useful, this is solving problems with a semantic gap between the input signal and what you want to understand from it. This article is not about the content of the course - even in Russian there are quite a few videos of good courses on similar topics.
But if the content is interesting
here is at least in the near future a valid link to the presentations of the course, which lie on . Most of what is there is torn from Anton Konushin's courses, csc and various articles on the Internet that fall into the top of the most relevant. Nevertheless, in some places there are things that I did not find clear descriptions and tried to come up with my own, in some places there are Russian descriptions of what I could only find in English - this especially applies to clustering, for example, to the mcl algorithm.
The plan of the article is approximately the following: first, the organization of the course I have chosen is briefly described, then there is a story about the problems that I consider it useful to solve, then about how I tried to do this when reading the “signal processing” course and how I evaluate the results, what problems I see What ideas do you have for solving them? All this is nothing more than my thoughts and ideas, and I will be very glad to comments, objections and more ideas! Moreover, this is all written in many respects in the hope of receiving your ideas and comments. Also, perhaps this text will help someone find interest in quality teaching, despite all that is happening around.
General scheme of the course organization
The course has two components: theoretical and practical. Both parts are very important: the theoretical one gives a great overview of existing algorithms and ideas for their design for solving problems with a semantic gap; the practical one should give at least some overview of existing libraries, as well as train the skills of constructing your own algorithms. Accordingly, for both parts, reporting was required that stimulated their study, setting the main line of work for students.
As usual, the theoretical part consisted of lectures. After each lecture, students were given a wide list of lecture questions to take home, consisting of both routine questions about the details of what was told, and creative questions about how and in what cases certain ideas told could be improved and where they could be used before asking students to come up with their own questions. according to the lecture (and you can also answer them at the same time). All questions were posted in a post in the VKontakte group, the answers had to be written in the comments: one could either answer a question that had not yet been raised by anyone, or comment / supplement an already existing answer, including one made by another student. The scope for creativity closely related to the subject, in my opinion, was huge!
Ranking was supposed to be an addition to the answers to the questions: after the deadline, the students had to email me the names of those who answered, ranked depending on the grades they deserved. Ranking comments were also welcome. After all this, I finally gave points for the lecture. Based on the results of these points and a number of additional bonuses, including those growing from the practical part of the course, grades were given for the semester. Dissenters and idlers could try to increase their grades on a harsh exam (absolutely everyone can use it, but I ask hard for understanding).
The general message of the theoretical part was something like this: I try to give an insane amount of material, hoping that all students will find a lot of new and useful things in it. At the same time, I do not require them to delve into everything, they can either choose interesting / useful moments for themselves and delve deeply into them, or do a little of everything. I perceive the exam more as a penalty for those who did not work well in the semester than as a norm.
The practical part consisted of
- three mini-laboratories in which students had to run ready-made code that actively uses different libraries and select data on which it works well or poorly,
- term paper, in which students were required to independently solve a problem with a semantic gap. They could take the initial task either from those offered, or choose it themselves and agree with me. Then they had to come up with a solution, code it, see that it works badly the first time, and then try to improve it, guided by their own and my advice. The ideal would be to achieve really good quality, convincing students that in this area, too, patience and work in the right direction will grind everything, but, of course, this can not always be hoped for.
All of this had to be done. The quality of work and the amount of effort expended could vary significantly. Due to the greater efforts, it was possible to get more additional points in addition to the lecture ones.
It happened in the spring semester of the 4th year, when the semester ends a little more than a month earlier because of the bachelor's. That is, I had about 10-11 weeks.
I also had an insider in the form of a sister who studied in one of the two groups with which I lectured. My sister was sometimes able to stop my crazy ideas with stories about her vision of the real situation in the group and downloading in other subjects. Combined with the successful theme of the course, fate really favored experimentation like never before!
Thinking about the problems you want to solve
In this section, I try to talk about the issues that led me to the structure of the course described. These problems are mainly related to two facts:
- There are creative and active students who are able to independently organize their education in the direction they really need. By driving everyone to the average level, the existing system of education in universities often creates difficult, nervous and meaningless conditions for such students.
- Many teachers, unfortunately, are not interested in the quality of their work. Often this disinterest is the result of disappointment in students. But the bad work of students cannot but be a consequence of the bad work of teachers. The situation can improve if quality work benefits teachers themselves, and not just students.
Of course, there are many more problems that are not very related to both the first and the second. For example, what to do with those students who are not able to organize themselves on their own? Or those who seem to be trying, but still can't do anything?
The problems connected with the two described facts, I have suffered the most, and I have thought a lot about their solution. It seems to me that there is at the same time a “silver bullet” that solves them: if smart students are in comfortable conditions, then they can be of great benefit to teachers.
Teacher motivation
Let's start with teacher motivation. Naturally, it is necessary for a good course. So, from the conduct of the course, the teacher can receive:
- Pleasure.
- Money. Ours are often symbolic. Especially for those who teach well in IT, this money is completely ridiculous. As a rule, these people have or can have many times more earnings in another job. And they definitely can't teach well just for the sake of a paycheck.
- The incentive is substantially better to immerse yourself in the material. I was very worried about the popularity of my lectures. And I, at least for now, was very afraid of the condemning views of students and their negative opinion: “there is nothing else for one to do but make us waste time on some nonsense that even he himself could not or did not consider it necessary to deal with” .
- The results of student immersion in the material. It is possible to create an atmosphere that will stimulate reasonable questions from students in lectures. Such questions can greatly help the teacher: point out some mistakes and shortcomings, encourage you to look at things from a different angle, or maybe even make you understand something new.
- It is possible to stimulate the activity of students that goes beyond the material read in lectures. Then they can collect a lot of new information and give results in a somewhat reworked form. Yes, it's still hard to understand and verify later. But it is in the course of such checks that the horizons expand. And there is another bonus: if something is not clear, sometimes you can ask the student instead of figuring it out yourself. Such a question will also be a test of how well the student understood.
- Training of communication with people. Training in evaluating people, understanding what you can expect from them, including depending on your own actions. You can try to assess in advance which student will cope with the task well and on time, which one will do poorly, which one will do what is needed, but for a very long time. Train different management approaches (reminders, etc.). Understand how easy it is and how exactly students (and probably not only them) can manipulate you. The room for experimentation is huge. Experimental results can be seen relatively quickly.
- The practice of competent presentation of thoughts, lectures and other oratory skills. Training in understanding answers and questions poorly formulated by students (sometimes you have to do all this on the fly - you can train your own reaction).
- The results of testing simple ideas in practice by the hands of students. Both the results of checking one's own idea and the idea that came to the student's mind can be useful. If you find a problem that is really interesting for the student, there is a high probability that the student will generate good ideas and check them qualitatively.
- 'Free' use of students to solve their practical problems.
It is widely believed that teachers benefit the most from this. I believed in this for quite a long time, but with each subsequent experiment, my faith in this decreases. So far I have had only one student, from cooperation with whom I eventually got exactly what I wanted, on time, and really saved my time. I probably managed to teach this student better than others. True, and then, later, during the project, it turned out that I needed to solve this problem in a slightly different form, but this is definitely my fault.
For all the other students that I encountered, I had to run endlessly, remind them of scientific work, explain to them the same thing several times. In the end, I received something very strange from them, and often at the moment when I had already solved this problem myself. I don’t understand how useful this format is for them (it seems that they are training to do something, but somehow very poor quality). For me, this process eats up a lot of nerves and time. The only good thing is that sometimes, in the course of discussions, my attention is drawn to some details of the problem that I had not noticed before.
- Fame, prestige - with quality teaching
- Visibility of the results of their activities and grateful students. True, it is often difficult to understand the truth here, students are often grateful for something completely different from what they need.
- Acquaintance with future specialists in their field. It is better to understand them, to understand how the new generation lives. You can select those who you like, and then invite them to work.
That's all I managed to collect. For myself, I am trying to be as clear as possible about what, besides pleasure and prestige, I hope to get from reading the course. What should it be so that I am ready to pay for it with my time all semester? Without this understanding, it is hard to believe in the possibility of conducting a course well. Your own motivation must be taken into account when thinking through the structure of the course.
Comfortable conditions for advanced students
The second part of the requirements for the course structure is aimed at creative and active students who have a good idea of what they need. Despite the fact that many teachers confidently deny even the possibility of the existence of such students, they definitely exist in advanced universities. By senior years, their number is growing significantly, especially with quality education. And it is smart students who are the hope of our fatherland and science.
In almost all universities, education is far from being as effective as it could be. At lectures, students are often told something that may be interesting, but strange: if necessary, then in some world to which students have not yet grown up. It often happens that advanced students have already heard or read about these things, understood, and then forgot - now they are forced to listen again. Often students have to do strange practical tasks that the teacher came up with only because he thought that students needed at least something to load. Write and correct reports that teachers often do not accept the first time, simply because it seems to them undignified, and besides, you need to teach at least something.
If it all falls on people who otherwise would not do anything, this is probably not bad. As practice shows, by the end of their studies, these people understand something, most of them are quite suitable for work in their specialty.
But it happens that such a system is applied to advanced students who already have their own plan of action, their own work, their own understanding of where to move. Moreover, this understanding is generally correct, and the work can be made very popular, if slightly corrected. And so lectures with abstract theoretical material, ill-conceived practical tasks and reports that need to be endlessly written and corrected fall on these students. Even if that's all it takes, it's much more efficient to link it to the student's academic interests. So that he understands how this information will help him in practice.
Otherwise, if the student does not understand, only a small part will be learned. Yes, and it will soon be forgotten if it is not closely used in other courses. Only a general idea remains. As well as from non-core uninteresting school subjects or students who are not interested in anything. Still, maybe there will be an understanding of where to run to figure it out.
But it takes a lot of personal time for students to get this information. Many advanced students could put it to good use. Such people are ready to absorb the knowledge they need almost on the fly and with amazing efficiency, especially in senior years.
Yes, maybe your course is exactly what an advanced student lacks. And he, the poor fellow, does not understand. But abstract theoretical lectures are unlikely to help him. If you understand the essence of some work that is interesting to him and advise him to apply at least a small fragment of the knowledge that you give in the right place, the student will surely understand and appreciate it. Especially if your proposal for improvement will help to achieve a qualitatively better result.
In reality, of course, everything is somewhat more complicated. Not all useful knowledge can be applied in the area that the student is interested in. Then, especially if it happens in senior years, it would be good to try to understand what is more useful for the student: to do what you think is necessary, or what he himself considers necessary for himself. And act accordingly.
In this course, I almost did not have such a problem: the course on solving problems with a semantic gap seems to me applicable everywhere and useful to everyone. In essence, this is a course on designing algorithms and models in complex situations. I think that it is useful for everyone to understand that this exists, and how it works at least at the top level. Also, the course trains well modeling skills and a reasonable approach to solving a lot of problems.
Much more I was afraid to tell only what many students already know. I didn’t want to force them to solve tasks that would not teach anything. I wanted advanced students not to be forced to do assignments for show, just to get credit.
To do this, you need to understand good students, understand what they know and what they strive for. Interrogate them, find out opinions, look at the results of the work, understand something from them. Make sure that the students are not afraid of me. Don't be afraid to answer the question incorrectly. Were not afraid to criticize my line.
But you have to be not only not scary, but also demanding. Reasonable exactingness helps even advanced students, builds them. The time allotted for the task helps to understand which path to take, how deep to dig, when to ask for help. Result requirements help you understand what to focus on. And all this organizes, helps to prioritize between a lot of fallen cases.
Being not scary and demanding for a teacher is far from easy. Especially if there are many students. Lazy people are more demanding. With them you are tormented to be fair in each case. For advanced students, the opposite is true. They are much more afraid of teachers' tyranny than others. Because they have more at stake, more dependent on standings and relegation. The very first unreasonable requirement throws one into doubt: “Is the teacher reasonable? Will he adequately respond to my criticism? Each subsequent doubt strengthens, the teacher in the eyes of the student turns into a nutcase who must be pleased, spending as little time as possible.
It seems that only a reasonable rigid reporting system can solve the problem. Pre-designed, which will not change in the semester. Compliance with this system should become more important than the opinion of the teacher, no matter how strange it may sound. This dictates a high level of requirements for the reasonableness of the original system. It is clear that it is impossible to foresee everything, and one does not want to waste time. Therefore, it is possible to explicitly indicate the boundaries, beyond which the teacher acts at his own discretion. For example, a lab handed over after the deadline will be checked no one knows when, and after two labs not submitted on time, the consequences can be unpredictable. Further, depending on the reasons that led to this, you can either pardon or punish. But, if what has been done satisfies the requirements, the teacher must do what he promised.
So, it was necessary to come up with a rigid reasonable reporting system. She needs to be more loyal to reasonable students. Positively took into account everything useful that they can come up with and that will be connected with the course. But she didn’t give good grades for anything, but stimulated precisely to quality work.
It is also important that they believe in the reporting system and feel at ease with it. So that the student can set himself the task of doing everything at the beginning of the semester, getting an assessment and feeling calm. Do not be afraid that the teacher in the middle of the semester will think: “something is already too good for him. Probably, it is possible to give more complex tasks and make the assessment dependent on them as well.”
Also, as follows from the previous section, the reporting system should take into account the wishes of the teacher. And it turned out that many requirements have already been taken into account: they coincided with the requirements of loyalty to reasonable students and quality work. If advanced students are free to ask questions, they will ask, among other things, what the teacher does not know. If you can go beyond the course, they will go out and find new information. If they understand what they are doing and why, they will do it well. And information about the results of such experiments, of course, broadens the horizons of the teacher. Let not immediately, but sooner or later there will be something new, useful for him.
A happy smart student is a happy teacher!
Assessment problems
A reporting system cannot motivate students without a reasonable assessment of their performance. How to evaluate, based on the results of the semester, which student deserves a higher mark, and which one deserves a lower one?
The most commonly used criterion for us is the grade for the exam. The teacher tries in the course of some communication or written to understand how well the student understands the topic at the time of passing the exam. This in itself is difficult. Often almost all understanding, but timid and unable to speak, students receive lower marks than students who do not know the subject, but are quirky and arrogant. The written exam allows you to lower the degree of impudence that the student can use. But interactivity is lost: it is impossible to understand whether the student understands what he did not finish (and even what he wrote). Another problem is cheating. I am familiar with some masters of pedagogy, whose grades were inversely correlated with the knowledge of students: the tasks covered an insane amount of material, and even someone who prepared well could not pass it for a normal grade. But those who cheated received 5 and the teacher confidently concluded on their basis that it was possible to cope - if you prepared.
Ideas for solving these problems exist. But, even if these problems can be solved, it will still not be possible to assess the residual knowledge of the student.
The probability of increasing the amount of residual knowledge increases if knowledge is in the student's head not only at the time of the exam, but also during most of the course. And if the knowledge is backed up by practical activities, they will certainly remain. It turns out that it would be good to evaluate the student's knowledge several times per semester. And at the end, put an automatic grade if the student did a good job in the semester. But this loses the general overview of the course, which the student should have received in preparation for the exam.
The problems do not end there: all students are different, it happens that something is obvious to one, and another needs to think about it for a long time. Perhaps it is fair to evaluate not only their final knowledge, but also the amount of effort expended? How to evaluate them? What is better: to overestimate a student or underestimate? When evaluating students, is it advisable to compare their level with the level of the group / stream? On the one hand, it seems yes: if there is a problem with the entire flow, then the teacher did a bad job. On the other hand, lowering the bar will contribute to a drop in the level of students.
There are systems in which students are initially placed in conditions of dependence on other students: for example, as I understand it, in a CSC course on similar topics, the scores of all students are clustered and the student receives an assessment in accordance with which cluster his score is in. Such approaches increase competitive interest, but create uncertainty, which can unnecessarily unnerve students, and can also hinder teamwork.
All this is so normal and I could not think it over. As someone who was very recently a student himself, it seems to me that the main thing is to ensure that a person can work hard in a semester to get himself a big grade - the one that he wants. There should be many ways to get this mark: for practice and for theory in a variety of formats. But, if the course is important, it is necessary that the student can get a good grade only if he either really did a good job and advanced a lot, or initially knows the course at the level of a teacher. This is the system I was trying to come up with.
In total, I tried to make the course as comfortable and useful as possible, primarily for diligent students. From them, I expected questions and messages that would push my knowledge further. But the problem, how not to forget about the others, of course, was also relevant. Here the situation is very unfavorable: I knew that as a result of a number of reasons, by the 4th year, many groups come in a very collapsed state: most of the students are still closing the previous semester; there are those who can no longer bring themselves to do almost nothing about their studies on time and who have been getting away with it for years. Timely feedback is insanely important for a teacher: you can readjust in time.
Detailed scheme of the course organization
Possible patterns of reporting and behavior of the teacher, solving the problems listed above, I began to actively think over even when I was a 5th year student. I already tried to test some of them, but there were a lot of reasons why it was not possible to get relevant ratings. With all this in mind, I put together a course and tell you exactly what happened.
The first question is: what do I want from this course? First of all, I was interested in trying my ideas in practice and really wanted something good to come out of them. The second most important argument was the improvement of one's own knowledge, but in general, to some extent, all the above goals of the teacher from pleasure to prestige took place.
In connection with the goal of improving knowledge, I would like students not to be afraid of me, to be able to freely ask questions and openly express dissatisfaction with what is happening - all this would be good incentives for me. I also wanted to receive knowledge from them - I wanted to stimulate them to collectively expand the information received and not limit the scope of their activities. Try to avoid thoughtless repetitions in their activities.
Thus, the idea appeared that students should answer various questions on the course (including creative ones and those to which I do not know the answers), see each other's answers and supplement them. But do not duplicate - in this way, I don’t have to figure out who copied and who didn’t, and for students there is another reason to expand their knowledge, to go beyond what has already been told in the lecture and written by classmates. It also becomes necessary to understand what those who preceded them wrote. It can also help encourage early responses by starting with a slightly larger selection of possible questions.
A VKontakte group was created, after each lecture, numbered questions were posted in it (15 pieces, quite voluminous). to which the students answered in the comments, complementing each other's answers.
The questions were mainly:
- To repeat what was said in the lecture. Sometimes the answer to such a question could be found directly in the presentation of the lecture given to the students after reading it.
- To come up with practical examples of using what was told.
- To identify the problems raised at the lecture in the described algorithms. As well as thinking through algorithms that solve the problems identified in the lecture. It was assumed that students could both drag algorithms from other sources and come up with their own.
- To evaluate the effectiveness of the described algorithms - including, for a better understanding of the algorithms themselves.
- To compare algorithms that solve similar problems.
- On mathematical proofs of some used or related facts (for example, convolution theorems, Kotelnikov's theorems).
It must be said that in lectures I almost did not tell formal proofs, I used proofs “on the fingers” more with a large number of approximations and simplifications. Firstly, because I myself do not particularly use formal proofs in practical life and, as a result, I do not understand them very well; secondly, I think that in the 4th year the main emphasis should already be placed on practical understanding, and not on theory, without which one can generally live. - Another reason: the lecture courses I watched on this topic, richly supplied with theoretical and mathematical definitions and proofs, seemed to me either very difficult to understand everything at once, or affecting too little information - diving into them now seems to me like digging into something that will almost never be be used.
- For personal impressions of the course and ideas for improving it - after the last lecture.
It was also possible to reasonably consolidate the students' answers and my comments into a single readable document - this was also evaluated by points. And the document itself would later be useful both to students and to me.
The main question that confused me was: well, everyone will really like it and they will start to write really a lot and well. But after all, then someone should check all this - will I have enough time for this? In addition to reading these lectures, I had a main job, graduate school + scientific work, which I, however, almost abandoned this semester. It seemed that this problem could be solved by a scheme that would allow shifting at least part of the test from the teacher to the students. In addition to making the teacher's job easier, it is also undeniably useful for students: through the search for mistakes and the vision of another person, a significantly better understanding often comes. Some students are additionally interested in such “ala teaching” activities.
In the current case, I settled on the ranking of the results by students:
there is a hypothesis that it is easier for students to compare two works than to give specific marks.
(from online education studies, e.g. Waters, AE, Tinapple, D., and Baraniuk, RG: "BayesRank: A Bayesian Approach to Ranked Peer Grading", 2015)
And ranking could help me a lot. Accordingly, after the deadline for answers, the students had to send me ranked lists of their colleagues, and comments on these lists were welcome. In principle, I did not insist on ranking, but only recommended it, whoever wanted what - he could send it. At the end of the course, it turned out that after the full ranking, the most common form of answer was the top k who wrote the most useful answers.
The semantic organization of the course
The next important part was the semantic content of the course. The plan of the theoretical part of the course turned out as follows:
- Zero lecture - introduction, what the course is about, what accents I am going to make + reporting (its rules are gigantic and I spent almost half of the lecture on their story)
- 1-3 lecture on how image processing problems were generally solved before the advent of machine learning. Convolutions for finding intensity differences and smoothings, canny, morphological image processing, image viewing in different spaces (Fourier transform / wavelets), ransac, Hough / Rodin transforms, singular point detectors, blobs, descriptors, building a recognition algorithm.
- 2-3 lectures (as many as you need) about the ideas of machine learning, the basic principles, how it helps to solve the problems of invented algorithms. Automatic enumeration of parameter values, conditions, their sequences, what can be done with data and what should be feared, which models are better to take as a basis, dimensionality reduction, approximating network data, clustering. I planned to tell the first part of this rather quickly (it is also found in other courses), about clustering in more detail (why it is dangerous to use them, which algorithm to choose and what you need to remember).
- Lectures that deal with examples of real problems (at a minimum, face recognition and video stream processing, and so - depending on how much time is enough, maybe students will have ideas or a desire to tell something of their own). A semi-seminar format was assumed, in which we first try to set a task, then bring student ideas to those that solve it, then move on to methods that they actually use and have not yet guessed. For example, in the task of identifying a face from an image, the ideas of PCA and LDA (Fisher metrics) are used, which is difficult to think of, at least in a lecture.
The practical part should illustrate some aspects of the theoretical part, introduce students to libraries and make them solve a difficult problem on their own. Accordingly, there were three mini-laboratories, in which it was necessary to take a set of ready-made scripts and run them, achieving various goals along the way:
- install python, pycharm and various libraries. The launch scripts are the simplest: uploading images, some simple filtering by color and pixel location.
- a set of scripts illustrated part of what was said in lectures 1-3, students had to choose pictures on which the scripts would work well or badly, explain why. True, I didn’t have enough scripts for this laboratory and they turned out to be rather short.
- on machine learning: I had to choose one of two libraries: catboost or tensorflow and see what they give on simple tasks (tasks and datasets were taken from the library examples almost unchanged, I also didn’t have enough time). At first I wanted to give both libraries together, but then it seemed that it might take too much time.
I tried to pick up all three labs so that they could be made in 3 hours - in one evening. The checked result of the lab was either selected sets of images and the results of working on them, or the values of the parameters of library functions in the script. All labs were obligatory, but it could be done either qualitatively or poorly, for high-quality performance and special tasks for labs one could get additional points that increase the grade for the semester.
Students could choose difficult tasks themselves: for example, take something related to a bachelor's degree or work, or from those offered. It was important that this task be a task with a semantic gap. It was important that the solution of the problem did not require programming in large volumes. Difficulty was not very important - I thought that a bad result would also be a result. 5 stages of work on the task were identified, the results of each stage had to be agreed with me.
- Select a task
- Data selection: an important stage, during which, as a rule, a much more realistic idea of the problem is formed at the same time, hypotheses of algorithms solving it are born.
- Compilation of the first approximation: an algorithm that would at least somehow solve the problem, from which one could build on and improve further.
- Iterative improvement of the problem solution.
- An informal report describing the final algorithm and the algorithm modifications of the original algorithm that were made to obtain it.
The task itself, like the mini-labs, was mandatory; for its high-quality performance, you could get a lot of extra points.
About a week before the test, I added an alternative version of the problem, in the solution of which it was possible to count on a maximum of 4ku: I take a signal that is described by a complex mathematical function and generate data for students on it for training / testing. Their task is to approximate the signal with anything. Thus, they avoid the data collection step and solve an artificial problem.
Evaluation
Above, I wrote a lot about points, it's time to explain what they gave.
There were several areas of activity for which points could be earned. At the end, the scores for all areas were multiplied and raised to the power of “1/<number of lectures read in the semester>”. Directions:
- Each lecture is a separate direction
- Mini laboratory
- Large (complex) laboratory
- Organizational issues
this includes points for advice and work that helps organize the course, such as pointing out objectively that something is missing, something is being done poorly, or trying to rewrite the reporting description to make it more readable. The number of points varied at my discretion, depending on the usefulness, relevance, intelligibility of the wording, etc.
- Everything else related to the topic of the course
for example, if a student wants to touch on an aspect of signal processing that I didn’t talk about, points will go here. You can touch on something, for example, by preparing a fragment of a lecture on this topic; depending on the quality of what was done and the situation over time, I may or may not allow this to be done at the lecture, but in any case I will give at least some points and write some of the comments that have arisen - the student will have the opportunity for the next iteration, deepening his knowledge and bringing new points.
Initially, for each direction, the student had 1 point (so that the multiplication does not exactly turn out 0). Another 1 point could be obtained for coming to a lecture (in the direction corresponding to this lecture), it was not so easy - the lectures were at 8 in the morning. I could not systematize the amount of points received for everything else, so I put it at my own discretion, definitely often making mistakes. There was only a general picture, according to which a student who perfectly understood the lecture could get 25 points, well understood - 10 points, tolerably understood - 5 points, less was given to someone who did at least something. Naturally, when evaluating, I could only rely on what the student wrote, although more often he could be lazy or something else, as a result of which his real knowledge did not reach me.
It is important to write about deadlines. Lectures were on Tuesdays at 8 am. First, the next Sunday was set for answers to the lectures, and the next Thursday after Sunday for ranking. Then, on the part of the students, what I came to in the first couple of lectures myself was clearly expressed: I need to write feedback on the answers, and after it it is desirable to give students the opportunity to improve. Along with this, voices began to be heard that 5 days for answers are very few. As a result, contrary to the expressed fears of other students, I added a week to answer questions, and began to comment on the answers that came before the first Sunday. The decision was unambiguously wrong: no more answers were given, and new lectures were held for an extended period, and even I was confused about what and what applies to. But he didn’t change anything anymore: he decided that there were already so many changes.
At the end of the semester, for those who received credit in practice, the points scored corresponded to the final grade for the course. This grade could be improved on the exam, which was supposed to take place like this:
Four difficult questions are given on various topics for understanding (I will choose the topics at my discretion). Everything that was said at lectures or fell into a group in VK can get into questions. A fully read answer to a question +1 point to the score scored in the semester (if a person understands only part of the question, then 0 points are given for the question, no matter what part it is). It will be possible to use everything, anything, but the questions will be really difficult - for a deep understanding.
The prohibition of the use of materials in the exam very often leads to the fact that students, instead of understanding, cramming or cheating.
I saw the dynamics of scoring during the semester like this: advanced students will score enough points for 5 automatic points in about the first 6-7 lectures. That is, somewhere by the end of March, just when I will tell the basic information and move on to examples of setting and solving real problems. With practice, I hoped that the diligent ones would also figure it out by April, at most by the middle, if its priority was lowered by the requirements of other courses. Evaluated it on my own: I think that when I was a 4th year student, I would have passed such a course approximately at the indicated time, if nothing unexpected had happened. From less advanced students, I expected that many of them would be interested in questions, at least as an opportunity to get an automaton, they would read the answers of their colleagues and fragments of lecture presentations. The topics are generally interesting, and it is possible that such students will be able to hook on, and they will try to understand more deeply.
I would like to make a remark about the chosen multiplicative combination of scores between directions, and not additive (the root of the product, not the sum divided by some number). This corresponds to the need to deal with a large number of directions at about the same level; even very, very deep knowledge in a couple of areas will not provide a student with a good grade for the course in the absence of his knowledge in other areas. For example, multiplicativity protects against the possibility of getting a 5 by bombarding me with suggestions for improving the organization of the course: each subsequent proposal, bringing the same points as the previous one, would contribute less and less to the final grade.
One of the immediately noticeable shortcomings of this system is its complexity. But, since the course itself is quite complex and solving problems with a semantic gap requires the construction and understanding of complex algorithms, I believe that students should be able to understand this easily. Moreover, this reporting system is somewhat similar to solving a problem with a semantic gap: some problems arose in the course model, the most important ones were selected, and approximations were searched for their solution.
Another disadvantage of the system is that it can really take a long time for students. Therefore, I tried to apply an old idea: to invite students who know the material well and without a course, or who consider themselves busy with more important things, to contact me in the first month. I am ready to talk with them, and, depending on the level of their knowledge and the reasons for crowding out my course, offer them an automatic machine or a simplified, adjusted for them, way to pass the course. After the first month, the offer is removed - otherwise it can be used at the end of the semester by weak students who could not force themselves to do something, but potentially would like to.
Approximately it was stated to students at the first lecture. Then I promised myself not to change it, even if I see that it does not work well and students do significantly less or worse than expected. The course has started.
The results
The results turned out to be much worse than my expectations, although a number of hopes were justified. I remember, after the first list of questions on the introductory lecture, I was anxiously waiting to see if any answers would appear and whether they would be meaningful. And now, finally, the first answers began to appear, some kind of discussion even began in the comments, however, rather on a philosophical topic. Further, during the semester, students continued to answer; however, as a rule, there were a couple of dominant students who contributed about 70% of everything useful that was written.
By the end of the semester, the activity decreased significantly, after the penultimate lecture, they sent me a ranked list consisting of one last name - the only person who answered at least some questions about that lecture. The reasons for this, I think, could be general fatigue, perhaps some kind of disappointment, inadequacy of assessment, an unsuccessful change in deadlines, which led to the need to wait 3 weeks to get the final result of the lecture, an increased workload in other subjects.
I was also increasingly disappointed with the quality of the answers: it often seemed that a lot of things were torn off without understanding, and the volume of new ideas was far from being as large as I expected. Even from students there was a comment that the current system stimulates at least some responses; the degree of how deeply the student understood, the scores do not depend so much. But there were definitely those who understood.
Since no one fit into my plans for scoring and this threatened that everyone except for a couple of people would have to take an exam, I began to try to put higher scores. It began to seem that I was overestimating the points for those who answered only with examples of tasks, and the difference between these replies and those who really tried hard was too small. By the end of the semester, I was increasingly overcome by the feeling that there are many students who understand almost nothing of what was said, although they have relatively acceptable scores. This feeling became even greater at the last lecture, when I began to try to ask everyone in a row in the hope of better understanding the final level and adding points to those who answered correctly - it turned out that many do not know basic things, for example, what neural networks or special points in the image are.
Hopes for ranking were not very justified either: there were very few comments in the ranked lists, by the end they disappeared altogether. Often it seemed that they were assessing visually rather than carefully reading. However, I remember at least a couple of times when ranking really helped and I adjusted my scores by looking at it. But there was no question of it estimating for me. It took quite a long time to assess, but I could do it on the way to the subway and in the end I was more likely to have timely answers than students.
A separate disappointment, though expected and arising from the current situation and the fact that I almost did not take this situation into account, was with practice.
No one passed the large laboratory even in April. And I didn’t really understand if it was difficult or if they couldn’t take it at all, and I didn’t know if something needed to be changed and how, what to demand in the end. I came up with a task for 4 maximum, but it did not change the situation either. At best, by the end of April, students have chosen tasks for themselves and sent data. Some of the selected tasks turned out to be frankly unsolvable at the students' level of knowledge. For example, a student wanted to recognize cancerous tumors, but at the same time he himself did not understand how exactly they should differ - naturally, I could not help here.
With mini-laboratories, things were much better, the first two were passed by many on time or without much getting out of it; almost everyone passed the third, too, but at the very end. Some made them qualitatively and better than I expected. But I wanted to make the main practical emphasis on a large laboratory.
Another of my mistakes in the field of organization of practice, I consider the initial planning of the main focus of work on a complex problem for the second half of the semester, by the time when I already tell most of the ideas of constructing algorithms in lectures.
The question of whether it is possible to demand from students in practice what has not yet been told in lectures, agitated the minds of many teachers I know. It seemed that the formal correct answer was: of course not - after all, this means first taking extra time from students to study on their own what will be told later, and then telling what they already understand. But now I think that the harm from this formal position is much greater: it is no longer possible to try the most difficult thing in practice in a timely manner. At the same time, it is clear that it is necessary for the student to deal with the material on his own, and the repetition of the material can be carried out in an original way, for example, to invite a well-understood student to carefully prepare himself and read this fragment of the lecture.
In the end, did such a system give more than, for example, the classical system with an exam? The question is difficult, I hope that, after all, yes, quite a lot of material was given, while preparing for the exam, part of it would certainly have dropped out of consideration even for good students. Although there were not as many additions to the course in the answers as I had hoped.
I would like to make an additional remark about the sad feature of the situation in which students are not afraid of the teacher.
It is connected with what happens, a miracle happens and the teacher manages to teach students something globally new. For example, before my very eyes, a student begins to take a much more reasonable approach to solving a problem with a semantic gap. On the whole, he takes the right steps, gets an acceptable result, but does not know how to explain. And here I am, a teacher, trying to figure out what he did. He explains incomprehensibly - I ask a lot of strange questions, make strange assumptions, finally, I wade through the terminology of the student and understand. I offer suggestions for improvement, sometimes bad ones, as a student who already understands the problem notices. And then I get a reaction similar to the usual: “Why else do you need to do this?” and “I don’t need your advice” to “I myself could do everything perfectly without you.”
This can be especially strong when it starts something like this: the student initially comes with his self-confident and ill-conceived proposal for solving a problem like “here you just need to take a neural network and train it.” You say that you can’t do it just like that, you still need to think a lot, at least, and in general it’s better not to solve this problem with neural networks. A student, sometimes, thinks through, suffers, but, well done, he really understands and brings a well-thought-out solution based on neural networks, and with all his appearance says “I would have done it without your advice at first.” I apologize to those students who do not do this, you exist and I know some of you, thank you. Nevertheless, there are students who show such ingratitude, and, unfortunately, I myself have behaved this way more than once.
The problem of expressing such ingratitude by many teachers is easily solved from a position of strength: you can impose your own solution to the problem, interrupt the student if he does not say what you want to hear, etc. This can be effective, especially for bad students, but it makes it impossible for good students to think and realize the incorrectness of their ideas, hypotheses - and get an experience that will really be remembered. Extravagant ultimatum requirements for solving a problem without clear explanations in such a subject cause rejection, the main task of the student becomes to please the teacher, and not to gain knowledge or solve the problem. Loyalty leads to the fact that lazy students do not do much, and some even offend the teacher.
I noticed this feature before, but after this semester I somehow felt it more, experienced it. Perhaps because he really taught some students. Such ingratitude, apparently, stems from the inner pride of such students, their complexes, the desire to show themselves to a teacher who has sunk almost to their level. In addition to complicating the organization of the educational process, such behavior and ostentatious ingratitude of students often infuriates them: they really want to somehow intelligibly show the student that he has crossed the line. At the same time, you understand with your mind that, in fact, the student has figured out, the assessment should be positive. You find yourself in an almost hopeless situation, you just have to look at this case with humor and write it off as a student's stupidity, but it's difficult. I did poorly and I was offended.
Thus, the ingratitude of students can very often poison the mood of the teacher who taught them something. There can be a lot of similar poisoning mood things. They are especially sick if all the teacher hoped to get from teaching these students was pleasure. This situation once again strengthened me in the belief that it is impossible to read a whole course well on pleasure alone, you need to expect to get something else, at least a dream.
What I am sure of is that the course was very successful in terms of promoting and systematizing my knowledge. Of course, most of what I said, I generally imagined, but I felt many things more deeply. There were algorithms that I knew about and even used, but did not fully understand how they work, did not know many alternatives, or knew only the names. When preparing the course, I had to figure it out. There were also a number of new things that I noticed under the clear influence of students, for example, autoencoders. I got a lot, maybe not very often used, but definitely necessary for a good orientation in the subject area of knowledge. I think that the improvement of knowledge that has happened has already even influenced some of the decisions that I made in my work when thinking through algorithms, I hope that for the better. Of course, reading the course also brought me pleasure, but at the same time it brought me grief and disappointment.
Extension
It may happen that I will have to read this course again, for example, next year. Not for all problems I have ideas for solving, but for some there are, and I will try to describe them.
- The main problem: the lack of timely progress on a complex task, I think to solve by discussing similar fragments of other tasks in seminars and clear homework with small deadlines. Each of the homework tasks will require the completion of a small fragment of a large laboratory, such as formulating a problem statement, the first selection of data, thinking over a quality criterion, ... Points will be awarded for each fragment completed on time. If a student falls behind, he will have to catch up to start getting them.
- I also plan to more clearly and more often pronounce the main idea of the course in various contexts. Although I'm not sure that this will help: often, when you say the same thing, it starts to cause rejection on the contrary. The main idea, if anything, was that the skill of solving a problem is not a thoughtless enumeration of various ML models in various configurations, but the manual construction of an individual model for a task using pieces of existing models suitable for the task with reasonable modifications. For some reason, many people either do not understand this, or diligently pretend to be. Maybe some people can realize this thought only through practice, through stuffed bumps.
- I also plan to stop giving 1 point to everyone who came to the lecture; and set, by default, significantly less, for example 0,1. In order to get more points, it will be necessary to send or show me the notes of the main theses of the lecture or their photos on the day of the lecture. Almost anything can be written, the format and volume do not interest me. But for good notes, I'm ready to put much more than 1 point.
I would like to add this in order to further stimulate students to listen to the lecture, and not sleep and go about their business. Many more remember what they write down much better. The intellectual load for creating such notes is not very necessary. It also seems that this will not be too much of a burden for students who do not take notes, those who take it will be able to simply provide it.
True, all the students surveyed are critics of this idea. In particular, they point out that it is not so difficult to copy these notes from a neighbor at the end of a lecture, or simply write down something from the slides without paying much attention to the lecture. In addition, the need to write for some can be distracting from understanding.
So maybe it would be nice to change the shape somehow. But in general, I like this form of reporting, it was used, for example, in the math statistics course in csc: you need to send a small completed lab on the day of the lab - and, it seems to me, this stimulated many students to sit down and finish it right away. Although, of course, there were those who said that this evening they could not do it and were at a disadvantage. Here, it seems to me, another idea can help: give each student the opportunity to shift deadlines by a few days per semester. - There was an idea to replace the flat structure of answers to questions with a tree one. So that the answers to all questions do not go in a continuous list, but are at least two-level: then the answers to one question will be side by side, and not mixed with answers to other questions. The two-level structure of comments on posts is supported, for example, by facebook. But they enter it much less often and do not want to make it the main means of communication. It's strange to have two groups at the same time: vkontakte and facebook. I would be glad if someone advises some other solution.
There are many problems that I do not yet know how to solve and do not know if it is possible at all. Main concerns:
- too simple student answers to my questions
- poor assessment of answers: my assessment does not always correlate with reality
- ranking, which almost does not help: it is still very far from checking student answers by the students themselves
In general, I definitely do not consider the time spent preparing and delivering the course wasted; at least it was very helpful for me.
That's all, it seems, and it turned out to be too overloaded.
Image basics taken from:
I want to thank:
- for reviewing: my mother, Margarita Melikyan (classmate, now graduate student of Moscow State University), Andrey Serebro (classmate, now Yandex employee)
- all students who participated in this and completed the survey / wrote reviews
- and everyone who has ever taught me something good
Source: habr.com