В I have considered several reasons for participating in hackathons. The motivation to learn a lot of new things and win valuable prizes attracts many, but often due to the mistakes of the organizers or sponsoring companies, the event ends unsuccessfully and the participants leave dissatisfied. So that such unpleasant cases occur less often, I wrote this post. The second part of the trilogy is devoted to the mistakes of the organizers.
The post is organized as follows: at the beginning, I talk about the event, explain what went wrong and what it led to (or could lead to in the long run). Then I give my assessment of what is happening, and what I would do in the place of the organizers. Since I participated in all events as a participant, I can only assume the true motivation of the organizers. As a result, my assessment may turn out to be one-sided. I do not rule out that some points that I see as erroneous were actually intended to be so.
At some point, it may seem to the reader that the author has decided to swing his fists after the fight. But I can assure you that it is not. In some of the listed hackathons, I managed to take a prize, which, however, does not prevent us from saying that the event was poorly organized.
Due to respect for the organizers and participants, there will be no references to specific companies in the post. An attentive reader, however, can guess (or google) who is being discussed.
Hackathon number 1. Strict framework
Six months ago, a large telecom company organized a data analysis hackathon. 20 teams fought for the prize fund. At the event, a dataset was provided for analysis, which contained information about calls to the company's support service, activity on social networks and encoded information about users (gender, age, etc.). The most interesting part of the dataset - the user's messages and the operator's response (text data) - was quite “noisy”, for further work it had to be cleaned.
The organizers set the task - to do something interesting with the provided data, and it was forbidden to use additional open datasets from the network or parse the data yourself. It was also forbidden to offer ideas not related to the dataset. Unfortunately, the data provided was rather “poor”: it was difficult to get any interesting products from them, and from communication with mentors it became clear that many of the proposed ideas are already being implemented (or will be implemented in the near future) in the company.
As a result, the vast majority of teams (15 out of 20) made chatbots. During the performances, the decision of one team was little different from the previous one. Unable to bear it, one of the jury members asked the next team entering the stage: “What, guys, do you also have a chatbot?”. As a result, out of three prizes, the first and second places went to the teams that did not start making chatbots.
For comparison, let's take a hackathon organized by an international consulting company for Zvezdochka two years ago. Since the specifics of Zvezdochka’s activities were not familiar to many hackathon participants, at the beginning of the event the organizers spoke about the metrics that are used in the company. After that, six datasets of various directions were provided: text, tables, geolocations - there was room for maneuver for all participants. The organizers did not prohibit the use of additional datasets and even supported such initiatives. In the final of the competition, ten teams with different solutions competed for the main prize, with all teams using the data provided by the company (despite the absence of bans), which indicated a good potential for obtaining quality products.
Morality
Do not limit the creative flow of participants. As an organizer, you must provide materials and trust their vision and professionalism. If you are a hackathon participant, any restrictions or prohibitions should alert you. Usually this is evidence of poor organization (a real-life example is the constant desire to stick a fence somewhere). If you still run into restrictions, then be prepared for the fact that you will have to create a project in a pool with a lot of competition. In this case, you must take risks: do something fundamentally new or offer an unusual “killer feature” in order to stand out from the flow of monotonous projects.
Hackathon #2. Impossible tasks
The hackathon in Amadora promised to be interesting. The sponsoring company, a major phone manufacturer, began preparations 4 months before the date of the event. PR of the event was carried out in social networks, potential participants had to pass a technical test and write about their past projects in order to be selected for this event. The prize fund was pleasantly large. A few days before the hackathon, the mentors held a technical session so that the participants had time to feel the specifics of the industry.
At the event itself, the organizers provided an 8 GB dataset of equipment logs, the task was a binary classification of breakdowns. They talked about the criteria for evaluating projects - the quality of classification, the creativity of creating features, the ability to work in a team, etc. That's just bad luck - for 8 GB of "features", there were only 20 examples in the train and 5 in the test. The final nail in the coffin of the hackathon was driven by a leak in the data: the equipment logs received on Wednesday contained an error in the operation of the equipment, and those created on Thursday did not (by the way, only two teams knew about this, and both were from Russia, the homeland of experienced dataminers ). Although even knowing the true labels of the test did not help to adjust the answer - the task was unsolvable. The organizers did not get the desired result, the participants spent a lot of time solving a poorly formulated problem. The hackathon failed.
Morality
Conduct technical reviews of assignments and check your assignments for adequacy. It is better to overpay for a preliminary examination (in this case, any data scientist would immediately point out that it is impossible to solve this problem) than to regret later.
In this case, in addition to wasted time and money, the company lost credibility from potential candidates and possibly write about the results. By the way, not only participants should write about successful results, but also the company, implementing the hackathon as much as possible in terms of PR. Unfortunately, not all companies do this, limiting themselves to just a post-announcement and a couple of photos from the event on Twitter.
Hackathon #3. take it or leave it
Most recently, our team took part in a hackathon in Amsterdam. Since I am an electric power engineer by education (in the field of renewable energy sources), the topic was just for us - energy. The hackathon was held online: we were given a task description and a month to complete. The organizers wanted to see a finished project that would help increase the energy efficiency of Amsterdam's houses.
We made a project that predicts electricity consumption (before that, I participated in a competition on this topic where I received a near-sota solution, which you can read about ) and generation by a solar panel. Based on these predictions, battery performance is optimized (this idea was partly taken from my master's thesis). Our project was in good agreement both with the task from the organizers (as it seemed to us then), and with the policy of the Amsterdam administration in the field of renewable energy for several years to come.
During the evaluation of projects, we, like many teams, were told that this was not what the customer expected, while adding that we had to redo the project if we wanted to compete for the prize. We did not change anything, resigned to defeat. Of the forty participating teams, we did not even make it to the top 7, although the choice of the organizers, it seems to me, was rather strange. For example, they missed the final team, which made an application for calculating wind speed and solar radiation (SI) using smartphone sensors: a microphone for the wind, a light sensor for SI. The killer feature was the classification of hotdog / not hotdog into three classes: Sun, wind, water and displaying the corresponding Wikipedia article ().
Let's leave the moral side of the issue for a moment: blackmailing participants with the possibility of winning is simply unethical. Since one of the motivations for participating in hackathons (especially experienced developers) is to implement their ideas, many strong participants can simply leave the event after hearing such feedback (which happened not only with our team, but also with a number of others who stopped updating their page). project after listening to the mentor). Still, let's say we agreed with the wishes of the organizers and redesigned our project to meet their requirements. What might happen next?
Since the organizers have their own understanding of the “ideal project”, all wishes (and, accordingly, changes) will lead us to this ideal. The contestants will waste their time and it will be more and more difficult for them to refuse further participation (since forces have already been invested, and it seems that there is very little before victory). But in reality, the competition for prizes will increase, and the participants will increasingly have to rework the project with corrections from the organizers in the hope of winning a prize. As a result, the guys who did not take prizes, looking back, will realize that they participated in freelancing without money: they made changes to the customer, but at the same time did not receive anything in return for this (except for the corresponding experience, of course).
Morality
Often the wishes and feedback from the organizers come to the aid of the project. In doing so, however, participants should not rely on the advice of mentors like a lame man on a cane. If you hear feedback from the organizers on your project in the spirit of “remove it, we didn’t order it” - your participation in the hackathon can be considered completed.
If you are organizing a hackathon with a clear vision of the project, but without the skills or ability to implement it yourself, then it is better to format your vision in the form of a freelancer's terms of reference. Otherwise, you will have to pay twice - for the hackathon and for the services of a freelancer.
Source: habr.com