Last December, we held our own startup hackathon with six other Skolkovo companies. Without corporate sponsors or any external support, we gathered two hundred participants from 20 cities of Russia through the efforts of the programming community. Below I will tell you how we succeeded, what pitfalls we encountered along the way, and why we immediately began collaborating with one of the winning teams.
Interface of the application that controls Watts Battery modules from the finalists of the track, “Wet Hair”
Company
Our company Watts Battery creates modular portable power stations. The product is a portable power station 46x36x11 cm, capable of delivering from 1,5 to 15 kilowatts per hour. Four such modules can provide the energy consumption of a small country house for two days.
Although we began shipping production samples last year, by all accounts Watts Battery is a startup. The company was founded in 2016 and since the same year has been a resident of the Skolkovo Energy Efficient Technologies Cluster. Today we have 15 employees and a huge backlog of things that we would like to do at some stage, but right now there is no time for that.
This also includes purely software tasks. Why?
The main task of the module is to provide uninterrupted, balanced energy supply at an optimal cost. If you experience a power outage due to reasons beyond your control, you should always have a reserve in order to fully power the required network load for the duration of the outage. And when the power supply is good, you can use solar energy to save money.
The simplest option is that you can charge the battery from the sun during the day and use it in the evening, but exactly to the level that is necessary so that in the event of a blackout, you are not left without electricity. So, you will never find yourself in a situation where you powered the lighting from a battery all evening (because it’s cheaper), but at night the electricity went out and your refrigerator defrosted.
It is clear that a person is rarely able to predict with great accuracy the amount of electricity he needs, but a system armed with a predictive model can. Therefore, machine learning as such is one of our priority areas. It’s just that we are currently focused on hardware development and cannot allocate enough resources to these tasks, which is what brought us to the Startup Hackathon.
Preparation, data, infrastructure
As a result, we took two tracks: data analytics and management system. In addition to ours, there were seven more tracks from colleagues.
While the format of the hackathon was not determined, we were thinking of creating “our own atmosphere”, with a point system: participants do some things that seem difficult and interesting to us, receiving points for it. We had a lot of tasks. But as we built the structure of the hackathon, other organizers asked to bring everything to a common form, which we did.
Then we came to the following scheme: the guys make a model based on their data, then they receive our data, which the model had not seen before, it learns and begins to predict. It was assumed that all this could be done in 48 hours, but for us this was the first hackathon on our data, and we may have overestimated the time resources or the degree of readiness of the data. At specialized machine learning hackathons, such a timeline would be the norm, but ours was not like that.
We unloaded the software and hardware of the module as much as possible, and made a version of our device specifically for the hackathon, with a very simple and understandable internal interface that any developer could support.
For the track based on the control system, there was an option to make a mobile application. To prevent the participants from racking their brains about what it should look like and wasting extra time, we gave them a design layout of the application, super-lightweight, so that those who want it could simply “stretch” the functions they need onto it. To be honest, we didn’t expect any moral dilemmas here, but one of the teams took it in such a way that we were limiting their flight of fancy, we wanted to get a ready-made solution for free, and not test them in practice. And they took off.
Another team chose to make a completely different application from scratch, and everything worked out. We did not insist that the application be exactly like this, we just needed it to contain some elements that demonstrate the technical level of the solution: graphs, analytics, etc. The finished design layout was also a hint.
Since analyzing a live Watts Battery module at a hackathon would be too time-consuming, we gave participants a ready-made slice of data for a month taken from our clients’ real modules (which we carefully anonymized beforehand). Since it was June, there was nothing to incorporate seasonal changes into the analysis. But in the future we will add external data to them, such as seasonal and climatic features (today this is the industry standard).
We didn’t want to create unrealistic expectations among the participants, so in the announcement of the hackathon we directly said: the work will be as close as possible to the field work: noisy, dirty data, which no one specially prepared. But this also had a positive side: in the spirit of agile, we were constantly in contact with the participants, and promptly made changes to the task and conditions of admission (more on this below).
In addition, we gave participants access to Amazon AWS (so actively that Amazon blocked one region for us, we’ll figure out what to do about it). There you can deploy infrastructure for the Internet of Things and, based on even simple Amazon templates, create a full-fledged solution within a day. But in the end, absolutely everyone went their own way, doing everything on their own to the maximum. At the same time, some managed to meet the time limit, others did not. One team, Nubble, used Yandex.cloud, someone raised it on their hosting. We were even ready to give domains (we have registered ones), but they were not useful.
To determine the winners in the analytical track, we planned to compare the results, for which we prepared numerical metrics. But in the end it was not necessary to do this, since for various reasons three of the four participants did not reach the final.
As for the household infrastructure, the Skolkovo Technopark helped here by providing us (free of charge) with one of its cozy modular rooms with a video wall for presentations and a couple of smaller rooms for a recreation area and for organizing catering.
Analytics
Task: a self-learning system that identifies anomalies in consumption and module operation based on control data. We deliberately kept the wording as general as possible so that participants could work with us to think about what could be done based on the available data.
Specificity: The more complex of the two tracks. Industrial data has some differences from data in closed systems (for example, digital marketing). Here you need to understand the physical nature of the parameters that you are trying to analyze; looking at everything as abstract number series will not work. For example, the distribution of electricity consumption throughout the day. It’s like rituals: the electric razor is turned on in the morning on weekdays, and the mixer is turned on on weekends. Then the essence of the anomalies themselves. And don’t forget that the Watts Battery is intended for personal use, so each client will have their own rituals, and one universal model will not work. Finding known anomalies in data is not even a task; creating a system that autonomously searches for unlabeled anomalies is another matter. After all, anything can be an anomaly, including the insidious human factor. For example, in our test data there was a case where the system was forced by the user into battery mode. Without any reason, users sometimes do this (I’ll make a reservation that this user is testing the module for us and it is for this reason that he has access to manual control of modes; for other users the control is completely automatic). As is easy to predict, in such a situation the battery is discharged quite actively, and if the load is large, the charge will end before the sun rises or another source of energy appears. In such cases, we expect to see some kind of notification that the system behavior has deviated from the normal one. Or the person left and forgot to turn off the oven. The system sees that usually at this time of day the consumption is 500 watts, but today - 3,5 thousand - an anomaly! Like Denis Matsuev on the plane: “I don’t understand anything about aircraft engines, but on the way there the engine sounded different.”
Graph of a predictive model on the opensource neural network Yandex CatBoost
What does the company really need?: self-diagnostic system inside the device, predictive analytics, including without network infrastructure (as practice shows, not all of our clients are in a hurry to connect batteries to the Internet - for most, it’s enough for everything to just work reliably), identification of anomalies, the nature of which we do not yet know , a self-learning system without a teacher, clustering, neural networks and the entire arsenal of modern analytical methods. We need to understand that the system began to behave differently, even if we do not know what exactly has changed. At the hackathon itself, it was very important for us to see that there are guys who are ready to step into industrial analytics or are already in it, and they are looking for new areas to apply their abilities. At first I was surprised that there were so many applicants: after all, this is a very specific cuisine, but gradually all but one of the four participants dropped out, so to some extent everything fell into place.
Why is it not feasible at this stage?: The main problem with data mining tasks is not enough data. There are several dozen Watts Battery devices in operation around the world today, but many of them are not connected to the network, so our data is not yet very diverse. We barely scraped together two anomalies - and those occurred on prototypes; industrial Watts Battery work quite stably. If we had an internal machine learning engineer, and we knew - yes, this can be squeezed out of this data, but we want to get a better quality of prediction - it would be one story. But up to this point we have not done anything with this data. In addition, this would require a deep immersion of the participants in the specifics of the operation of our product; a day and a half is not enough for this.
How did you decide?: They didn’t immediately set the exact final task. Instead, throughout the entire 48 hours, we were in dialogue with the participants, promptly finding out what they were able to get and what they could not. Based on this, in the spirit of compromise, the task was finalized.
What did you get as a result?: the winners of the track were able to clean up the data (at the same time they found the “features” of calculating some parameters that we ourselves had not noticed before, since we did not use some of the data to solve our problems), highlight deviations from the expected behavior of Watts Battery modules, and set up a predictive model that able to predict energy consumption with a high degree of accuracy. Yes, this is only a feasibility stage of developing an industrial solution; then weeks of painstaking technical work will be needed, but even this prototype, created directly during the hackathon, can form the basis of a real industrial solution, which is rare.
The main conclusion: Based on the data we have, it is possible to set up predictive analytics, we assumed this, but did not have the resources to check. The hackathon participants tested and confirmed our hypothesis, and we will continue to work with the track winners on this task.
Graph of a predictive model on the opensource neural network Facebook Prophet
Advice for the future: when drawing up a task, you need to look not only at your production roadmap, but also at the interest of the participants. Since our hackathon has no cash prizes, we play on the natural curiosity of data scientists and the desire to solve new, interesting problems in which no one has yet shown anything or where they can show themselves better than existing results. If you immediately take into account the factor of interest, you won’t have to shift your focus along the way.
Management
Task: (application) that manages a network of Watts Battery modules, with a personal account, data storage in the cloud, and status monitoring.
Specificity: in this track we were not looking for some new technical solution; we, of course, have our own consumer interface. We chose him for the hackathon to demonstrate the capabilities of our system, immerse ourselves in it, and check whether the community is interested in the topic of development for smart systems and alternative energy. We positioned the mobile application as an option; you could do it or not do it at your discretion. But in our opinion, it shows well how people managed to organize data storage in the cloud, with access from several different sources at once.
What does the company really need?: a community of developers who will come up with business ideas, test hypotheses and create working tools for their implementation.
Why is it not feasible at this stage?: The market volume is still too small for the organic formation of such a community.
How did you decide?: As part of a hackathon, we conducted a kind of physicality study to see if it was possible to come up with not just features, but full-fledged business models around our very specific product. Moreover, in order for people capable of implementing a prototype to do this, after all, here - I don’t want to offend anyone - this is not the level of programming a blinking LED on Arduino (although this can be done with innovations), rather specific skills are required here: development of backend and frontend systems, understanding of the principles of building scalable Internet of Things systems.
What did you get as a result?: two teams proposed full-fledged business ideas for their work: one focused more on the Russian segment, the other on the foreign one. That is, in the finale they didn’t just tell how they came up with the application, but essentially came to do business around Watts. The guys outlined how they see the use of Watts in several business models, provided statistics, showed which regions have what problems, what laws are adopted where, outlined the global trend: it is unfashionable to mine bitcoins, it is fashionable to mine kilowatts. They deliberately came to alternative energy, which we really liked. The fact that the participants, in addition to this, were able to create a working technical solution suggests that they can launch a startup on their own.
The main conclusion: There are teams ready to take Watts Battery as the basis of their business model, develop it, and become partners/companions of the company. Some of them even know how to identify the MVP of a business idea and work on it first, something that is lacking everywhere in the industry today. People don’t understand when to stop, when to release a solution to the market, albeit early, but working. In fact, the stage of polishing the solution often does not end, technically the solution crosses the line of reasonable complexity, it enters the market overloaded, it is no longer clear what the original idea was, what customer targeting is, what business models are included. As in the joke about Akunin, who wrote another book while signing the previous one for someone. But here it was done in its purest form: here is a chart, here is a counter, here are indicators, here is a prediction - that’s all, nothing else is needed to run it. With this, you can go to an investor and receive money to start a business. Those who found this balance came out of the track as winners.
Advice for the future: at the next hackathon (we are planning it ), perhaps it makes sense to experiment with hardware. We have our own hardware development (one of the advantages of Watts), we fully control the production and testing of everything we do, but we do not have enough resources to test some “hardware” hypotheses. It may very well be that in the community of system and low-level programmers and hardware developers there are those who will help us with this and in the future will become our partner in this area.
People
At the hackathon, we expected those who want to try themselves in a new field (for example, graduates of various programming schools) rather than those who specialize in this kind of development. But still, we expected that before the hackathon they would do a little preparatory work, read about how energy consumption is predicted in general and how Internet of Things systems work. So that everyone comes not just for fun, looking for interesting data and tasks, but also with a preliminary immersion in the subject area. For our part, we understand that for this it is necessary to publish in advance the available data, their description and more precise requirements for the result, publish API modules, etc.
Everyone had approximately the same technological level, plus or minus the same capabilities. Against this background, the level of harmony was not the last factor. A number of teams did not shoot because they could not clearly divide themselves into areas of work. There were also those in which one person did all the development, the rest were busy preparing the presentation, in others, someone was given tasks that they were doing, probably for the first time in their lives.
Most of the participants were young, this does not mean that there were no strong machine learning engineers and developers among them. Most came in teams; there were practically no individuals. Everyone dreamed of winning, someone wanted to find a job in the future, about 20% have already found one, I think this figure will grow.
We didn’t have enough hardware geeks, but we hope to make up for it at the second hackathon.
Hackathon progress
As I wrote above, we were with the participants for most of the 48 hours of the hackathon and, monitoring their successes at checkpoints, tried to adapt the task and conditions for accepting the first, analytical track so that, on the one hand, the participants could complete it in the remaining time time, and on the other hand, it was of interest to us.
The last clarification to the task was made somewhere around the last checkpoint, on Saturday afternoon (the final was scheduled for Sunday evening). We simplified everything a little more: we removed the requirement to recalculate the model on new data, leaving the data that the teams were already working with. Comparing metrics no longer gave us anything, they already had ready-made results based on the available data, and by the second day the guys were already tired. Therefore, we decided to torture them less.
However, three out of four participants did not reach the finals. One team already realized at the start that they were more interested in the track of our colleagues, the other, just before the final, realized that during the processing process they had filtered out the necessary data ahead of time and refused to present their work.
The “21 (Wet Hair Effect)” team participated in both of our tracks until the very end. They wanted to cover everything at once: machine learning, development, application, and website. Until we threatened them with withdrawal at the last moment, they believed that they were doing everything in time, although already at the second checkpoint it was obvious that with the main thing - machine learning - they could not make significant progress: they generally coped with the second block, but could not predict electricity consumption were not ready. As a result, when we determined the minimum task for qualifying for the first, they still chose the second track.
Fit-predict had a balanced composition tailored for data analytics, so they were able to overcome everything. It was noticeable that the guys were interested in “touching” real industrial data. They immediately concentrated on the main thing: analyzing, cleaning the data, dealing with every anomaly. The fact that they were able to build a working model during the hackathon is a great achievement. In working practice, this usually takes weeks: while the data is being cleaned, while they are delving into it. Therefore, we will definitely work with them.
In the second track (management), we expected everyone to do everything in half a day and come ask to make the task more difficult. In practice, we barely had time to complete the basic task. We worked on JS and Python, which reflects the current state of the industry.
Here, too, the results were achieved by well-coordinated teams in which the division of labor was built, it was clear who was doing what.
The third team, FSociety, seemed to have a solution, but in the end they decided not to show their development, they said that they did not consider it to work. We respect this and did not argue.
The winner was the team “Strippers from Baku”, which was able to stop itself, not to chase after “trinkets”, but to create an MVP that is not ashamed to show and which is clear that it can be further developed and scaled. We immediately told them that we were not too interested in additional opportunities. If they want registration via QR code, facial recognition, let them first make graphs in the application, and then take on the optional ones.
In this track, “Wet Hair” confidently entered the final, and we discussed further cooperation with them and “Hustlers.” We have already met the latter in the new year.
I hope everything works out, and we look forward to seeing everyone at the second hackathon in March!
Source: habr.com