In this tutorial, we will walk through the creation of a drone program with voice control using Node.js and the Web speech API. Copter - Parrot ARDrone 2.0.
We remind you: for all readers of "Habr" - a discount of 10 rubles when enrolling in any Skillbox course using the "Habr" promotional code.
Skillbox recommends: Practical course .
Introduction
Drones are amazing. I really enjoy playing with my quadcopter, taking photos and videos, or just having fun. But unmanned aerial vehicles (UAVs) are used for more than just entertainment. They work in cinema, study glaciers, are used by the military and representatives of the agricultural sector.
In this tutorial, we will walk through the creation of a program that will allow you to control the drone. using voice commands. Yes, the copter will do what you tell it to. At the end of the article - the finished program and video of the UAV control.
Hardware
We need the following:
- Parrot ARDrone 2.0;
- Ethernet cable;
- good microphone.
Development and management will be carried out on workstations with Windows/Mac/Ubuntu. Personally, I have worked with Mac and Ubuntu 18.04.
Software
Download the latest version of Node.js from .
Also need .
Dealing with the copter
Let's try to understand how the Parrot ARDrone works. This quad has four motors.
The opposing motors run in the same direction. One pair rotates clockwise, the other counter-clockwise. The drone moves by changing the angle of inclination relative to the ground, changing the speed of rotation of the motors, and a few more maneuvering movements.
As we can see in the diagram above, changing various parameters leads to a change in the direction of movement of the copter. For example, a decrease or increase in the speed of rotation of the left and right rotors creates a roll. This allows the drone to fly forward or backward.
By changing the speed and direction of movement of the motors, we set the angles of inclination, allowing the copter to move in other directions. Actually, for the current project, you donβt need to study aerodynamics, you just need to understand the basic principles.
How the Parrot ARDrone works
The drone is a Wi-Fi hotspot. In order to receive and send commands to the copter, you need to connect to this point. There are many different applications that allow you to control quadcopters. It all looks like this:
Once the drone is connected, open the terminal and telnet 192.168.1.1 is the IP of the copter. For Linux you can use .
Application architecture
Our code will be divided into the following modules:
- user interface with speech API for voice detection;
- command filtering and comparison with the standard;
- sending commands to the drone;
- live video broadcast.
The API works subject to an internet connection. To provide it, we add an Ethernet connection.
It's time to create an app!
Kodim
First, let's create a new folder and switch to it using the terminal.
Then we create a Node project using the following commands.
First, install the required dependencies.
npm installβ
We will support the following commands:
- takeoff;
- landing;
- up β the drone rises half a meter and freezes;
- down - drops by half a meter and freezes;
- left - goes half a meter to the left;
- to the right - goes half a meter to the right;
- rotation - rotates clockwise by 90 degrees;
- forward - goes forward half a meter;
- back - goes back half a meter;
- Stop.
Here is the code that allows you to receive commands, filter them and control the drone.
const express = require('express');
const bodyparser = require('body-parser');
var arDrone = require('ar-drone');
const router = express.Router();
const app = express();
const commands = ['takeoff', 'land','up','down','goleft','goright','turn','goforward','gobackward','stop'];
var drone = arDrone.createClient();
// disable emergency
drone.disableEmergency();
// express
app.use(bodyparser.json());
app.use(express.static(__dirname + '/public'));
router.get('/',(req,res) => {
res.sendFile('index.html');
});
router.post('/command',(req,res) => {
console.log('command recieved ', req.body);
console.log('existing commands', commands);
let command = req.body.command.replace(/ /g,'');
if(commands.indexOf(command) !== -1) {
switch(command.toUpperCase()) {
case "TAKEOFF":
console.log('taking off the drone');
drone.takeoff();
break;
case "LAND":
console.log('landing the drone');
drone.land();
break;
case "UP":
console.log('taking the drone up half meter');
drone.up(0.2);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "DOWN":
console.log('taking the drone down half meter');
drone.down(0.2);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "GOLEFT":
console.log('taking the drone left 1 meter');
drone.left(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},1000);
break;
case "GORIGHT":
console.log('taking the drone right 1 meter');
drone.right(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},1000);
break;
case "TURN":
console.log('turning the drone');
drone.clockwise(0.4);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "GOFORWARD":
console.log('moving the drone forward by 1 meter');
drone.front(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "GOBACKWARD":
console.log('moving the drone backward 1 meter');
drone.back(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "STOP":
drone.stop();
break;
default:
break;
}
}
res.send('OK');
});
app.use('/',router);
app.listen(process.env.port || 3000);And here is the HTML and JavaScript code that listens to the user and sends a command to the Node server.
<!DOCTYPE html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>Voice Controlled Notes App</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/shoelace-css/1.0.0-beta16/shoelace.css">
<link rel="stylesheet" href="styles.css">
</head>
<body>
<div class="container">
<h1>Voice Controlled Drone</h1>
<p class="page-description">A tiny app that allows you to control AR drone using voice</p>
<h3 class="no-browser-support">Sorry, Your Browser Doesn't Support the Web Speech API. Try Opening This Demo In Google Chrome.</h3>
<div class="app">
<h3>Give the command</h3>
<div class="input-single">
<textarea id="note-textarea" placeholder="Create a new note by typing or using voice recognition." rows="6"></textarea>
</div>
<button id="start-record-btn" title="Start Recording">Start Recognition</button>
<button id="pause-record-btn" title="Pause Recording">Pause Recognition</button>
<p id="recording-instructions">Press the <strong>Start Recognition</strong> button and allow access.</p>
</div>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script src="script.js"></script>
</body>
</html>And more JavaScript code to work with voice commands by sending them to the Node.
try {
var SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
var recognition = new SpeechRecognition();
}
catch(e) {
console.error(e);
$('.no-browser-support').show();
$('.app').hide();
}
// other code, please refer GitHub source
recognition.onresult = function(event) {
// event is a SpeechRecognitionEvent object.
// It holds all the lines we have captured so far.
// We only need the current one.
var current = event.resultIndex;
// Get a transcript of what was said.
var transcript = event.results[current][0].transcript;
// send it to the backend
$.ajax({
type: 'POST',
url: '/command/',
data: JSON.stringify({command: transcript}),
success: function(data) { console.log(data) },
contentType: "application/json",
dataType: 'json'
});
};Run the application
The program can be launched as follows (it is important to make sure that the aircraft is connected to Wi-Fi and the Ethernet cable is connected to the computer).
Open localhost:3000 in a browser and click Start Recognition.
We try to control the drone and rejoice.
Drone video streaming
In the project, create a new file and copy this code there:
const http = require("http");
const drone = require("dronestream");
const server = http.createServer(function(req, res) {
require("fs").createReadStream(__dirname + "/public/video.html").pipe(res);
});
drone.listen(server);
server.listen(4000);And here is the HTML code, we place it inside the public folder.
<!doctype html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<title>Stream as module</title>
<script src="/dronestream/nodecopter-client.js" type="text/javascript" charset="utf-8"></script>
</head>
<body>
<h1 id="heading">Drone video stream</h1>
<div id="droneStream" style="width: 640px; height: 360px"> </div>
<script type="text/javascript" charset="utf-8">
new NodecopterStream(document.getElementById("droneStream"));
</script>
</body>
</html>Run and connect to localhost:8080 to view video from the front camera.
Useful Tips
- Control this drone indoors.
- Always put the protective cover on the drone before taking off.
- Check if the battery is charged.
- If the drone is behaving strangely, hold it from below and turn it over. This action will put the aircraft into emergency mode and the rotors will stop immediately.
Ready code and demo
Happened!
Write code and then watch how the machine starts to obey, it will give you pleasure! Now we figured out how to teach the drone to listen to voice commands. In fact, there are many more possibilities: face recognition of the user, autonomous flights, gesture recognition and much more.
What can you suggest to improve the program?
Skillbox recommends:
- Applied online course .
- Online course .
- Practical year course .
Source: habr.com