I work in outsourcing, where the main principle can be described by the phrase "sell a lot, do it quickly." The faster we do it, the more we earn. And, it is desirable that everything works not on crutches and snot, but with an acceptable level of quality. I will tell you about my experience when it was necessary to develop a promotional service in a short time.
Given: root account on AWS, no restrictions on the choice of technology stack, one backender, and one month for development.
Problem: implement a promotional service where human users upload from one to four videos lasting from one to four seconds, which are then embedded in the original video sequence.
Solution
Writing your bike service in such a short time is a so-so idea. In addition, in order for the service to cope with the loads and for everyone to receive the coveted video, infrastructure will be required. And preferably not with a price tag from an airplane. Therefore, we immediately focus on ready-made solutions with minimal customization.
The standard solution for working with video is FFmpeg, a cross-platform console utility that, through arguments, allows you to cut and overlay sound. It's up to the small thing - to write a wrapper and release it into life. We write a prototype that glues two videos, and ... the fun begins. The library on .NET Core 2 should run on any virtual machine, so we take an AWS EC2 instance and it will work
Hidden textno, it won't work
.
Although FFmpeg simplifies the task, for a really working solution you need to create an EC2 instance, design a network infrastructure for it, including a Load Balancer. The simple task of deploying from scratch becomes “a little” more complicated, and the infrastructure begins to demand money immediately - every hour the amount for runtime is withdrawn from the client account.
Our service does not involve Long-Running processes, does not require a large and fat relational database, and fits perfectly into an event-driven architecture with a chain of microservice calls. The solution suggests itself - we can abandon EC2 and implement a true-serverless application, like the standard Image Resizers based on AWS Lambda.
By the way, despite the obvious dislike of AWS developers for .NET, they support .NET Core 2.1 as a runtime, which gives a full range of development opportunities.
And the icing on the cake - AWS provides a separate service for working with video files - AWS Elemental MediaConvert.
The essence of the work is incredibly simple: we take the S3 link to the outgoing video, through the AWS Console, .NET SDK or just JSON we write what we want to do with the video and call the service. It itself implements queues for processing incoming requests, uploads the result to S3 itself, and, most importantly, generates a CloudWatch Event for each status change. This allows us to implement lambda triggers to complete video processing.
This is what the final architecture looks like
The entire backend is housed in two lambdas. Another one is for rotating vertical videos, since such work cannot be done in one pass.
The front in the form of a SPA application written in JS and compiled using pug will be placed in a public S3 bucket. To download the videos themselves, we do not need any server code - just open the REST endpoints that S3 suggests to us. The only point - do not forget to configure policies and CORS.
Pitfalls
- AWS MediaConvert, for some unknown reason, overlays sound only on each video segment individually, and we need a perky song from the entire intro.
- vertical videos need to be processed separately. AWS doesn't like black bars and puts rollers at 90°.
Easy ice rink
Despite all the beauty of Stateless, you need to keep track of what you want to do with the video: glue or overlay sound on an already finished video sequence. Luckily, MediaConvert supports passing metadata through its Jobs, and we can always apply a simple flag like “isMasterSoundJob” to parse this metadata at any stage.
Serverless is a great way to work with NoOps, an approach that eliminates the need for a separate team responsible for the project infrastructure. Therefore, the matter was small - we deploy the solution on AWS without the participation of system administrators, who always have something to do anyway.
And in order to speed up all this, we automate the deployment script on AWS CloudFormation as much as possible, which allows deploying with one button directly from VS. As a result, a file of 200 lines of code allows you to roll out a ready-made solution, although the CloudFormation syntax can be shocking from the habit.
Total
Serverless is not a panacea. But it will make life much easier in situations with three limits: "limited resources - short time - little money."
Characteristics of Applications Suitable for Serverless
- without long-running processes. API Gateway hard limit - 29 seconds, lambda hard limit - 5 minutes;
- described by Event-Driven architecture;
- broken down into loosely coupled components by SOA type;
- does not require much work with his condition;
- written in .NET Core. To work with the .NET Framework, you will still need at least Docker with the appropriate runtime.
Benefits of the Serverless Approach
- reduces infrastructure costs;
- reduces the cost of delivering the solution;
- automatic scalability;
- development at the forefront of technological progress.
Disadvantages, on a specific example
- Distributed tracing and logging - partially solved through AWS X-Ray and AWS CloudWatch;
- inconvenient debugging;
- Cold Start at no load;
- The user-hostile interface of AWS is a universal problem :)
Source: habr.com