The translation of the article was prepared specifically for the students of the course . Are you interested in developing in this direction? Watch the master class of Egor Zuev (TeamLead at InBit) and join the next group of the course: start on September 26th.
More and more people are moving to AWS Lambda for the scalability, performance, cost savings, and the ability to process millions and even trillions of requests per month. To do this, you do not need to manage the infrastructure on which the service runs. And autoscaling allows you to serve thousands of concurrent requests per second. I think AWS Lambda can rightly be called one of the most requested AWS services.
AWS Lambda
AWS Lambda is an event-driven serverless computing service that allows you to run code without provisioning or administering servers and extend other AWS services based on custom logic. Lambda automatically responds to various events (so-called triggers), such as HTTP requests through Amazon API Gateway, data changes in Amazon S3 buckets, or Amazon DynamoDB tables; or you can run your code through API calls using the AWS SDK and state transitions in AWS Step Functions.
Lambda runs code on a highly available computing infrastructure and is fully responsible for the administration of the underlying platform, including server and operating system maintenance, resource provisioning, autoscaling, code monitoring, and logging. That is, you just need to upload your code and configure how and when it should be executed. In turn, the service will take care of its startup and make your application highly available.
When to switch to Lambda?
AWS Lambda is an easy-to-use computing platform suitable for many use cases, as long as your code's language and runtime is supported by the service, of course. If you want to focus on code and business logic while outsourcing server maintenance, provisioning, and scaling to a third-party vendor for a reasonable price, you should definitely consider switching to AWS Lambda.
Lambda is ideal for building APIs, and when used with API Gateway, you can significantly reduce costs and get to market faster. There are various ways to use Lambda functions and options for organizing a serverless architecture - everyone can choose something suitable for their goal.
Lambda allows you to perform a wide range of tasks. So, thanks to CloudWatch support, you can create pending tasks and automate individual processes. There are no restrictions on the nature and intensity of use of the service (memory consumption and time are taken into account), and nothing prevents you from systematically working on a full-fledged Lambda-based microservice.
Here you can create service-oriented actions that are not constantly performed. A typical example is image scaling. Even in the case of distributed systems, Lambda functions do not lose their relevance.
So, if you do not want to deal with the allocation and administration of computing resources, try AWS Lambda; if you do not need heavy, resource-intensive calculations, also try AWS Lambda; if your code is running periodically - everything is correct, you should try AWS Lambda.
Security
So far, there are no complaints about safety. On the other hand, because many of the internal processes and implementation details of this model are hidden from the user of the AWS Lambda managed runtime, some common cloud security rules become irrelevant.
Like most AWS services, Lambda is provided under the shared responsibility of AWS and the customer for security and compliance. This principle reduces the operational burden on the customer as AWS takes care of maintaining, administering, and monitoring the components of the service, from the host operating system and virtualization layer to the physical security of infrastructure assets.
Specifically with AWS Lambda, AWS is responsible for managing the underlying infrastructure, associated underlying services, operating system, and application platform. While the client is responsible for the security of his code, storage of confidential data, access control to them, as well as to the Lambda service and resources (Identity and Access Management, IAM), including within the limits of the functions used.
The diagram below shows the shared responsibility model that applies to AWS Lambda. The AWS area of ββresponsibility is in orange, while the customer's responsibility is in blue. As you can see, AWS is taking more responsibility for the applications that are deployed on the service.
Shared Responsibility Model Applicable to AWS Lambda
Lambda runtime
The main advantage of Lambda is that, by executing a function on your behalf, the service itself allocates the necessary resources. You can save time and effort on system administration and focus on business logic and coding.
The Lambda service is divided into two planes. The first is the control plane. According to Wikipedia, the control plane is the part of the network responsible for transporting signaling traffic and routing. It is the main component that makes global decisions about the allocation, maintenance and distribution of workloads. In addition, the control plane acts as the network topology of the solution provider responsible for routing and managing traffic.
The second plane is the data plane. She, like the control plane, has its own tasks. The control plane provides APIs for managing functions (CreateFunction, UpdateFunctionCode) and controls how Lambda interacts with other AWS services. The data plane manages the Invoke API, which runs the Lambda functions. After a function is called, the control plane either allocates or selects an existing runtime environment prepared for that function, and then executes the code in it.
AWS Lambda supports many programming languages ββincluding Java 8, Python 3.7, Go, NodeJS 8, .NET Core 2, and more through their respective runtimes. AWS regularly updates them, distributes security patches, and performs other maintenance activities on these environments. Lambda allows you to use other languages, as long as you implement the appropriate runtime yourself. And then you will have to deal with its maintenance, including monitoring security.
How does it all work and how will the service perform your functions?
Each feature runs in one or more dedicated environments that exist only for the duration of that feature's lifecycle and then are destroyed. Each environment only makes one call at a time, but it is reused if there are many serial calls to the same function. All runtimes run on virtual machines with hardware virtualization, so-called microVMs. Each microVM is assigned to a specific AWS account and can be reused by environments to perform various functions within that account. MicroVMs are packaged into the building blocks of a Lambda Worker hardware platform owned and operated by AWS. The same runtime cannot be used by different functions, nor are microVMs unique to different AWS accounts.
Isolation Model in AWS Lambda
Runtime isolation is implemented using several mechanisms. At the top level of each environment, there are separate copies of the following components:
- Function code
- Any Lambda layers selected for the function
- Function runtime
- Minimum user space based on Amazon Linux
The following mechanisms are used to isolate different runtimes:
- cgroups - restrict access to CPU, memory, storage and network bandwidth per runtime environment;
- namespaces is a grouping of process IDs, user IDs, network interfaces, and other resources managed by the Linux kernel. Each runtime runs in its own namespace;
- seccomp-bpf - Limits the system calls that can be used in the runtime.
- iptables and routing tables - isolation of runtime environments among themselves;
- chroot - granting limited access to the underlying file system.
Combined with AWS proprietary isolation technologies, these mechanisms ensure that runtime environments are securely separated. Environments isolated in this way cannot access or modify other environments' data.
While multiple runtimes of the same AWS account can run on the same microVM, under no circumstances can microVMs be shared between different AWS accounts. AWS Lambda uses only two mechanisms to isolate microVMs: EC2 instances and Firecracker. Guest isolation in Lambda based on EC2 instances has been around since 2015. Firecracker is a new open source hypervisor specifically developed by AWS for serverless workloads and introduced in 2018. The physical hardware that microVMs run on is shared between workloads from different accounts.
Saving environments and process states
Although Lambda runtimes are unique across functions, they can call the same function repeatedly, meaning the runtime can live for several hours before being destroyed.
Each Lambda runtime also has a writable filesystem accessible through the /tmp directory. Its contents cannot be accessed from other runtimes. As far as process state persistence is concerned, files written to /tmp exist for the lifetime of the runtime. This allows the results of multiple calls to be accumulated, which is especially useful for expensive operations such as loading machine learning models.
Transferring call data
The Invoke API can be used in two modes: event mode and request/response mode. In event mode, the call is queued for later execution. In request-response mode, the function is called instantly with the provided payload, after which the response is returned. In both cases, the function is executed in the Lambda environment, but with different payload paths.
During request-response calls, the payload travels from an API caller (API Caller), such as AWS API Gateway or AWS SDK, to a load balancer, and then to a Lambda call service (Invoke Service). The latter determines the appropriate environment for the function to execute and passes the payload there to complete the call. The load balancer receives TLS-protected traffic over the Internet. Traffic within the Lambda service - after the load balancer - passes through an internal VPC in a specific AWS region.
AWS Lambda Call Processing Model: Request-Response Mode
Event calls can be made immediately or added to a queue. In some cases, the queue is implemented using Amazon Simple Queue Service (SQS), which passes calls to the Lambda call fulfillment service through an internal poller process. The transmitted traffic is protected by TLS, while there is no additional encryption of data stored in Amazon SQS.
Event calls do not return responses - Lambda Worker simply ignores any response information. Calls based on events from Amazon S3, Amazon SNS, CloudWatch, and other sources are handled by the Lambda service in event mode. Calls from Amazon Kinesis and DynamoDB threads, calls to SQS Queues, Application Load Balancer, and API Gateway are handled in a request-response mode.
Monitoring
You can monitor and audit Lambda functions using a variety of AWS mechanisms and services, including the following.
Amazon Cloud Watch
Collects various statistics such as the number of requests, the duration of requests, and the number of requests that failed.
Amazon Cloud Trail
Allows you to log, continuously monitor, and save account activity related to your AWS infrastructure. You will have a complete history of activities performed using the AWS Management Console, AWS SDK, command line tools, and other AWS services.
AWS X Ray
Provides complete visibility into all stages of request processing in your application based on a map of its internal components. Allows you to analyze applications during development and in a production environment.
AWSConfig
You will be able to monitor changes to the configuration of Lambda functions (including their removal) and runtimes, tags, handler names, code size, memory allocation, timeout settings, and concurrency settings, as well as the Lambda IAM execution role, subnetting, and security group associations.
Conclusion
AWS Lambda provides a powerful set of tools for building secure and scalable applications. Many of the security and compliance practices in AWS Lambda are the same as those used in other AWS services, although there are exceptions. As of March 2019, Lambda complies with SOC 1, SOC 2, SOC 3, PCI DSS, HIPAA, and other regulations. So when you think about implementing your next application, consider AWS Lambda service - it might be the best fit for your task.
Source: habr.com