The translation of the article is specially prepared for the students of the course . Interested in developing in this area? Watch the master class by Yegor Zuev (Team Lead at InBit) and join the upcoming course group: starts September 26.
More and more people are switching to AWS Lambda for its scalability, performance, cost-effectiveness, and the ability to handle millions or even trillions of requests per month. There's no need to manage the infrastructure on which the service runs. Automatic scaling allows processing thousands of simultaneous requests per second. I think AWS Lambda can rightfully be called one of the most in-demand services of AWS.
AWS Lambda
AWS Lambda is an event-driven serverless computing service that allows you to run code without provisioning or managing servers, enhancing other AWS services based on user logic. Lambda automatically responds to various events (so-called triggers), such as HTTP requests via Amazon API Gateway, data changes in Amazon S3 buckets, or in Amazon DynamoDB tables; you can also invoke your code through API calls using AWS SDK and transitions between states in AWS Step Functions.
Lambda executes code on a highly available computing infrastructure, fully managing the underlying platform, including server and operating system maintenance, resource provisioning, automatic scaling, code monitoring, and logging. This means you only need to upload your code and set up when and how it should run. The service will take care of executing it and ensuring high availability of your application.
When should you switch to Lambda?
AWS Lambda is a convenient computing platform suitable for a multitude of use cases, of course, provided that the language and runtime environment of your code are supported by the service. If you want to focus on code and business logic, while delegating server maintenance, resource provisioning, and scaling to an external provider at a reasonable cost, then you should definitely switch to AWS Lambda.
Lambda is ideal for creating application interfaces, and when used alongside API Gateway, it can significantly reduce costs and speed up time to market. There are various ways to utilize Lambda functions and organize serverless architecture—everyone can find something suitable based on their goals.
Lambda enables a wide range of tasks. Thanks to CloudWatch support, you can create scheduled jobs and automate individual processes. There are no restrictions on the nature and intensity of service usage (memory consumption and time are considered), and nothing prevents you from steadily working on a full-fledged microservice based on Lambda.
Here, you can create service-oriented actions that are not executed continuously. A typical example is image scaling. Even in distributed systems, Lambda functions remain relevant.
So, if you don't want to deal with provisioning and managing computing resources—try AWS Lambda; if you don't need heavy, resource-intensive computations—also try AWS Lambda; if your code runs periodically—it's clear you should give AWS Lambda a try.
Security
So far, there have been no complaints regarding security. On the other hand, since many internal processes and implementation features of the AWS Lambda runtime environment are hidden from the user, some common cloud security rules lose relevance.
Like most AWS services, Lambda operates on the principle of shared responsibility between AWS and the client concerning security and compliance. This principle reduces the operational burden on the client, as AWS takes care of maintenance, administration, and control of service components—from host operating systems and virtualization layers to the physical security of infrastructure assets.
Specifically regarding AWS Lambda, AWS is responsible for managing the underlying infrastructure, associated core services, operating system, and application platform. Meanwhile, the client is responsible for the security of their code, storing sensitive data, controlling access to it, and to the Lambda service and resources (Identity and Access Management, IAM), including within the utilized functions.
The diagram below presents the shared responsibility model applicable to AWS Lambda. The scope of AWS's responsibility is highlighted in orange, while the client's responsibility is depicted in blue. As you can see, AWS takes on more responsibility for applications deployed on the service.
The shared responsibility model applicable to AWS Lambda
Lambda Runtime
The main advantage of Lambda is that by executing a function on your behalf, the service automatically allocates the necessary resources. You can save time and effort on system administration and focus on business logic and writing code.
The Lambda service is divided into two planes. The first is the control plane. According to Wikipedia, the control plane is the part of the network responsible for transporting signaling traffic and routing. It is the main component making global decisions about resource allocation, maintenance, and workload distribution. Additionally, the control plane acts as the network topology of the solution provider, responsible for traffic routing and management.
The second plane is the data plane. Like the control plane, it has its own tasks. The control plane provides APIs for managing functions (CreateFunction, UpdateFunctionCode) and regulates Lambda's interaction with other AWS services. The data plane manages the API calls (Invoke API) that trigger Lambda functions. After a function is invoked, the control plane allocates or selects an existing, pre-prepared runtime environment for that function, and then executes the code within it.
AWS Lambda supports multiple programming languages, including Java 8, Python 3.7, Go, NodeJS 8, .NET Core 2, and others through respective runtime environments. AWS regularly updates them, rolls out security patches, and performs maintenance operations on these environments. Lambda also allows the use of other languages as long as you implement the respective runtime yourself. In this case, you'll need to handle its maintenance, including monitoring security.
How does all this work, and how will the service execute your functions?
Each function operates in one or several dedicated environments, which exist only during the lifecycle of that function and are then destroyed. Only one invocation runs in each environment at a time, but it is reused when multiple serial invocations of the same function occur. All runtime environments operate on virtual machines with hardware virtualization — the so-called microVM. Each microVM is assigned to a specific AWS account and can be reused multiple times by environments to execute different functions within that account. MicroVMs are packaged into the structural blocks of the Lambda Worker hardware platform, which is owned and managed by AWS. The same runtime environment cannot be used by different functions, just as microVMs are unique to different AWS accounts.
Isolation Model in AWS Lambda
The isolation of runtime environments is implemented through several mechanisms. At the highest level, each environment has separate copies of the following components:
- Function Code
- Any Lambda layers selected for the function
- Function Runtime
- Minimal user space based on Amazon Linux
The following mechanisms are used to isolate different runtime environments:
- cgroups — restricting access to CPU, memory, storage bandwidth, and network resources for each runtime environment;
- namespaces — grouping together process IDs, user IDs, network interfaces, and other resources managed by the Linux kernel. Each runtime operates in its own namespace;
- seccomp-bpf — restricting the system calls that can be used in the runtime;
- iptables and routing tables — isolating runtime environments from each other;
- chroot — providing restricted access to the underlying file system.
Combined with proprietary isolation technologies from AWS, the listed mechanisms ensure reliable separation of execution environments. Environments isolated this way cannot access or modify the data of other environments.
Although multiple execution environments of a single AWS account can run on one microVM, under no circumstances can microVMs be shared between different AWS accounts. AWS Lambda utilizes two mechanisms for microVM isolation: EC2 instances and Firecracker. Guest isolation in Lambda based on EC2 instances has been in place since 2015. Firecracker is a new open-source hypervisor specifically designed by AWS for serverless workloads and was introduced in 2018. The physical hardware on which microVMs run is shared by workloads from different accounts.
Preserving environments and process states
Although Lambda execution environments are unique to different functions, the same function can be invoked repeatedly, meaning an execution environment can persist for several hours before it is destroyed.
Each Lambda execution environment also has a writable file system accessible via the /tmp directory. Its contents cannot be accessed from other execution environments. Regarding the preservation of process states, files written to /tmp exist for the entire lifecycle of the execution environment. This allows for the accumulation of results from multiple invocations, which is especially helpful for resource-intensive operations like loading machine learning models.
Data transfer during invocations
The Invoke API can be invoked in two modes: event mode and request-response mode. In event mode, the invocation is queued for later execution. In request-response mode, the function is called immediately with the provided payload, after which a response is returned. In both cases, the function executes in the Lambda environment, but with different payload pathways.
During request-response type calls, the payload comes from the Request Processing API (API Caller), such as AWS API Gateway or AWS SDK, to the load balancer, and then to the Lambda Invoke Service. The latter determines the appropriate environment for function execution and passes the payload there to complete the call. The load balancer receives traffic with TLS protection over the Internet. Traffic within the Lambda service—after the load balancer—travels through an internal VPC in a specific AWS region.
AWS Lambda Call Processing Model: Request-Response Mode
Event-driven calls can be executed immediately or queued. In some cases, the queue is implemented using the Amazon SQS (Amazon Simple Queue Service), which passes calls to the Lambda Invoke Service through an internal polling process (poller). The transmitted traffic is secured with TLS, while no additional data encryption for the information stored in Amazon SQS is provided.
Event-driven calls do not return responses—any response information from Lambda Worker is simply ignored. Calls from Amazon S3, Amazon SNS, CloudWatch, and other sources are processed by the Lambda service in event mode. Calls from Amazon Kinesis and DynamoDB streams, SQS queue calls, application load balancer calls, and API Gateway are processed in request-response mode.
Monitoring
You can monitor and audit Lambda functions using various AWS mechanisms and services, including the following.
Amazon CloudWatch
Collects various statistics such as the number of requests, the duration of request execution, and the number of requests that ended in error.
Amazon CloudTrail
Enables logging, continuous monitoring, and storing information about activities in your account related to your AWS infrastructure. You will have a complete history of actions performed using the AWS Management Console, AWS SDK, command line tools, and other AWS services.
AWS X-Ray
Provides full visibility of all stages of request processing in your application based on a map of its internal components. Allows you to analyze applications during development and in production environments.
AWS Config
You will be able to track changes in the configuration of Lambda functions (including their deletion) and runtime, tags, handler names, code size, memory allocation, timeout settings, and concurrency parameters, as well as the IAM execution role, subnet, and security group bindings.
Conclusion
AWS Lambda offers a powerful set of tools for building secure and scalable applications. Many security and compliance measures in AWS Lambda are similar to those used in other AWS services, although there are exceptions. As of March 2019, Lambda complies with SOC 1, SOC 2, SOC 3, PCI DSS, the U.S. Health Insurance Portability and Accountability Act (HIPAA), and other regulations. Therefore, when considering the implementation of your next application, consider AWS Lambda — it may be just the right fit for your needs.
Source: habr.com
