Building our own serverless based on Fn

Building our own serverless based on Fn

Serverless computing is one of the most notable trends in cloud computing. The basic principle of work is that the infrastructure is not the concern of DevOps, but of the service provider. Resource scaling automatically adjusts to load and has a high rate of change.

Another common feature is the tendency to minimize and focus code, which is why serverless computing is sometimes referred to as "function as a service" (FaaS).

Historically, the first cloud provider to offer FaaS with AWS Lambda was Amazon, which is where the name comes from. Other cloud service providers also offer equivalents:

  • Cloud Functions by Google
  • Microsoft Azure Functions

All of these companies provide serverless computing, autoscaling, and pay-only-as-usage, yet they tie customers to their proprietary product. However, there are free and open source alternatives for serverless computing. It is worth noting:

  • Platform Apache OpenWhisk, developed in an incubator by IBM,
  • Spring Cloud Functions, as part of a fairly rich Spring Framework ecosystem that can also be used as an AWS Lambda, Azure Functions, and OpenWhisk facade,
  • Project Fn, maintained by Oracle.

All of them are completely independent of the clouds, that is, they are installed in any cloud, including your own, public or private, and of course in Exoscale.

How the Fn project works

Fn is completely based on Docker, consists of two main components:

  • CLI program designed to manage all aspects of the Fn infrastructure, and interact with the Fn server,
  • The Fn server itself, a regular application packaged in a Docker container.

Functions deployed in Fn also run in separate containers, which allows you to support quite a lot of programming languages, for example ... Clojure!

Function arguments are passed to standard input (STDIN), results are written to standard output (STDOUT). If the arguments or return values ​​are not simple values ​​(for example, a JSON object), they can be converted using the abstraction layer provided by Fn itself as a Function Development Kit (FDK).

For convenience, built-in sets of templates are offered to facilitate the deployment of FaaS in an extensive list of different languages ​​\uXNUMXb\uXNUMXband their versions (Go, different versions of Java, Python, etc.).

Creating FaaS is simple, following this pattern:

  • Deploying a function using the CLI Fn: an application configuration file for Fn is created based on the selected template.
  • We roll out our own function, again using the CLI Fn: the container image is placed in a certain repository, after which the server is notified of the existence and placement of this image.

Building our own serverless based on Fn
The principle of supplying functions in Fn

Local installation and testing of serverless features

Let's start installing Fn on the local machine. Docker is installed first, as required by Fn. Assuming we are on Debian/Ubuntu:

$ sudo apt-get update
$ sudo apt-get install docker.io

Or use a Docker package manager/build according to your system. Then you can go straight to installing the Fn CLI. For example, using curl:

$ curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh

If you're on OSX with Homebrew installed, you can go the other way:

$ brew install fn

==> Downloading https://homebrew.bintray.com/bottles/fn-0.5.8.high_sierra.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/b1/b1767fb00e2e69fd9da73427d0926b1d1d0003622f7ddc0dd3a899b2894781ff?__gda__=exp=1538038849~hmac=c702c9335e7785fcbacad1f29afa61244d02f2eebb
######################################################################## 100.0%
==> Pouring fn-0.5.8.high_sierra.bottle.tar.gz
  /usr/local/Cellar/fn/0.5.8: 5 files, 16.7MB

Now everything is ready for the initial deployment of our function using the CLI. For simplicity, we will use the built-in environment to run, for example Node:

$ fn init --runtime node --trigger http hellonode

Creating function at: /hellonode
Function boilerplate generated.
func.yaml created.

A new directory will be created hellonode to further develop our Fn function with some basic config files. Inside the newly created directory, you can create your application following the standards of the language or runtime of your choice:

# Каталог с node выглядит так:

   hellonode
   ├── func.js
   ├── func.yaml
   └── package.json

# Свежеустановленное окружение Java11 такое:

   hellojava11
   ├── func.yaml
   ├── pom.xml
   └── src
       ├── main
       │   └── java
       │       └── com
       │           └── example
       │               └── fn
       │                   └── HelloFunction.java
       └── test
           └── java
               └── com
                   └── example
                       └── fn
                           └── HelloFunctionTest.java

Fn creates the initial structure of the project, creates a file func.yaml, which contains the necessary settings for Fn, and sets the template for the code in the language you have chosen.

In the case of the Node runtime, this means:

$ cat hellonode/func.js

const fdk=require('@fnproject/fdk');

fdk.handle(function(input){
  let name = 'World';
  if (input.name) {
    name = input.name;
  }
  return {'message': 'Hello ' + name}
})

Now we will quickly test our function locally to see how everything works.

First, we will start the Fn server. As already mentioned, the Fn server is a Docker container, so after launch it will go and take an image from the Docker registry.

$ fn start -d                    # запускаем локальный сервер в фоне

Unable to find image 'fnproject/fnserver:latest' locally
latest: Pulling from fnproject/fnserver
ff3a5c916c92: Pull complete
1a649ea86bca: Pull complete
ce35f4d5f86a: Pull complete

...

Status: Downloaded newer image for fnproject/fnserver:latest
668ce9ac0ed8d7cd59da49228bda62464e01bff2c0c60079542d24ac6070f8e5

To run our function, we need to “roll it out”. This requires имя приложения: in Fn, all applications must be specified as namespaces for related functions.

Fn CLI will search for the file func.yaml in the current directory that will be used to configure the function. So first you need to go to our directory hellonode.

$ cd hellonode
$ fn deploy --app fnexo --local  # выкатываем функцию локально, имя приложения - fnexo.
                                 # параметр local не заливает образ в удаленный реестр,
                                 # запуская его напрямую

Deploying hellonode to app: fnexo
Bumped to version 0.0.2
Building image nfrankel/hellonode:0.0.3 .
Updating function hellonode using image nfrankel/hellonode:0.0.3...
Successfully created app:  fnexo
Successfully created function: hellonode with nfrankel/hellonode:0.0.3
Successfully created trigger: hellonode-trigger

As you can see from the output of the command, a new Docker container image is created containing our function. The function is ready to be called, and we have two ways to do this:

  • using the Fn command invoke
  • calling directly via http

Вызов invoke via Fn, it simply emulates work over HTTP for tests, which is convenient for a quick check:

$ fn invoke fnexo hellonode      # вызываем функцию hellonode приложения fnexo

{"message":"Hello World"}

In order to call the function directly, you need to know the full URL:

$ curl http://localhost:8080/t/fnexo/hellonode-trigger

{"message":"Hello World"}

The Fn server exposes its functions on port 8080 and the function URL appears to follow the pattern t/app/function, but not completely. Through HTTP, the function is not called directly, but through the so-called trigger, which, according to its name, "starts" the function call. Triggers are defined in `func.yml project:

schema_version: 20180708
name: hellonode
version: 0.0.3
runtime: node
entrypoint: node func.js
format: json
triggers:
- name: hellonode-trigger
  type: http
  source: /hellonode-trigger    # URL триггера

We can change the name of the trigger to match the name of the function, this will simplify everything:

triggers:
- name: hellonode-trigger
  type: http
  source: /hellonode    # совпадает с именем функции

Then we start supplying the function again and call it from a new trigger:

$ fn deploy --app fnexo hellonode --local
$ curl http://localhost:8080/t/fnexo/hellonode

{"message":"Hello World"}

Everything is working! It's time to move on to field experiments and publish our FaaS on the server!

Installing serverless function services on your own infrastructure

Let's quickly set up a virtual machine using the Exoscale CLI. If you haven't set it up yet, you can use our quick start guide. This is a cool tool that will further increase your productivity. Don't forget to set up a rule to open port 8080 in the Security Group! The following commands will launch a clean virtual machine, ready to host our functions:

$ exo firewall create fn-securitygroup
$ exo firewall add fn-securitygroup ssh --my-ip
$ exo firewall add fn-securitygroup -p tcp -P 8080-8080 -c 0.0.0.0/0
$ exo vm create fn-server -s fn-securitygroup

Then you can ssh into the virtual machine and install the remote Fn server:

$ exo ssh fn-server

The authenticity of host '185.19.30.175 (185.19.30.175)' can't be established.
ECDSA key fingerprint is SHA256:uaCKRYeX4cvim+Gr8StdPvIQ7eQgPuOKdnj5WI3gI9Q.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '185.19.30.175' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 18.04 LTS (GNU/Linux 4.15.0-20-generic x86_64)

Then install Docker and the Fn server in the same way as it was already done on the local machine, start the server:

$ sudo apt-get update
$ sudo apt-get install docker.io
$ sudo systemctl start docker
$ curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh
$ sudo fn start

...

    ______
   / ____/___
  / /_  / __ 
 / __/ / / / /
/_/   /_/ /_/
    v0.3.643

Fn is ready to receive functions! To target the transfer of functions to a remote server, we will use the command deploy from the local computer, omitting the flag --local.

In addition, Fn requires you to specify the location of the Fn server and the Docker registry. These options can be set via environment variables FN_API_URL и FN_REGISTRY accordingly, but offers a more convenient way to easily manage the creation and management of configurations for deployment.

In terms of Fn, the configuration to deploy is called context. The following command will create a context:

$ fn create context exoscale --provider default --api-url http://185.19.30.175:8080 --registry nfrankel

You can view the available contexts like this:

$ fn list contexts

CURRENT NAME      PROVIDER      API URL                      REGISTRY
    default       default       http://localhost:8080/
    exoscale      default       http://185.19.30.175:8080    nfrankel

And switch to the context that has just been created, like this:

 $ fn use context exoscale

 Now using context: exoscale

From this point on, the Fn feature supply will download Docker images using the selected account on DockerHub (in my case - nfrankel), and then notify the remote server (in this example, http://185.19.30.175:8080) about the location and version of the latest image containing your function.

$ fn deploy --app fnexo .   # выполняется на локальной машине из каталога hellonode

Deploying function at: /.
Deploying hellonode to app: fnexo
Bumped to version 0.0.5
Building image nfrankel/hellonode:0.0.5 .

Finally:

$ curl http://185.19.30.175:8080/t/fnexo/hellonode

{"message":"Hello World"}

Building our own serverless based on Fn
Function lifecycle in Fn-based serverless computing

Benefits of serverless computing at its own facilities

Serverless computing is a convenient solution for quickly implementing independent parts of an application that interact with more complex applications or microservices.

This is often due to the hidden cost of being tied to the chosen vendor, which, depending on the specific use case and volume, can result in higher costs and less flexibility down the road.

Multi-cloud and hybrid cloud architectures also suffer in this case, as you can easily find yourself in a situation where you would like to use serverless computing, but due to corporate policy, this may not be possible.

Fn is quite easy to use, can give almost the same FaaS interface, with little overhead. It will get rid of any bindings to the provider, you can install it locally or in any convenient cloud provider of your choice. There is also freedom in choosing a programming language.

This article only covers the basics of Fn, but creating your own runtime environment is quite simple, and the overall architecture can be deployed more widely using an Fn load balancer, or placing Fn behind a proxy for protection.

Source: habr.com

Add a comment