Kubernetes tips & tricks: custom error pages in NGINX Ingress

In this article, I want to talk about two features of NGINX Ingress related to displaying personalized error pages, as well as their limitations and ways to get around them.

1. Change the default backend

By default, NGINX Ingress uses the default backend, which performs the corresponding function. This means that when requesting an Ingress with a host that is not in the Ingress resources, we get this page with a 404 response code:

However, more and more often, our clients come with a request to show their page with a company logo and other amenities instead of the standard 404. For this, NGINX Ingress has built-in capability redefine default-backend-service. We pass the format record as an argument to the option of the same name namespace/servicename. The service port must be 80.

To do this, you need to create your own pod (deployment) and a service with your application (example implementation in YAML from the ingress-nginx repository), which will be given instead of the default backend.

Here is a small illustration:

~$ curl -i -XGET http://sadsdasdas.kube-cloud.my/
HTTP/1.1 404 Not Found
Date: Mon, 11 Mar 2019 05:38:15 GMT
Content-Type: */*
Transfer-Encoding: chunked
Connection: keep-alive

<span>The page you're looking for could not be found.</span>

So all domains that are not explicitly created via YAML with kind: Ingress, fall into the default-backend. In the listing above, this domain became sadsdasdas.

2. Handling HTTP errors in the application by the default backend

Another situation is requests ending with HTTP errors (404, 500, 502…) to an application that does not handle such situations (corresponding beautiful pages are not generated). It may also be caused by the desire of developers to return the same error pages in multiple applications.

To implement this case on the server side, we need:

1. Follow the instructions above from the paragraph about the default backend;
2. Add the key to the configuration ConfigMap nginx-ingress custom-http-errors, for example, with the value 404,503 (obviously matches the error codes covered by the new rule).

The expected result is achieved: when the client application is running and receiving an error with a 404 or 503 response code, the request will be automatically redirected to the new default backend ...

However, when developing an application for the default backend and custom-http-errors, an important feature must be taken into account:

!!! Important The custom backend is expected to return the correct HTTP status code instead of 200. NGINX does not change the response from the custom default backend.

The fact is that when the request is redirected, the headers will contain useful information with the previous response code and additional information (their full list is available here).

This means that you must take care of the correct response code. Here's an example from the documentation how it works.

Different applications - different default backend

So that the solution is not global for the entire cluster, but applies only to specific applications, first you need to check the version of Ingress. If it matches 0.23 or higher, use the native Ingress annotations:

1. We can redefine default-backend for each Ingress with annotation;
2. We can redefine custom-http-errors for each Ingress with annotation.

As a result, the Ingress resource will look something like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: {{ .Chart.Name }}-app2
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/custom-http-errors: "404,502"
    nginx.ingress.kubernetes.io/default-backend: error-pages
spec:
  tls:
  - hosts:
    - app2.example.com
    secretName: wildcard-tls
  rules:
  - host: app2.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: {{ .Chart.Name }}-app2
          servicePort: 80

In this case, 404 and 502 errors will be redirected to the error-pages service with all the necessary headers.

В previous versions of Ingress did not have this feature. (fateful commit in 0.23). And if you have 2 completely different applications running in your cluster and you want to specify different default-backend-services and handling different error codes for each of them, you will have to use workarounds for this, of which we have two.

Ingress < 0.23: approach one

This option is simpler. As an application that renders its pages, we have plain HTML, which does not know how to look at headers and return correct response codes. Such an application rolls out with Ingress with url /error-pages, and in the directory ws will be given HTML.

Illustration in YAML:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: {{ .Chart.Name }}-app2
  annotations:
    kubernetes.io/ingress.class: "nginx"
    ingress.kubernetes.io/server-snippet: |
      proxy_intercept_errors on;
      error_page 500 501 502 503 504 @error_pages;
      location @error_pages {
        rewrite ^ /error-pages/other/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }
spec:
  tls:
  - hosts:
    - app2.example.com
    secretName: wildcard-tls
  rules:
  - host: app2.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: {{ .Chart.Name }}-app2
          servicePort: 80

The service for this deployment must be of type ClusterIP.

At the same time, in the application where we will handle the error, in Ingress we add server-snippet or configuration-snippet with the following content:

nginx.ingress.kubernetes.io    /server-snippet: |
      proxy_intercept_errors on;
      error_page 500 501 502 503 504 @error_pages;
      location @error_pages {
        rewrite ^ /error-pages/ws/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }

Ingress < 0.23: second approach

A variant for an application that can process headers ... And in general, this is a more correct way, borrowed from custom-http-errors. Using it manually (copying) will allow you not to change the global settings.

The steps are as follows. We create same deployment with an application that can listen to the right headers and respond correctly. We add server-snippet applications to the Ingress with the following content:

nginx.ingress.kubernetes.io    /server-snippet: |
      proxy_intercept_errors off;
      error_page 404 = @custom_404;
      error_page 503 = @custom_503;
      location @custom_404 {
        internal;
        proxy_intercept_errors off;
        proxy_set_header       X-Code             404;
        proxy_set_header       X-Format           $http_accept;
        proxy_set_header       X-Original-URI     $request_uri;
        proxy_set_header       X-Namespace        $namespace;
        proxy_set_header       X-Ingress-Name     $ingress_name;
        proxy_set_header       X-Service-Name     $service_name;
        proxy_set_header       X-Service-Port     $service_port;
        proxy_set_header       Host               $best_http_host;
        rewrite ^ /error-pages/ws/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }
      location @custom_503 {
        internal;
        proxy_intercept_errors off;
        proxy_set_header       X-Code             503;
        proxy_set_header       X-Format           $http_accept;
        proxy_set_header       X-Original-URI     $request_uri;
        proxy_set_header       X-Namespace        $namespace;
        proxy_set_header       X-Ingress-Name     $ingress_name;
        proxy_set_header       X-Service-Name     $service_name;
        proxy_set_header       X-Service-Port     $service_port;
        proxy_set_header       Host               $best_http_host;
        rewrite ^ /error-pages/ws/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }

As you can see, for each error that we want to process, we need to make our own location, where all the necessary headers will be substituted, as in "native" custom error pages. This way we can create different personalized error pages even for individual locations and servers.

PS

Others from the K8s tips & tricks series:

• «Transfer of resources running in the cluster under the control of Helm 2";
• «About the allocation of nodes and the loads on the web application";
• «Access to dev sites";
• «Speeding up bootstrap for large databases».

Read also on our blog:

• «Back to microservices with Istio. Part 1";
• «[Illustrated] Kubernetes Networking Guide. Part 3».

Source: habr.com