🥇Kubernetes tips & tricks: personalized error pages in NGINX Ingress

In this article, I want to talk about two features of NGINX Ingress related to displaying customized error pages, as well as the limitations within them and ways to bypass them.

1. Changing the default backend

By default, NGINX Ingress uses a default backend that serves the corresponding function. This means that when an Ingress request is made with a host that is not listed in the Ingress resources, we receive a page with a 404 response code:

However, more and more of our clients come with a request to show their own page with a branded logo and other conveniences instead of the standard 404. To achieve this, NGINX Ingress has a built-in option to override default-backend-service. The option of the same name takes as an argument a record in the format namespace/servicename. The service port must be 80.

To do this, you need to create your own pod (deployment) and service with your application (YAML implementation example from the ingress-nginx repository), which will be served instead of the default backend.

Here’s a small illustration:

~$ curl -i -XGET http://sadsdasdas.kube-cloud.my/
HTTP/1.1 404 Not Found
Date: Mon, 11 Mar 2019 05:38:15 GMT
Content-Type: */*
Transfer-Encoding: chunked
Connection: keep-alive

<span>The page you're looking for could not be found.</span>

Thus, all domains that are explicitly not created through YAML with kind: Ingress, fall into the default backend. In the listing above, such a domain became sadsdasdas.

2. Handling HTTP errors in the application using the default backend

Another situation involves HTTP-errors (404, 500, 502…) in requests to the application, where such situations are not handled (the appropriate pretty pages are not generated). This can also be caused by developers wanting to serve the same error pages across multiple applications.

To implement this case on the server side, we need to:

Follow the instructions above from the section on the default backend;
Add the key custom-http-errors, for example, with the value 404,503 (obviously, this corresponds to the error codes to which the new rule applies).

The expected result is achieved: when the client application is running and receives an error with a 404 or 503 response code, the request will be automatically redirected to the new default backend…

However, when developing an application for the default backend and custom-http-errors, it is important to consider a crucial feature:

!!! Important The custom backend is expected to return the correct HTTP status code instead of 200. NGINX does not change the response from the custom default backend.

The thing is, when redirecting a request, there will be useful information in the headers with the previous response code and additional information (a complete list is available) here).

This means that you have to take care of the correct response code. Here’s an example from the documentation on how this works.

Different applications require a different default backend.

To ensure the solution is not global for the entire cluster, but only applies to specific applications, you need to first check the Ingress version. If it is 0.23 or higher, use the ‘native’ Ingress annotations:

We can override the default-backend for unstructured output of the Ingress with an annotation.;
We can override the custom-http-errors for unstructured output of the Ingress with an annotation..

As a result, the Ingress resource will look something like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: {{ .Chart.Name }}-app2
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/custom-http-errors: "404,502"
    nginx.ingress.kubernetes.io/default-backend: error-pages
spec:
  tls:
  - hosts:
    - app2.example.com
    secretName: wildcard-tls
  rules:
  - host: app2.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: {{ .Chart.Name }}-app2
          servicePort: 80

In this case, errors 404 and 502 will be redirected to the error-pages service with all the necessary headers.

In In previous versions of Ingress, this capability was not available. (the fateful commit in 0.23). And if you have 2 completely different applications running in your cluster and you want to specify different default-backend-services and handle different error codes for each of them — you will need to use workarounds, of which we have two.

Ingress < 0.23: approach one

This option is simpler. As an application serving its pages, we have a regular HTML that does not check headers or return correct response codes. Such an application is deployed with Ingress at the url /error-pages, and in the directory ws will lie the served HTML.

Illustration in YAML:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: {{ .Chart.Name }}-app2
  annotations:
    kubernetes.io/ingress.class: "nginx"
    ingress.kubernetes.io/server-snippet: |
      proxy_intercept_errors on;
      error_page 500 501 502 503 504 @error_pages;
      location @error_pages {
        rewrite ^ /error-pages/other/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }
spec:
  tls:
  - hosts:
    - app2.example.com
    secretName: wildcard-tls
  rules:
  - host: app2.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: {{ .Chart.Name }}-app2
          servicePort: 80

The service for this deployment must be of type ClusterIP.

Meanwhile, in the application where we will handle the error, we add a server-snippet or configuration-snippet in the Ingress with the following content:

nginx.ingress.kubernetes.io    /server-snippet: |
      proxy_intercept_errors on;
      error_page 500 501 502 503 504 @error_pages;
      location @error_pages {
        rewrite ^ /error-pages/ws/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }

Ingress < 0.23: second approach

Option for an application that can handle headers… In general, this is a more correct method, borrowed from custom-http-errors. Manual usage (copying) allows you to avoid changing global settings.

The steps are as follows. We create the same deployment with an application that can listen to the necessary headers and respond correctly. We add the server-snippet to the application's Ingress with the following content:

nginx.ingress.kubernetes.io    /server-snippet: |
      proxy_intercept_errors off;
      error_page 404 = @custom_404;
      error_page 503 = @custom_503;
      location @custom_404 {
        internal;
        proxy_intercept_errors off;
        proxy_set_header       X-Code             404;
        proxy_set_header       X-Format           $http_accept;
        proxy_set_header       X-Original-URI     $request_uri;
        proxy_set_header       X-Namespace        $namespace;
        proxy_set_header       X-Ingress-Name     $ingress_name;
        proxy_set_header       X-Service-Name     $service_name;
        proxy_set_header       X-Service-Port     $service_port;
        proxy_set_header       Host               $best_http_host;
        rewrite ^ /error-pages/ws/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }
      location @custom_503 {
        internal;
        proxy_intercept_errors off;
        proxy_set_header       X-Code             503;
        proxy_set_header       X-Format           $http_accept;
        proxy_set_header       X-Original-URI     $request_uri;
        proxy_set_header       X-Namespace        $namespace;
        proxy_set_header       X-Ingress-Name     $ingress_name;
        proxy_set_header       X-Service-Name     $service_name;
        proxy_set_header       X-Service-Port     $service_port;
        proxy_set_header       Host               $best_http_host;
        rewrite ^ /error-pages/ws/index.html break;
        proxy_pass http://error-pages.prod.svc.cluster.local;
      }

As can be seen, for each error we want to handle, we need to create a specific location where all necessary headers will be substituted, just like in the "native" custom-error-pages. This way, we can create different personalized error pages even for specific locations and servers.

P.S.

Another from the K8s tips & tricks series: