While waiting for the PS5 and Project Scarlett to support ray tracing, I've been thinking about lighting in games. I found material where the author explains what light is, how it affects design, changes gameplay, aesthetics and experience. All with examples and screenshots. During the game, you do not immediately pay attention to this.
Introduction
Lighting is needed not only so that the player can see the scene (although this is very important). Light affects emotions. Many lighting techniques in theater, cinema and architecture are used to enhance the emotional component. Why not borrow these principles for game designers? The connection between picture and emotional response provides another powerful tool that helps you work with character, narrative, sound, game mechanics, and so on. At the same time, the interaction of light with the surface allows you to influence the brightness, color, contrast, shadows and other effects. All this translates into a base that every designer should master.
The purpose of this post is to determine how lighting design affects game aesthetics and user experience. Let's look at the nature of light and how it is used in other areas of art to analyze its role in video games.
"Swan Lake", Alexander Ekman
I - The nature of light
“Space, light and order. These are the things that people need as much as they need a piece of bread or a place to sleep.” – Le Corbusier
Natural light guides and accompanies us from the moment we are born. It is essential, it sets our natural rhythm. Light controls the processes of our body and affects the biological clock. Let's figure out what is the luminous flux, light intensity, color and focal points. And then we will understand what light consists of and how it behaves.
1 - What the human eye sees
Light is the part of the electromagnetic spectrum that is perceived by the eye. In this section, the wavelengths are in the range from 380 to 780 nm. During the day, we see colors through the cones, but at night, the eye uses the rods and we only see grayscale.
The main properties of visible light are direction, intensity, frequency, and polarization. Its speed in vacuum is 300 m/s, and this is one of the fundamental physical constants.
Visible electromagnetic spectrum
2 - Direction of propagation
There is no matter in a vacuum, and light travels straight ahead. However, it behaves differently when it encounters water, air, and other substances. Upon contact with a substance, part of the light is absorbed and converted into thermal energy. When colliding with a transparent material, some of the light is also absorbed, but the rest passes through. Smooth objects, such as mirrors, reflect light. If the surface of the object is uneven, the light will scatter.
Direction of light propagation
3 - Basic characteristics
Light flow. The amount of light emitted by a light source.
Unit of measurement: lm (lumen).
The power of light. The amount of light carried in a particular direction.
Unit of measurement: cd (candela).
Illumination. The amount of light flux falling on the surface.
Illumination = luminous flux (lm) / area (m2).
Unit of measure: lx (lux).
Brightness. This is the only basic characteristic of light that the human eye perceives. On the one hand, it takes into account the brightness of the light source, on the other hand, it takes into account the surface, which means that it strongly depends on the degree of reflection (on color and surface).
Unit of measure: cd/m2.
4 - Color temperature
Color temperature is measured in Kelvin and represents the color of a particular light source. British physicist William Kelvin heated a piece of coal. It became hot, shimmering with different colors that corresponded to different temperatures. At first, the coal glowed dark red, but as it heated, the color changed to bright yellow. At maximum temperature, the emitted light became blue-white.
Natural light, 24 hours, Simon Lakey
II - Light Design Techniques
In this section, we'll look at what lighting patterns can be used to influence the expressiveness of content/visuals. To do this, we will identify the similarities and differences in lighting techniques used by artists and lighting designers.
1 - Chiaroscuro and tenebrism
Chiaroscuro is one of the concepts of art theory, which refers to the distribution of illumination. With its help, tone transitions are displayed to convey volume and mood. Georges de Latour is known for his work with night chiaroscuro and scenes lit by candle flames. None of his predecessor artists worked out such transitions so masterfully. Light and shadow play a crucial role in his work and are part of the composition in a variety of and often alternative variations. The study of de Latour's paintings helps to understand the use of light and its properties.
Georges de La Tour "Penitent Mary Magdalene", 1638-1643
a - High contrast
In this picture, the light face and clothes stand out against the dark background. Due to the high contrast tones, the viewer's attention is focused on this part of the image. In reality, there would be no such contrast. The distance between the face and the candle is greater than between the candle and the hands. However, when compared to the face, we can see that the tone and contrast on the hands are muted. Georges de Latour uses different contrast to draw the viewer's attention.
b - Contour and rhythm of light
Due to the high tonal difference, contours appear at the edges of the figure in some areas. Even in the dark parts of the painting, the artist liked to use different tones to emphasize the boundaries of the subject. The light is not concentrated in one area, it slides down: from the face to the feet.
c — Light source
In most of his work, Georges de Latour uses candles or lamps as the source of light. The picture shows a burning candle, but we already know that chiaroscuro here does not depend on it. Georges de Latour positioned his face against a dark background and placed a candle to create a sharp transition between tones. For high contrast, light tones are adjacent to dark tones, achieving an optimal effect.
d - Chiaroscuro as a composition of geometric shapes
If we simplify the light and shadow in this work, we will see the basic geometric shapes. The unity of light and dark tones forms a simple composition. It indirectly sets the feeling of space, in which the position of objects and figures shows the foreground and background, creating tension and energy.
2 - Basic Cinematic Lighting Techniques
2.1 - Lighting from three points
One of the most popular and successful ways to illuminate any object is three-point lighting, a classic Hollywood scheme. This technique allows you to convey the volume of the subject.
Drawing light (Key Lighting, that is, the main light source)
As a rule, this is the most powerful light in each scene. It can come from anywhere, its source can be from the side or behind the object (Jeremy Byrne "Digital Lighting and Rendering").
Fill light (Fill Lighting, that is, light to control contrasts)
From the name it is obvious that it is used to "fill in" and remove the dark areas that the key light creates. The fill light is noticeably less intense and is at an angle to the main light source.
Background light (Backlighting, that is, a background separator)
It is used to convey the volume of the scene. It separates the subject from the background. Like the fill light, the background light is less intense and covers a larger area of the subject.
2.2 - Bottom
Due to the movement of the Sun, we are used to seeing people illuminated from any angle, but not from below. This way looks very unusual.
"Frankenstein", James Weil, 1931
2.3 - Behind
The object is placed between the light source and the viewer. Because of this, a glow appears around the object, and the rest of its parts remain in shadow.
"Alien", Steven Spielberg, 1982
2.4 - Side
Such lighting is used to illuminate the scene from the side. It creates a sharp contrast that displays textures and emphasizes the contours of the object. This method is close to the chiaroscuro technique.
Blade Runner, Ridley Scott, 1982
2.5 - Practical lighting
This is real lighting in the scene, that is, lamps, candles, a TV screen and others. Such additional light can be used to increase the intensity of the illumination.
"Barry Lyndon", Stanley Kubrick, 1975
2.6 - Reflected light
Light from a powerful source is scattered by a reflector or some surface, such as a wall or ceiling. Thus, the light covers a larger area and falls more evenly.
The Dark Knight Rises Christopher Nolan 2012
2.7 - Hard and soft light
The main difference between hard and soft light is the size of the light source in relation to the object. The sun is the largest source of light in the solar system. However, it is 90 million kilometers away from us, which means it is a small source of light. It creates hard shadows and, accordingly, hard light. If clouds appear, the whole sky becomes a huge source of light, and the shadows are harder to see. And so, there is a soft light.
3D examples with LEGO, João Prada, 2017
2.8 - High and low key
High key lighting is used to create very bright scenes. It is often close to overexposed. All light sources are approximately equal in power.
Unlike high key lighting, low key lighting makes the scene very dark, and there can be a powerful light source in it. The main role is given to shadows, not light, to convey a sense of suspense or drama.
"THX 1138", George Lucas, 1971
2.9 - Motivated Lighting
Such lighting imitates natural light - sunlight, moonlight, street lights, and so on. It is used to enhance practical lighting. Special techniques help to make motivated lighting natural, for example, filters (gobos) to create the effect of curtained windows.
Drive, Nicolas Winding Refn, 2011
2.10 - Outdoor light
It can be sunlight, moonlight, or street lights that are visible in the scene.
“Very strange things. Season 3, Duffer Brothers, 2019
III - Rendering Basics
Level designers are aware of the importance of lighting and use it to achieve a certain perception of the scene. To illuminate the level and achieve their desired visual goals, they need to define static light sources, their propagation angles and colors. They set a certain atmosphere and the necessary overview. But everything is not so simple, because lighting depends on such technical characteristics - for example, on the power of the processor. Therefore, there are two types of lighting: pre-computed lighting and real-time rendering.
1 - Precomputed lighting
Designers use static lighting to set the characteristics of each light source—including its position, angle, and color. As a rule, it is impossible to implement real-time global illumination due to performance.
Precomputed static global illumination can be used in most engines, including Unreal Engine and Unity. The engine "bakes" such lighting into a special texture, the so-called "light map" (lightmap, lightmap). These lightmaps are stored along with other map files and are referenced by the engine when rendering the scene.
The same scene: no lighting (left), direct lighting only (middle), and indirect global illumination (right). Illustrations from Unity Learn
In addition to lightmaps, there are shadow maps, which, accordingly, are used to create shadows. First, everything is drawn taking into account the light source - it generates a shadow that reflects the pixel depth of the scene. The resulting pixel depth map is called a shadow map. It contains information about the distance between the light source and the objects closest to it for each pixel. Then rendering is performed, where each pixel of the surface is checked against a shadow map. If the distance between the pixel and the light source is greater than that recorded in the shadow map, then the pixel is in shadow.
The algorithm for applying the shadow map. Illustration from OpenGl-tutorial
2 - Rendering in real time
One of the classic real-time lighting models is called the Lambert model (after the Swiss mathematician Johann Heinrich Lambert). When rendering in real time, the GPU usually sends objects one at a time. This method uses the object's display (its position, rotation, and scale) to determine which of its surfaces to draw.
In the case of Lambert illumination, light comes from every point on the surface in all directions. This does not take into account some subtleties, such as reflections (article by Chandler Prall). To make the scene look more realistic, additional effects are superimposed on the Lambert model - highlights, for example.
Shading Lambert on the example of a sphere. Illustration from the materials of Peter Dyachikhin
Most modern engines (Unity, Unreal Engine, Frostbite and others) use physically correct rendering (Physically Based Rendering, PBR) and shading (article by Lukas Orswarn). PBR shading offers more intuitive and convenient ways and parameters to describe a surface. In Unreal Engine, PBR materials have the following options:
- Base Color (base color) - the actual texture of the surface.
- Roughness - How rough the surface is.
- Metallic - Whether the surface is metal.
- Specular (specularity) - the amount of highlights on the surface.
Without PBR (left), PBR (right). Illustrations from Meta 3D studio
However, there is another approach to rendering - ray tracing. Previously, this technology was not considered due to problems with performance and optimization. It was used only in the film and television industry. But the release of a new generation of video cards allowed for the first time to use this approach in video games.
Ray tracing is a rendering technology that creates more realistic lighting effects. It repeats the principles of light propagation in a real environment. Rays emitted by a light source behave in the same way as photons. They bounce off surfaces in an arbitrary direction. At the same time, when entering the camera, the reflected or direct rays transmit visual information about the surface from which they are reflected (for example, they report its color). A lot of projects from E3 2019 will support this technology.
3 - Types of light sources
3.1 - Point light
Emits light in all directions, just like a normal light bulb in real life.
Unreal Engine Documentation
3.2 - Spot light
Emits light from a single point, while the light spreads like a cone. Real life example: flashlight.
Unreal Engine Documentation
3.3 - Area light
Emits direct light rays from a defined contour (such as a rectangle or circle). Such light heavily loads the processor, because the computer calculates all the points that emit light.
Unity Documentation
3.4 - Directional light source
Simulates the sun or other distant light source. All rays move in the same direction and can be considered parallel.
Unity Documentation
3.5 - Emissive light
Emissive light source or emitting materials (Emissive Materials in UE4) easily and effectively create the illusion that the material emits light. There is a blur effect of light - it can be seen if you look at a very bright object.
Unreal Engine Documentation
3.6 - Ambient Light
The scene from Doom 3 is illuminated by lamps on the walls, the engine creates shadows. If the surface is in shadow, it colors it black. In real life, light particles (photons) can bounce off surfaces. In more advanced rendering systems, light is baked into textures or rendered in real time (global illumination). Older game engines like ID Tech 3 (Doom) used too many resources to calculate indirect lighting. To solve the problem with the lack of indirect lighting, diffused light was used. And all surfaces were at least slightly lit.
Doom 3 engine (IdTech 4 engine)
3.7 - Global illumination
Global illumination is an attempt to calculate the reflection of light from one object to another. This process is much more CPU intensive than ambient light.
Unreal Engine Documentation
IV - Light Design in Video Games
The visual composition (position of the light, its angles, colors, field of view, movement) has a big impact on how users perceive the game environment.
Designer Will Wright at GDC talked about the functions of visual composition in the game environment. In particular, it directs the player's attention to important elements - this happens by adjusting the saturation, brightness and color of objects in the level.
All this affects the gameplay.
The right atmosphere emotionally engages the player. Designers must take care of this while creating visual integrity.
Maggie Seif El-Nasr conducted several experiments - she invited users who are not familiar with FPS shooters to play Unreal Tournament. Due to poor lighting design, players noticed enemies too late and died quickly. They got frustrated and in most cases abandoned the game.
Light creates effects, but in video games it can be used in a different way than in theater, cinema and architecture. In terms of design, there are seven categories that describe lighting patterns. And here we must not forget about emotions.
Design Elements in Level Art, Jeremy Price
1 - Guide
Uncharted 4
In 100 Things Every Designer Needs to Know About People, Susan Weinshank explores the importance of central and peripheral vision.
Since central vision is the first thing we see, it should include critical elements that the player must see according to the designer's idea. Peripheral vision provides context and reinforces central vision.
The games of the Uncharted series are a good example of this - the light enters the central field of view and guides the player. But if the elements that enter the peripheral vision contradict the central one, the connection between the designer and the player breaks down.
Until Dawn
Lighting is used to guide the player. Studio creative director Will Biles said: “The hardest part for us was to create an atmosphere of fear without taking everything into darkness. Unfortunately, when a picture gets too dark, the game engine tries to make it brighter, and vice versa. We had to invent new techniques to deal with this problem.”
As seen in the illustration below, the warm light stands out against the blue background, drawing the player's attention.
2 - Lighting/Cropping
Resident Evil 2 Remake
Light in RE2 Remake can change the frame. As you walk through the dark corridors of the Racoon City Police Station, the player's flashlight serves as the primary source of light. This kind of lighting is a powerful mechanic. The changed perspective draws the player's eye to the illuminated area and cuts off everything else due to the strong contrast.
Dark Souls I
The Tomb of the Giants is one of the very dark locations in the game with a bunch of dangerous cliffs. It can be passed if you follow the glowing stones and move carefully so as not to fall. You should also beware of white bright eyes, because this is the enemy.
The radius of illumination from the player is greatly reduced, visibility in the dark is limited. By holding the lantern in the left hand, the player increases both the illumination and their field of view. At the same time, the flashlight greatly reduces the damage done, and you have to choose: vision or protection.
3 - Narration
Prey
Since the station where the action takes place is in orbit, the game has a special light cycle. It determines the direction of the light and, accordingly, greatly affects the gameplay. This game makes it harder to find items and locations than usual. In the far sections, the player can solve problems that arise by looking at them from one angle from inside the station and from a different angle from the outside.
Alien Isolation
In Alien, light is used to guide the player and create a sense of fear. The user is in constant tension - somewhere out there in the dark a xenomorph is hiding.
4 - Disguise
Splinter Cell: Blacklist
The light in it not only guides the user, but is also used as a game mechanic.
In many locations, players use shadows to keep from falling off a safe course and running into enemies. In Splinter Cell, the role of a "visibility meter" is performed by a light on the character's equipment - the more hidden the player is, the brighter the light bulb glows.
Mark of the Ninja
In Mark of the Ninja, light and dark are completely opposed to each other. The game's lead designer, Nels Andersen, said, "The way a character looks shows whether you're visible or not. If you are hidden, you are dressed in black, only some details are highlighted in red, in the light - you are completely painted ”(article Mark of the Ninja's five stealth design rules).
5 - Fight/Defend
Alan Wake
The flashlight in Alan Wake is a weapon. Without it, it is impossible to eliminate enemies. You need to direct light at them and hold it for a certain time - this way they become vulnerable, and they can be killed. When the light falls on the enemy, a halo appears, then it decreases and the object begins to glow. At this point, the player can shoot the enemy.
Also, to eliminate opponents, you can use flares and stun grenades.
A Plague Tale: Innocence
In a project from Asobo Studio, you can use rats against people. For example, if you break the enemy's lantern, then he will instantly plunge into darkness, which does not hold back hordes of rats.
6 - Alert/Feedback
Deus Ex: Mankind Divided
In Deus Ex, security cameras monitor what is happening in their field of view, which is limited by a cone of light. The light is green when they are neutral. Upon detecting an enemy, the camera changes its light to yellow, beeps, and tracks the target either for a few seconds or until the enemy escapes from its field of view. After a few seconds, the light turns red and the camera sounds an alarm. Thus, with the help of light, interaction with the player is implemented.
Hollow Knight
Metroidvania by Team Cherry changes lighting more often than the player notices.
For example, every time you take damage, the picture freezes for a moment, and a broken glass effect appears next to the hero. The general lighting is dimmed, but the light sources closest to the hero (lamps and fireflies) do not go out. This helps to emphasize the significance and power of each hit received.
7 - Separation
Assassin's Creed Odyssey
The cycle of day and night is central to the Odyssey. There are fewer patrols at night, and the player is more likely to go unnoticed.
The time of day can be changed at any time - this is provided in the game. At night, the vision of enemies is weakened, and many of them go to bed. It becomes easier to avoid and attack opponents.
The change of day and night here is a special system, and the rules of the game change dramatically depending on the time of day.
Do not starve
Survival simulator Don't Starve does not spare beginners at night - here walking in the dark is fatal. After five seconds, the player is attacked and takes damage. A source of light is essential for survival.
Mobs go to sleep as soon as night falls and wake up as the sun rises. Some creatures that sleep during the day may wake up. Plants don't grow. The meat does not dry out. The cycle of day and night sets the system, dividing the rules of the game into two categories.
V - Conclusion
Many of the lighting techniques we see in visual arts, film, and architecture are used in game development to complement the aesthetics of virtual space and enhance the player experience. However, games are very different from cinema or theater - the environment in them is dynamic and unpredictable. In addition to static lighting, dynamic light sources are used. They add interactivity and the necessary emotions.
Light is a whole spectrum of tools. It gives artists and designers ample opportunity to engage the player even more.
The development of technology has also influenced this. Now game engines have much more light settings - now it's not just highlighting locations, but also influencing game design.
References
- Seif El-Nasr, M., Miron, K. and Zupko, J. (2005). Intelligent Lighting for a Better Gaming Experience. Proceedings of the Computer-Human Interaction 2005, Portland, Oregon.
- Seif El-Nasr, M. (2005). Intelligent Lighting for Game Environments. Journal of Game Development, 1(2),
- Birn, J. (Ed.) (2000). Digital Lighting & Rendering. New Riders, Indianapolis.
- Calahan, S. (1996). Storytelling through lighting: a computer graphics perspective. Siggraph Course Notes.
- Seif El-Nasr, M. and Rao, C. (2004). Visually Directing User's Attention in Interactive 3D Environments. Siggraph Poster Session.
- Reid, F. (1992). The Stage Lighting Handbook. A&C Black, London.
- Reid, F. (1995). Lighting the stage. Focal Press, Boston.
- Petr Dyachikhin (2017), Modern Videogame Technology: Trends and Innovations, Bachelor's thesis, Savonia university of applied sciences
- Adorama learning center (2018), Basic Cinematography Lighting Techniques, from (https://www.adorama.com/alc/basic-cinematography-lighting-techniques)
- Seif El-Nasr, M., Niendenthal, S. Knez, I., Almeida, P. and Zupko, J. (2007), Dynamic Lighting for Tension in Games, the international journal of computer game research
- Yakup Mohd Rafee, Ph.D. (2015), Exploring Georges de la Tour's painting based on Chiaroscuro and tenebrism theory, University Malaysia Sarawak
- Sophie-Louise Millington (2016), In-Game Lighting: Does Lighting Influence Player Interaction and Emotion in an Environment?, University of Derby
- Prof. Stephen A. Nelson (2014), Properties of Light and Examination of Isotropic Substances, Tulane University
- Creative Commons Attribution-ShareAlike License (2019), The Dark Mod, from (https://en.wikipedia.org/wiki/The_Dark_Mod)
Source: habr.com