With today’s 3D cameras, autonomous vehicles can reliably detect obstacles in their path. Modern systems deliver information so accurate that it can even be determined whether it is an object or a person causing
an obstruction.
Precise detection of the surrounding area is a crucial basis for the successful application of autonomous vehicles. Alongside sensor systems such as lidar, radar and ultrasound, 3D cameras can also be used to enable an autonomous vehicle to precisely recognise its own position and that of the objects around it at any time in order to facilitate the accurate coordination of manoeuvres. A variety of technologies are employed.
Stereo cameras simulate a pair of eyes
In the case of stereo cameras, two digital cameras work together. Similar to the stereoscopic vision of a pair of eyes, their images enable the depth perception of a surrounding area, providing information on aspects including the position, distance and speed of objects. The cameras capture the same scene from two different viewpoints. Using triangulation and based on the arrangement of pixels, software compares both images and determines the depth information required for a 3D image. The result becomes even more precise when structured light is added to the stereo solution. Geometric brightness patterns are projected onto the scene by a light source. This pattern is distorted by three-dimensional forms, enabling depth information to also be determined on this basis.
ToF cameras measure the speed of light
Another method is time-of-flight (ToF), which determines distance based on the transit time of individual light points. Achieving centimetre accuracy calls for rapid and precise electronics. Time-of-flight technology is highly effective in obtaining depth data and measuring distances. A time-of-flight camera provides two types of information on each pixel: the intensity value – given as grey value – and the distance of the object from the camera, known as the depth of field. Modern ToF cameras are equipped with an image chip with several thousand receiving elements. This means that a scene can be captured in its entirety and with a high degree of detail in a single shot.
More precise information by combining cameras
While the basic technologies are already in use to a large extent – in car assistance systems, industrial robots, on the land as well as in drones – research is looking to further optimise systems. 3D cameras that need to function in vary-ing lighting conditions are hindered by large pixels and therefore lower resolution. To offset this, work is under way to develop a piece of software which can fuse together 3D camera images with those of a high-resolution 2D camera, for example. This will enable high-resolution 3D data to be obtained, which can then be further processed with the help of artificial intelligence: thanks to high-resolution images, the detected objects can be classified – and it is a safe bet that a person will not be mistaken for a rubbish bin. Other projects are also using colour cameras to enable classification to be made according to colour as well as shape.
Eagle-eyed vision
A further aim is to reduce the number of cameras required. Until now, a whole host of cameras and sensors all around the vehicle, or a rotating camera on the roof, was needed to generate as wide a viewing range as possible. At the University of Stuttgart, the widening of a single camera’s field of view was modelled on the eye of an eagle. An eagle’s eye has an extraordinary number of photoreceptors in its central fovea – the part of the eye where vision is at its sharpest. Additionally, eagles have a second fovea at the corner of their eye, allowing for sharp peripheral vision. Scientists have developed a sensor which all but emulates an eagle’s eye across a small area. Research was carried out under the umbrella of the SCoPE research centre at the University of Stuttgart and was able to be put into practice thanks to the very latest in 3D printing technology from Karlsruhe-based company Nanoscribe. The researchers in Stuttgart imprinted a wide range of micro-objective lenses with different focal lengths and fields of vision directly onto a high-resolution CMOS chip. The smallest has a focal length equivalent to a wide-angle lens, two lenses have a medium field of view, and the largest lens has a very long focal length and a small field of view just like a typical telephoto lens. All four images created by the lenses on the chip are electronically and simultaneously read and processed. In the process, a small computer program constructs the image to display the telephoto lens’ high-resolution image in the centre, and that of the wide-angle lens on the very outer edge. Owing to the fact that the sensor system as a whole has dimensions of only a few square millimetres – the lenses have a diameter in the region of just one to several hundred micrometres – a new generation of minidrones could also be set to profit from the technology alongside the automotive industry.