Precise detection of the surrounding area is a crucial basis for the successful application of autonomous vehicles. Alongside sensor systems such as lidar, radar and ultrasound, 3D cameras can also be used to enable an autonomous vehicle to precisely recognise its own position and that of the objects around it at any time in order to facilitate the accurate coordination of manoeuvres. A variety of technologies are employed.

Stereo cameras simulate a pair of eyes

In the case of stereo cameras, two digital cameras work together. Similar to the stereoscopic vision of a pair of eyes, their images enable the depth perception of a surrounding area, providing information on aspects including the position, distance and speed of objects. The cameras capture the same scene from two different viewpoints. Using triangulation and based on the arrangement of pixels, software compares both images and determines the depth information required for a 3D image. The result becomes even more precise when structured light is added to the stereo solution. Geometric brightness patterns are projected onto the scene by a light source. This pattern is distorted by three-dimensional forms, enabling depth information to also be determined on this basis.

ToF cameras measure the speed of light

Another method is time-of-flight (ToF), which determines distance based on the transit time of individual light points. Achieving centimetre accuracy calls for rapid and precise electronics. Time-of-flight technology is highly effective in obtaining depth data and measuring distances. A time-of-flight camera provides two types of information on each pixel: the intensity value – given as grey value – and the distance of the object from the camera, known as the depth of field. Modern ToF cameras are equipped with an image chip with several thousand receiving elements. This means that a scene can be captured in its entirety and with a high degree of detail in a single shot.

More precise information by combining cameras

While the basic technologies are already in use to a large extent – in car assistance systems, industrial robots, on the land as well as in drones – research is looking to further optimise systems. 3D cameras that need to function in vary-ing lighting conditions are hindered by large pixels and therefore lower resolution. To offset this, work is under way to develop a piece of software which can fuse together 3D camera images with those of a high-resolution 2D camera, for example. This will enable high-resolution 3D data to be obtained, which can then be further processed with the help of artificial intelligence: thanks to high-resolution images, the detected objects can be classified – and it is a safe bet that a person will not be mistaken for a rubbish bin. Other projects are also using colour cameras to enable classification to be made according to colour as well as shape.

Eagle-eyed vision

A further aim is to reduce the number of cameras required. Until now, a whole host of cameras and sensors all around the vehicle, or a rotating camera on the roof, was needed to generate as wide a viewing range as possible. At the University of Stuttgart, the widening of a single camera’s field of view was modelled on the eye of an eagle. An eagle’s eye has an extraordinary number of photoreceptors in its central fovea – the part of the eye where vision is at its sharpest. Additionally, eagles have a second fovea at the corner of their eye, allowing for sharp peripheral vision. Scientists have developed a sensor which all but emulates an eagle’s eye across a small area. Research was carried out under the umbrella of the SCoPE research centre at the University of Stuttgart and was able to be put into practice thanks to the very latest in 3D printing technology from Karlsruhe-based company Nanoscribe. The researchers in Stuttgart imprinted a wide range of micro-objective lenses with different focal lengths and fields of vision directly onto a high-resolution CMOS chip. The smallest has a focal length equivalent to a wide-angle lens, two lenses have a medium field of view, and the largest lens has a very long focal length and a small field of view just like a typical telephoto lens. All four images created by the lenses on the chip are electronically and simultaneously read and processed. In the process, a small computer program constructs the image to display the telephoto lens’ high-resolution image in the centre, and that of the wide-angle lens on the very outer edge. Owing to the fact that the sensor system as a whole has dimensions of only a few square millimetres – the lenses have a diameter in the region of just one to several hundred micrometres – a new generation of minidrones could also be set to profit from the technology alongside the automotive industry.

The post 3D Cameras in autonomous vehicles appeared first on Future Markets Magazine.

Radar is a fundamental part of the automated driving equation,” explains Peter Austen, Global Portfolio Director, Driver Assist Systems at ZF’s Active & Passive Safety Technology division – or ZF TRW for short. “When combined with cameras, intelligent control units and actuators, it can help to enable partially automated driving functions.” ZF TRW has been designing and developing radar in Brest since 1999.

Luxury early on

As early as the start of World War II, radar technology was being used on board aeroplanes and ships. Despite this, its first appearance in cars wasn’t until 1998, when Mercedes-Benz introduced a distance radar in the S-Class. Still, the cost of this technology was very high at the time, owing to the fact that until 2009, the required semiconductors could only be manufactured using gallium arsenide (GaAS) – an expensive and difficult material to process. A further disadvantage is the low level of integration; the ability to combine an increasing number of functions on an unchanging chip area. Only with the manufacturing of sensors in silicon germanium technology (SiGe) – the most frequently used materials for semiconductor manufacturing – did the systems become affordable. This enabled tried-and-tested standard processes to be implemented in mass production for the first time. What’s more, a whole host of functions could now be condensed onto just two SiGe components, where previously up to eight GaAs chips had been required.

Detection is becoming ever more powerful

Radar systems applied in autonomous vehicles work with millimetre precision – nowadays in the range of either 24/26 GHz or 77/79 GHz. This means that high resolutions are possible for detecting objects and determining their position and movement with centimetre accuracy. When compared to other technologies, such as camera-based methods, the radar works reliably even in conditions of poor visibility such as snow, fog, torrential rain and dazzling backlight. What’s more, the complete systems are not much bigger than a matchbox.

One fundamental difference can be drawn between two types of radar: the frequency-modulated continuous-wave (FMCW) and the impulse radar. Unlike an impulse radar, which emits only one pulse, FMC radars transmit pulses continually. With the FMCW method, the signal is modulated over the entire range during transmission, meaning that the frequency varies over time – this is called chirp. This chirp is repeated in cycles and enables FMC radars to simultaneously measure the absolute distance between the transmitter and object in addition to the differential speed between the two. The devices have admittedly had a weakness – until now: if objects approach at different speeds, it is possible that the radar might “overlook” one of them. Previous devices could therefore only reliably detect objects up to a relative speed of 50 km/h. One solution is to increase the modulation rate. What is referred to as fast chirp modulation (FCM) brings about an increase in the accuracy of distance measurement, enabling a wider range of target object speeds to be covered. Yet an increase in outside temperature causes the pulses of standard CMOS signal generators to slow, resulting in errors. To tackle this issue head-on, Fujitsu released a CMOS-based millimetre-wave signal generator in late 2016, which is capable of maintaining its modulation rate reliably and precisely even at temperatures of 150° Celsius. This allows detection errors to be reduced and even objects approaching the vehicle at a relative speed of 200 km/h are reliably detected.

Over 100 years old

Yet radar technology is in fact much older: it was as early as 1904 in Düsseldorf that the German engineer Christian Hülsmeyer developed the first practical application for the reflection of electromagnetic waves on objects – the Telemobiloscope. Similar to modern radar sensors, it transmitted focussed electromagnetic radiation – radio waves. In modern radar systems, analysing the reflected radiation allows objects to be detected along with their respective distance and speed.

Radar navigation

Despite all this, radar devices are not used in autonomous vehicles merely to detect and locate objects. At some point in the future at least, radar is intended to be used for navigation. With this in mind, Bosch and the Dutch map and traffic information provider TomTom have now become the first to create a high-resolution map with a localisation layer using radar reflection points – albeit solely for road vehicles. So far, video data has been used for this purpose. Bosch’s “radar road signature” is made up of billions of individual reflection points. These are formed everywhere that radar signals hit – for example, on crash barriers or road signs – and reproduce the course a road takes. Automated vehicles can use the map to determine their exact location in a lane down to a few centimetres. The huge advantage of the radar map is its robustness: localisation with the radar road signature also works reliably at night and in conditions of poor visibility. Moreover, only five kilobytes of data are transmitted to a Cloud per kilometre. This data volume is at least twice as high using a video map. It is expected that by 2020 at the latest, the first vehicles will provide data for the radar road signature in Europe and the U.S. “Cars arriving on the market in years to come with the assistance functions of tomorrow will be running the map for the automated vehicles of the future,” says Bosch Board Member Dr Dirk Hoheisel.

The post Radar technology in autonomous vehicles appeared first on Future Markets Magazine.

Self-driving cars, unmanned aircraft or driverless tractors – autonomous vehicles stopped being merely an idea on paper a long time ago. Now they are becoming increasingly “active” among us, at least in the form of prototypes: machines that act autonomously, independent of human commands, and make the “right” decisions, at least in everyday situations. New sensor technologies, networking possibilities and self-learning algorithms make it possible for the new vehicles to react quickly and sensitively to their environment, taking a wide range of data into consideration.

Driving without human control

According to the definition of the German Specialist Forum for Autonomous Systems (Fachforum Autonome Systeme), an autonomous vehicle exists when a system can reach a specified destination independently, irrespective of the driving or environmental situation in question. In line with this definition, the ability to learn is not a prerequisite, but rather a possible characteristic of autonomous vehicles. With regard to automated road traffic, this objective is achieved if, for example, the on-board system takes over the task of driving “completely, on all road types and in all speed zones and environmental conditions”. This means that driverless cars make decisions and take on tasks in unstructured environments, without a human driver exercising a control function.

46.8 % of people worldwide would allow their children to be driven by an autonomous car.
Source: Cisco Systems, 2013

New business models are opening up

At present, vehicles with these capabilities are not only being developed for the roads, but also for deep-sea journeys and flights in the upper atmosphere. They do not just replace the driver, helmsman or pilot, but have the potential to create completely new business models worth many billions: autonomous drones that can remain in the air for months and bring the Internet to 4.5 billion people who were previously offline are just one example of this.
Autonomous vehicles will probably lead to the most significant revolution in road travel. Automotive manufacturers can establish innovative business models based on this new technology, for example with entertainment offers or individually customised servicing packages that pilot the vehicle into the manufacturer’s own workshops.
At the same time, companies must adapt to shorter development cycles and new competitors from the IT and high-tech sector. Above all, however, there will be significantly less revenue from car sales: analysts from Barclays investment bank are working on the assumption that, thanks to car sharing and autonomous taxis, sales of private cars will fall by up to 50 per cent in the next 25 years. The companies still have time to prepare for these upheavals, as completely autonomous vehicles are not expected in complex road traffic until around 2030. However, in controllable environments such as agriculture or mining, self-driving vehicles are already in use today.

Autonomous vehicles do not just replace the driver, helmsman or pilot, but have the potential to create completely new business models worth many billion.

Diverse benefits

There are a wide range of arguments in favour of autonomous vehicles, with improved use of infrastructure and a reduction in the number of accidents being the key points of focus for road transport. After all, 90 per cent of all road accidents are due to driver error. The same is true of air travel – electronic systems remain alert 24 hours a day and the current microprocessors can react approximately 1,000 times faster than humans in dangerous situations. However, the staffing costs for pilots are also a reason for using autonomous aircraft. The same argument can be made in the construction sector for driverless vehicles – after all, autonomous excavators and HGVs will save up to 90 per cent of labour costs. At the same time, the machines can remain in use 24 hours a day, without the breaks that a person would require. Within the logistics sector, fully automated HGVs could therefore enable better usage of fleet capacity and make supply chains more efficient in the medium term. A shortage of workers is a further reason for fully automated vehicles: this is true of both HGV drivers and for maritime shipping. At the same time, autonomous vehicles also reduce the impact on the environment: with construction and agricultural machines, it should be possible to reduce the amount of CO2 produced by up to 60 per cent. What is more, completely new methods of farming can be implemented in the agricultural sector, making it possible to significantly reduce the use of spray agents and protect the soil.

Who is responsible?

However, there are still a number of ethical, legal and social questions that must be answered before we can make use of these advantages: after all, who is responsible for the “actions” of autonomous vehicles if users themselves are not involved in the vehicles’ decisions, or only marginally involved? And in the event of conflict, what criteria should machines use to “decide”, and who will determine said criteria? Nevertheless, the experts are certain that these questions will be clarified and that autonomous vehicles will trigger a revolution in mobility in the near future.

2 m is the maximum distance a current NASA Mars Rover is able to travel in one go before it has to stop and recalculate. (Source: NASA)

57 % of people worldwide would ride in a driverless car. (Source: Cisco Systems)

1⁄3 of land area in big US cities could be freed up by autonomous parking. (Source: www.2025ad.com)

50 % of a ship’s operating costs are down to the crew. (Source: Moore Stephens LLP)

70 % is the potential increase in global farming yield through the use of autonomous vehicles, drones and other related technologies. (Source: Goldman Sachs)

Thanks to quick-responding electronics, the safety clearance of connected truck convoys can be reduced from 50 to 15 m (Source: www.2025ad.com)

The post Autonomous vehicles are the future appeared first on Future Markets Magazine.

The roots of autonomous vehicles reach further back than is generally assumed: as early as the start of the 20th century, Elmer Sperry developed the first control system controlled by a gyrocompass, enabling ships to be kept on course automatically. Then, in 1928, the first automated aeroplane control system, developed by Johann Maria Boykow, was showcased at the International Air Exhibition in Berlin. However, true autonomous driving requires far more than simply keeping the vehicle on a set course: the vehicle must be able to reach a specified destination independently, without human control or detailed programming. In doing so, it must be able to respond to both obstacles and unforeseen events.

From assistance systems to self-driving vehicles

The path to a fully autonomous system is gradual, with developments on a sliding scale. A worldwide system with six levels for classifying degrees of automation is now recognised throughout the world – it was defined, among others, by SAE International (Society of Automotive Engineers), but is now also used for other vehicle segments. According to this scale, level 0 corresponds to a vehicle without any assistance system, where the driver is solely responsible for all functions. At level 1, the first assistance systems, such as cruise control, support the driver. Partly-automated vehicles with parking and lane guidance systems, which can already carry out automated steering manoeuvres, constitute level 2. At level 3, the vehicle controls itself for the most part, and the driver no longer has to oversee the vehicle at all times. The fully-automated vehicles classed as level 4 can master even high-risk situations without human help, but are restricted to known sections of road. Only at level 5 do we find completely autonomous driving, in every environment and all situations. Within limited areas, such as agriculture, intralogistics, light-rail systems or mining, highly and fully-automated vehicles classed as levels 3 and 4 have been in use for quite some time. However, only level 3 cars are currently found on the roads. The first series-production cars that can cope without drivers in real road traffic, at least under specific conditions (level 4), are set to be on offer from 2020 onwards.

Sensors pick up on the surroundings

In order for a vehicle to reach its destination autonomously, it requires various capabilities: firstly, it must pick up on the environment through which it is moving – otherwise it would simply fall at the first hurdle. To prevent this, autonomous vehicles are equipped with a very wide range of sensors: ultrasound sensors are required for automated driving, particularly for detecting close-up surroundings up to six metres away and at low speeds, for example when parking. Radar sensors provide important information on the environment at a greater distance, through 360 degrees. The main task of a radar sensor is to detect objects and to measure their speed and position compared with the movement of the vehicle on which it is fitted. A relatively new addition is lidar sensors, which “scan” the environment with invisible laser light and can generate a high-resolution 3D map of the surroundings. Video sensors, above all in stereo-video cameras, supply additional important visual information such as the colour of an object. Each of these sensor systems has its strengths and weaknesses. In order to obtain an image of the environment that is as exact and reliable as possible, multiple sensors are used together in autonomous vehicles – depending on the application – and the data from these is “merged” or drawn together.

High-resolution maps via the cloud

As well as the ability to “see” the surroundings, an autonomous vehicle must also be able to navigate. Thanks to satellite navigation systems such as GPS, the vehicles know where they are currently located and can calculate their route based on this information. In doing so, they rely on high-resolution maps that are kept extremely up to date, with these maps not just showing the topology, but also incorporating current events such as traffic jams in a dynamic manner wherever possible. These maps can be stored locally in the vehicle or in the Cloud. In the latter case, a high-performance communication system is particularly necessary, so that the map data can be updated in real time. For example, the 5G mobile telecommunications standard can form the basis for this. It enables a “tactile Internet” that, in addition to transmission rates in excess of ten gigabits per second, guarantees an ultrafast response with a delay of less than one millisecond. With this type of networking, the almost unlimited resources of Cloud computing can be called on to carry out complex calculations involved in analysing the situation or route finding.

Learning as a basis for the correct response

After all, analysing the huge data volumes that are generated by the vehicle’s sensor systems requires considerable computing capacity, as does interpreting situations. Technologies that are grouped under “artificial intelligence” are becoming increasingly important. Machine learning, in particular, is an essential part of an autonomous system: only with this is it possible for vehicles to act intelligently and independently of humans. Through machine learning, autonomous systems can generate new knowledge from data that has been collected and provided, and are able to constantly extend their knowledge base. Without this independent learning, it would be almost impossible to specify appropriate reactions to all theoretically possible situations in programming.

The post Foundations of autonomous vehicles appeared first on Future Markets Magazine.

In the late summer of 2016, a project in the Swedish town of Eskilstuna provided a glimpse of the construction site of the future: at the site, Volvo Construction Equipment (CE) demonstrated how an autonomous wheel loader and a driverless dumper truck worked together. The wheel loader handled around 70 per cent of the volume normally loaded by a human-controlled machine. That’s a lot less – though the autonomous construction machines are able to operate round the clock. “The machines are able to perform the same task on a predetermined route time and time again, over protracted periods of time. But the technology is still in its infancy. We are working to devise solutions capable of delivering the safety and performance the market demands,” explains Jenny Elfsberg, Volvo CE’s Director of Emerging Technologies. “We still have a long way to go. So we don’t yet have any plans for implementation on an industrial scale,” she adds. The machine prototypes do not yet communicate with each other, for example. Yet that is vital in terms of avoiding collisions and simplifying efficient material flow. Nevertheless, Elfsberg is certain: “Autonomous machines will improve safety in a hazardous working environment, and eliminate the risk of accidents caused by human error. They will also perform repetitive tasks more efficiently and precisely than a human operator.”

24/7 operation: Drones supply topographical information, autonomous machines then move the required volume of earth precisely and highly efficiently on this basis.

Autonomous construction machines working together

Japanese manufacturer Komatsu is also working on autonomous construction machines. Its “Smart Construction” concept is already well advanced. The excavators and wheel loaders are only partially autonomous so far, however: the excavator operator now merely controls the boom, for example, while the bucket operates automatically. Its height and position are adjusted with the aid of cameras on the excavator and GPS sensors. The system “knows” how much earth has to be moved where. The necessary data is provided by thousands of aerial photographs captured by drones from US manufacturer Skycatch. The Skycatch software uses the images to compute a three-dimensional topographical model, accurate to within three centimetres. Based on several million measuring points, the volume of earth to be moved can then be calculated very much more precisely than using conventional manual methods.

Fully automated iron ore transportation

It will be a while before fully autonomous excavators are available. But fully autonomous trucks are already operating today: Australian mining company Rio Tinto, for example, introduced so-called Autonomous Haulage Systems (AHS) at its iron ore mines in the Pilbara region as far back as 2008. Today the company is the world’s largest owner-operator of autonomous trucks, running 71 autonomous dumper trucks. Actually, the word “truck” does not quite cover it: the Komatsu driverless dumper trucks transporting iron ore at the Yandicoogina, Hope Downs 4 and Nammuldi mines are the height of a three-storey building.

Higher productivity, less risk for staff

The dumper trucks are equipped with powerful computers which control the standard driving functions: starting the engine, accelerating and braking. The navigation system is GPS-based, and the trucks are fitted with distance sensors and collision-avoidance systems in order to identify and avoid any hazards. The vehicles are additionally monitored remotely from an operations centre in Perth, 1,500 kilometres away. Rio Tinto’s Mining Operations Manager at Yandicoogina, Josh Bennett, explains: “What we have done is map out our entire mine and put that into a system, and the system then works out how to manoeuvre the trucks through the mine.” The trucks are programmed to transport their loads as efficiently as possible. And they are a success: since 2008, the autonomous fleet has cut loading and transportation costs at the mines by 13 per cent. “Autonomous trucks reduce employee exposure to hazards and risks associated with operating heavy equipment, such as fatigue-related incidents, sprains and other soft-tissue injuries, and exposure to noise and dust,” says Yandicoogina Mining Operations Manager Josh Bennett.

The post Autonomous construction machines appeared first on Future Markets Magazine.

Now that we have driverless cars and increasingly intelligent drones, an autonomous aircraft without a pilot on board hardly seems visionary any more. In fact, as far back as some ten years ago, the IFATS (Innovative Future Air Transport System) project demonstrated the technical feasibility of aircraft without pilots, although it was thought at the time that this would not happen before 2050. The biggest hurdle was considered to be passenger acceptance. However, acceptance levels are likely to increase every time there is a pilots’ strike or an aviation accident, because up to 90 per cent of aviation accidents are attributable to pilot error. This explains why all large aircraft manufacturers – along with many small start-ups – are working on unmanned aircraft.

90% Pilot Error: The majority of all aviation accidents can be traced back to human error. It is hoped that autonomous systems will considerably reduce this figure.

Nothing escapes the electronic eye

British manufacturer BAE Systems started testing a pilotless aircraft at the end of 2016. A standard Jetstream 31 plane was converted to provide a flying test bed. The machine has an antenna which detects transponder signals from other aircraft as well as a cockpit-mounted camera acting as an electronic eye. This links to the aircraft’s computer systems and enables the Jetstream to “see” potential hazards, even if no signals are being emitted. The electronic eye of the Jetstream can also recognise different cloud types and, if needed, plot a course that allows evasive action from challenging weather conditions. Pilots on board the aircraft remain responsible for take-off and landing, but as soon as the Jetstream is in the air, it flies autonomously. It has already clocked up almost 500 kilometres of flying at a height of approximately 4.5 kilometres. “Our priority, as always, is to demonstrate the safe and effective operation of autonomous systems,” explains Maureen McCue, BAE Systems’ Head of Research and Technology for the military aircraft and information business. “The trials will give us technology options that could be applied to our own manned and unmanned aircraft, as well as potentially enabling us to take some new unmanned aircraft technologies to market.”

Requiring continuous corrections about all three axes, a helicopter is much more difficult to control than a fixed wing aircraft like the Jetstream. Yet for this type of craft too, the first test flights are already under way. In June 2017, Airbus Heli-copters started test flights with a preliminary study involving the VSR700. The prototype of the light optionally piloted vehicle (OPV) helicopter is due to take off in 2018. Optionally piloted means that the machine can fly both autonomously and with a pilot. “The OPV is able to autonomously take-off, hover and perform stabilised flight and manoeuvres,” said Regis Antomarchi, Head of the VSR700 programme at Airbus Helicopters. This phase of flight trials with a safety pilot will focus on refining the automatic flight control system aboard the helicopter, eventually leading to fully autonomous flights without a safety pilot.

A new dawn for local transport 

Airbus actually wants to go much further with autonomous aircraft. The Group has a number of visions associated with this technology in its test laboratories, with the focus being on the urban traffic of the future. A3, a subsidiary company based in Silicon Valley, is currently working on a project called Vahana. Vahana is a self-piloted flying vehicle platform for individual passenger and cargo transport – an air taxi, essentially. “The ability to be transported safely and quickly through a city in a self-piloted aircraft is no longer science fiction,” says Rodin Lyasoff, CEO of A3. “Advances in propulsion, battery performance, air traffic management, autonomy and connectivity mean that this mode of transportation is capable of benefitting millions of people in years, not decades.” In his view, the only remaining challenges to be overcome are associated with reliable sense-and-avoid systems, which detect hazards or obstacles and invoke evasive action. At the current time, there are not yet any fully developed solutions for aviation. “Urban air mobility will significantly change how we live and work for the better, but bridging from feasibility to reality will require close cooperation between the public and private sectors to define appropriate regulations,” says Lyasoff. Another Airbus vision for urban air traffic is the City Airbus. In terms of technology, this aircraft is comparable to a small drone. Like a drone, it relies on several electrically driven propellers to stay in the air. Where it differs from a drone is that it will be designed to carry several passengers. To speed up its time to market, the machine will initially be controlled by a pilot. The intention, though, is that it will fly autonomously at a later point in time and passengers will use an app to hail a ride. A feasibility study has already been successfully completed.

(Picture credit: Airbus)

The post Autonomous Aircraft: A new way to fly appeared first on Future Markets Magazine.

Nowadays, sensor systems such as lidar, radar and cameras can already provide a highly precise image of an autonomous vehicle’s environment. These vehicles can react to what is happening around them in combination with artificial intelligence and complex algorithms – almost like a human being. Yet like people, the vehicles can only use these sensors to react to things in their field of view. Consequently, to be able to “look around the corner”, autonomous vehicles will have to be connected and capable of communicating with their surrounding area and infrastructure.

“The transfer of data to vehicles is a core technology for automated driving, for real-time application, as well as for maintenance and service,” Armin G. Schmidt, CEO of Advanced Telematic Systems (ATS), emphasises. The German software company focuses on connected vehicles and develops solutions for the future of mobility, for which it relies on the establishment of industry-wide standards and open-source technology. “The exchange of data between vehicles, such as between different provider platforms, also requires further standardisation development,” Schmidt continues.

Warning the Surrounding Area in Milliseconds

Standard ITS-G5 of the European Telecommunications Standards Institute (ETSI) has been established for the communication of vehicles in road traffic. This is a variant of the IEEE 802.11 WLAN standard that has been optimised for data exchange between vehicles and has since also been recognised in the United States. It uses the 5.9-GHz frequency band and makes communication possible over short distances in close to real time. The technology ensures a reliable transfer at a high vehicle speed, as well as allowing direct communication between individual vehicles (car-to-car) and between the vehicle and infrastructure (car-to-X, vehicle-to-X or V2X) without a router. This means that sudden events or hazardous situations can be communicated to the surrounding area within only a few milliseconds. For example, if a car has been involved in an accident or a bank of fog forms ahead, it can automatically warn approaching traffic heading in the same direction. “The benefits of safety and awareness of V2X as a sensor – with its ability to ‘see around the corner’ – have already been proven beyond doubt as a means to providing relevant and reliable early warning messages for advanced driver assistance systems,” explains Jozef Kovacs, CEO of US firm Commsignia, a provider of software and hardware solutions for connected cars.

In the future, however, road vehicles will most likely not only be equipped with communications technology. Initial solutions are implementing an “adaptive hybrid network concept”. This includes integrating various wireless technologies – such as ITS-G5, LTE mobile communications or 60-GHz technologies – into a communications stack (the waveform and the typical receiver algorithms of the IEEE 802.11ad WLAN standard are used here, providing a common framework for vehicle communication and radar technologies at 60 GHz). The ideal communication technology is selected adaptively, meaning that it depends on the situation in real time. Criteria for selecting wireless technology include its predicted availability or its signal quality.

Quantity of data generated each day by an autonomous vehicle.
Source: Intel

Cameras: 20-24 MB each second
Lidar: 10-70 MB each second
Sonar: 10-100 KB each second
Radar: 10-100 KB each second
GPS: 50 KB each second

To the Cloud with 5G

Wireless technologies offer the advantage of unlimited range, whereas ITS-G5 can only bridge distances of up to one kilometre. For example, LTE networks are already in use in the automatic emergency call system ECall. Though yet to be implemented, the 5G network will assume this role in the future. “We expect 5G to become the worldwide dominating mobile communications standard of the next decade,” says Dr Christoph Grote, Senior Vice President of Electronics at BMW, adding that: “for the automotive industry, it is essential that 5G fulfills the challenges of the era of digitalisation and autonomous driving.” For this reason, BMW, along with Audi, Daimler, Ericsson, Huawei, Intel, Nokia and Qualcomm, founded the 5G Automotive Association which aims to develop, test and promote communications solutions. “Cloud, communications and networking technologies and innovations have the potential to transform the car into a fully connected device to revolutionise the driver experience and address society’s mobility needs,” explains Dr Marc Rouanne, Chief Innovation & Operating Officer at Nokia.

“Cloud, communications and networking technologies and innovations have the potential to transform the car into a fully connected device to revolutionise the driver experience.” 

Dr Marc Rouanne, Chief Innovation & Operating Officer, Nokia

Swarm Data Paves Way for Automated Driving

These types of technologies, for instance, provide the means to share the experiences that an adaptive autonomous vehicle has gained with other vehicles via the Cloud. Cloud technologies are also opening up new possibilities for navigation. By way of example, Mobileye, an Israeli company that creates accident prevention and automated driving technologies, has developed the camera-based mapping and localisation technology Road Experience Management (REM). Real-time data from numerous vehicles – a swarm of cars – is collected via crowdsourcing before being used for precise localisation and to record high-definition lane data. To accomplish this, the vehicles deploy optical sensor systems to detect road markings and road information, which flow to the Cloud in compressed format. This fleet data is used to continuously improve HD navigation maps with high-precision localisation capability and, in turn, is a basic necessity for automated driving and for assistance systems to be further developed. “The future of autonomous driving depends on the ability to create and maintain precise high-definition maps and scale them at minimal cost,” co-founder and Chief Technology Officer of Mobileye, Prof. Amnon Shashua, summarises. At the start of 2017, Mobileye made an agreement with the Volkswagen Group to implement a new navigation standard for automated driving from 2018 onwards. Future Volkswagen models will use REM. The agreement will facilitate the worldwide consolidation of data from different automobile manufacturers into an “HD world map”, the first of its kind. That will form an industry-wide standard. According to Dr Frank Welsch, Member of the Board of Management of the Volkswagen brand with responsibility for development: “Every day, millions of Volkswagen vehicles drive on our streets. Many of them are equipped with sensors to monitor the surroundings. We can now utilise this swarm to obtain various environmental data in anonymised form related to traffic flow, road conditions and available parking places, and we can make these highly up-to-date data available in higher-level systems. More services are planned which will be able to utilise these data and make car driving and mobility easier with greater convenience and comfort overall.”

(picture credit: Unsplash)

The post Communication in autonomous vehicles appeared first on Future Markets Magazine.

For a vehicle to drive autonomously, it is not enough to simply equip it with a large number of sensors for detecting the immediate surroundings. It must also be able to handle the huge volumes of data, and to do so in real time. This overburdens conventional computer systems. The solution comes from electronics and software that provide the means for imitating the functions of the human brain. Artificial intelligence (AI), cognitive computing and machine learning are terms used to describe different aspects of these types of modern computer systems. “In essence, it is all about emulating, supporting and expanding human perception, intelligence and thinking using computers and special software,” says Dr Mathias Weber, IT Services Section Head at the German digital industry association Bitkom.

Growing Demand

Nowadays, artificial intelligence is used as standard; for instance, it is embedded in digital assistants like Siri, Cortana and Echo. The basic assumption of AI is that human intelligence results from a variety of calculations. This allows artificial intelligence itself to be created by different means. There are now systems whose main purpose is to detect patterns and take appropriate actions accordingly. In addition, there are variants known as knowledge-based AI systems. These attempt to solve problems using the knowledge stored in a database. In turn, other systems use methods derived from probability theory to respond appropriately to given patterns. “An artificial-intelligence system continuously learns from experience and by its ability to discern and recognise its surroundings,” says Luca De Ambroggi, Principal Automotive and Semiconductor Analyst at IHS Technology. “It learns, as human beings do, from real sounds, images, and other sensory inputs. The system recognises the car’s environment and evaluates the contextual implications for the moving car.” In terms of AI systems built into infotainment and driver assistance systems alone, IHS expects sales to increase to 122 million units by 2025. By comparison, the 2015 figure was only 7 million.

New Processors for Artificial Intelligence

The roll-out of artificial intelligence also has direct impacts on processor technology: conventional computational cores, CPUs, are being replaced with new architectures. Graphics processing units (GPUs) have thus been viewed as a crucial technology for AI for several years. CPU architectures perform tasks as a consecutive series, whereas GPUs – with their numerous small and efficient computer units – process tasks in parallel, making them much faster where large volumes of data are concerned. The new chips’ control algorithms already contain elements of neural networks, which are used in self-learning machines. A neural network of this type consists of artificial neurons and is based on the human brain in terms of its workings and structure. This enables a neural network to make highly realistic calculations.

Tyres with AI

In 2016, tyre manufacturer Goodyear introduced the concept of a spherical tyre featuring artificial intelligence. With the aid of a bionic “outer skin” containing a sensor network, along with a weather-reactive tread, the tyre can act on the information it collects by directly implementing it in the driving experience. It connects and combines information, processing it immediately via its neural network, which uses self-learning algorithms. This allows the Eagle 360 Urban to make the correct decision every time in standard traffic situations. Its artificial intelligence helps it to learn from previous experiences, enabling it to continuously optimise its performance. Consequently, the tyre adds grooves in wet conditions and retightens when dry.

Adaptive Control Systems

Like human beings, cognitive computing systems can integrate information from their immediate surroundings – though rather than eyes, ears and other senses, they use sensors such as cameras, microphones or measuring instruments for this purpose. The new processor architectures give vehicles the ability to evaluate these huge data volumes, and to constantly improve and expand these evaluations. This machine learning is seen as a key technology on the road to artificial intelligence. Machine learning also includes deep learning, which interprets signals not by relying on mathematical rules, but rather knowledge gained from experience. In this case, the software systems change their programming by experimenting themselves – the behaviour that leads most reliably to a desired result “wins”.

Several automotive suppliers are now offering control systems pre-equipped with deep learning capabilities. Contemporary electronic control units (ECUs) in vehicles generally consist of various processing units, each of which controls a system or a specific function. The computing power of these units will no longer be adequate for autonomous driving. AI-based control units, on the other hand, centralise the control function. All information from the various data sources of an autonomous vehicle – including from infrastructure or from other road users – are gathered here and processed with a high-performance AI computing platform. In this way, the control system comes to “understand” the full 360-degree environment surrounding the vehicle in real time. It knows what is happening around the vehicle and can use this to deduce actions. Jensen Huang, CEO of Nvidia, works with his company to partner with various automotive manufacturers in developing control systems of this type. He is certain of one thing: “Artificial intelligence is the essential tool for solving the incredibly demanding challenge of autonomous driving.”

(picture credit: istockphoto: BlackJack3D, Feverpitched)

The post Artificial Intelligence in autonomous vehicles appeared first on Future Markets Magazine.

Autonomous vehicles are nothing other than mobile computers with innumerable communications interfaces – exchanging data with infrastructure or other vehicles, updating on-board software, or accessing real-time navigation maps. However, the increasing number of interfaces aboard vehicles also means there are more potential vulnerabilities for cyber-attacks. “Hundreds of articles on autonomous driving appear in the media every day, but almost none mention the elephant in the room: auto-makers do not yet have a reliable defence against cyber-threats. Period. One serious hack could immediately halt progress in automated driving. But we have the remedy,” says David Uze, Trillium’s CEO. Consequently, the Japanese company founded in 2014 is planning to launch a software-based, multi-layer security solution onto the market in 2018 – at a tenth of the cost of existing solutions. “Since defence must continually evolve, our infrastructure will deliver Security as a Service (SaaS) via real-time-update platforms that auto-makers or insurers can on-sell to car owners.”

There is no wonder-weapon

Whether a purely software-based add-on cybersecurity solution is enough on its own to protect a vehicle against the highly sophisticated attacks of modern-day hackers is questionable, however. “There is no wonder-weapon capable of protecting cars against sophisticated dynamic cyber-attacks,” stresses Ofer Ben-Noon, co-founder and CEO of Israeli vehicle cybersecurity company Argus. “Our customers need protection on multiple levels, so as to be prepared for any conceivable scenario.” The company offers a multi-layer security solution for connected vehicles: it starts with the infotainment and telematics devices, encompasses the internal network communications, and also extends to selected electronic control units (ECUs). ECU security protects vital systems such as the brakes, assistance systems and other key units against attack.

Cybersecurity as part of the development process

Cybersecurity should be integrated into the development process for an autonomous vehicle right from the start as a matter of policy. That principle is affirmed in a manifesto published by FASTR, stating that cybersecurity should begin at the very foundation of the vehicle’s architecture and be coordinated throughout the supply chain. In that way, a connected vehicle can be made “organically secure”. FASTR – which stands for Future of Automotive Security Technology Research – is a neutral, non-profit consortium established in 2016 by three companies: Aeris, Intel Security and Uber.

The provision of such all-round protection should be approached according to the bottom-up principle: the security concept starts with a high-security core (root of trust), implemented by a physically secured cryptographic device such as a Hardware Security Module (HSM). It securely holds cryptographic keys and algorithms, protecting them against being read, modified or deleted. The keys are in turn used to detect and prevent manipulation of the ECU firmware. This then also ensures that the software-based security functions in the firmware can be used safely for on-board communications. At the same time, it means on-board networks on different security levels are reliably isolated from one another – preventing access to the engine management system via an entertainment interface, for example. The secure on-board network this creates then also in turn permits secure communication with other vehicles or infrastructure.

Autonomous vehicles are protected against the many types of cyber-attack by multi-layered security concepts.

Hardware security module

More secure E/E architecture

Secure on-board communication

Secure vehicle IT infrastructure

Secure control unit

Secure V2X communication

Security systems must be updateable

The challenges of cybersecurity are changing continually, however. Security experts are continually having to confront new conditions and methods of attack. That means a vehicle’s on-board security systems must be capable of being regularly updated throughout the product life-cycle. Consequently, a security solution for autonomous vehicles should be designed right from the start in such a way that vital security parameters and functions are held in modifiable storage devices (such as HSMs with firmware update facilities). Also, available IT resource capacities should not be fully utilised right from the start – leaving adequate spare storage space, for example. With appropriate update mechanisms, new security patches can then be downloaded “over the air” – and the vehicle can stay protected against attacks from cyberspace even ten years down the line.

“Car hacking is a very real threat that will continue to increase as we move towards greater connectivity and autonomous vehicles, with more and more new technologies becoming part of the Internet of Things,” says Saar Dickman, Vice President, Automotive Cyber Security at Harman. “Automotive cybersecurity is an increasingly critical piece in enabling connectivity and autonomous driving.”

(picture credit: istockphoto: Kodochigov)

The post Cybersecurity – Protection for autonomous vehicles appeared first on Future Markets Magazine.

Over 1.2 billion people spend more than 50 minutes a day in their cars – a large portion of that time, however, is spent in traffic jams. Wouldn’t it be nice if you could take your hands off the steering wheel during this time and get on with other things? This dream became reality in 2017: this year has seen the presentation of the world’s first production cars to be developed for highly automated driving. Vehicles can now take over driving functions such as parking or autonomously accelerating and braking in traffic jams.

Sharp eye for the surroundings

A basic requirement for automated driving is the ability to reliably perceive vehicle surroundings and evaluate them accurately on the fly. “In order for the system to acquire this information step-by-step, a range of sensors such as radars, cameras, and surround view systems are needed. The aim is to achieve an understanding of the vehicle’s surroundings which is as good as or better than a person’s own understanding. More range, more sensors, and the combination of acquired data with powerful computer systems will help to sharpen the view and is the key to achieving a consistent view of our surroundings,” says Karl Haupt, Head of Continental’s Advanced Driver Assistance Systems business unit.

British car-maker Jaguar Land Rover has taken this a step further. “We don’t want to limit future highly automated and fully autonomous technologies to tarmac,” says Tony Harper, Head of Research, Jaguar Land Rover. “When the driver turns off the road, we want this support and assistance to continue. In the future, if you enjoy the benefits of autonomous lane-keeping on a motorway at the start of your journey, we want to ensure you can use this all the way to your destination, even if this is via a rough track or gravel road.” For this purpose, Jaguar Land Rover has combined cameras, ultrasound, radar and lidar sensors in a concept vehicle. These systems not only enable a 360-degree view of the car’s surroundings, but are so highly developed that they can determine surface properties down to the dimension of a tyre width – even in rain or snow. Ultrasonic sensors can also detect the surface conditions within a range of up to five metres so that the vehicle can automatically adjust its traction and driving behaviour when switching from tarmac to snow or from grass to sand.

The car comes to the driver

The sensor system is only one part of the solution; the other is having the intelligence to generate commands from the data collected. This requires cars to be equipped with high-performance control units. The Audi A8 revealed in July 2017, for example, features a central driver assistance controller with deep-learning-based software which constantly froms an image of the surroundings from the sensor data during piloted driving. Daimler is also pushing to develop the software behind fully automated and driverless driving. In April 2017, the manufacturer entered into a development agreement with Bosch to bring fully automated and driverless driving to urban roads by the beginning of the next decade. The objective is to work together to develop software and algorithms for an autonomous driving system. The idea behind this thinking is that the car will come to the driver and not the other way around. Users will be able to conveniently order an automated shared car or robot taxi via their smartphone. The vehicle will then make its way autonomously to the user. “The car as we know it will soon be history,” says Dr Volkmar Denner, Chairman of the Board of Management of Robert Bosch. “Today you use the Internet to book a hotel room; in the future, you’ll arrange your mobility online as well.”

An autonomous fleet can effectively replace a much larger number of private vehicles.

Tapping into new business areas

Car-sharing is one of the big advantages of autonomous cars. David Alexander, Senior Research Analyst with Navigant Research: “Studies have shown that an autonomous fleet can effectively replace a much larger number of private vehicles in a city centre, which represents both an opportunity and a challenge for original equipment manufacturers (OEMs).” On the one hand, according to Navigant, 120 million autonomous cars will be sold between 2020 and 2035. On the other hand, automobile manufacturing is expected to reach its zenith and then decline because of shared cars. Therefore, automotive manufacturers should adapt their business models accordingly and sell additional value-added services, for example. “The more popular autonomous driving becomes, the greater the demand by users for services to meaningfully utilise the time freed up in the car,” concludes Ralf Gaydoul, Partner and Head of the Automotive Center at Horváth & Partners Management Consultants. “If the values were to be added up across all categories of need, this would give rise to a monthly amount of well in excess of 100 euros per driver.” Together with the Fraunhofer Institute for Industrial Engineering (IAO), Gaydoul has studied the willingness of motorists to pay for such services: according to the study, three-quarters of those surveyed would pay for value-added services. The willingness to pay for services is at its highest in relation to communication and productivity. “These services are the most heavily in demand in all three countries examined, though with different variations,” says Dr Jennifer Dungs, Head of the Mobility and Urban System Design Division at the Fraunhofer IAO. “For example, interest in in-car social media services is much higher in Japan than here in Germany (64 per cent compared with 23 per cent).”

(Picture Credit: Continental)

The post The future of autonomous driving appeared first on Future Markets Magazine.