Artificial Intelligence and Machine Learning developments and issues towards autonomous driving

Machine learning (ML) and artificial intelligence (AI) are key enabling technologies that will empower proliferation of automated (Level 3) and autonomous (Level 4 – 5) vehicles (AV) into everyday lives.

Machine learning for sensor fusion in intelligent transportation
Safe, reliable and perceptive autonomous driving is relying on machine learning as a prerequisite step towards intelligent transportation domain specific AI. However, learning itself requires access to stimuli rich environment on one side and learning goals on the other. Gained experiences from sensing, sensor fusion and learning the environment is key to AVs self thought navigation within known and new, previously unknown streets, situations and interactions with other vehicles, pedestrians, cyclists. Transportation industry is very involved in developments of various sensor types that enable 360 degree sensing, seeing, hearing and almost feeling surrounding the AVs environment, for example: LiDAR, radar, IR cameras, video cameras, ultrasound.  

Evolution paths of Machine Learning towards Level 5
An interesting RAND study from 2016 was modeling the requirements of AVs to achieve safe and reliable operation on par with human driven vehicles. Analysis was based on known performance of then developed AVs under testing and statistical figures of reported disengagements of driven AVs on roads or test sites under controlled environments.

ML has made major development strides since, beyond classical deep learning, like playing GO and surpassing best human players, learning to walk a simulated humanoid, devising new models, namely GAN and Capsule networks. These achievements promise significantly better learning abilities, faster objective oriented self-learning and requiring much less sensor data samples to achieve performance levels of previously known solutions.

ML new algorithms’ capabilities are essential component that will eventually lead to intelligent processing of sensory information and safe behaviour of AVs also under not previously known situations. There are numerous tool sets available, some even free, that put working with ML and neural network algorithms to sufficiently high abstraction level so that intricate knowledge of mathematical foundations of these algorithms is not required. Web access to cloud tool sets with venerable Python integration,  like Jupyter is even more popular during development phases of solutions, where we focus on proof of concept and accuracy performance rather than real time execution is key objective.

Machine Learning integration into Autonomous Vehicles may prove challenging
Transfering solutions into real environment, i.e. AV, under strict real time execution constraints may prove challenging. Real time reaction requirements to events in AV environments is counted in milliseconds, regardless of sensor source (vision camera or LiDAR), e.g. detection of pedestrian crossing road or car infront breaking. Path to real time event processing is a two pronged requirements issue:

  1. CPU processing capabilities for ML algorithms (FLOPS, OPS)
  2. Power consumption of solution (W)

CPU processing vs power consumption issues are evolutionary solved by technology drivers, typically letting Moore’s law do the job. Forthcoming AVs will require more aggressive approaches, if we are going to meet the 2022/2025 predicted time frames of Level 5 autonomous solutions.  Advances in technology are solved by disruptive advances, i.e. jumping the technology S curves, by adoption of radically new technological solutions. Deep learning produced a typical jump on the S technology curve for ML algorithms.   

On the CPU processing side there is light at the end of the tunnel, too. One school of thought is pushing the parallel execution model of advanced classical CPU architectures, namely GPU and recent TPU or neural network processing engines. These architectures are really descendants and high integration variations of decades of work that in earnest started at mid of 1970’s, with much advanced front end software support for many models of parallel algorithm development and execution.

The second approach is taking radically new architectures for processing of sensory signals, e.g. those based on neural networks that closely mimic how human brain work on micro level. The most interesting aspect of these solutions is the innovative implementation of neural processing, not by mathematical operations (e.g. multiply-accumulate, non-linear threshold function), but rather other means, e.g. width modulated signaling. Typical example is TrueNorth that can handle most complex 2D sensor, vision processing tasks in real time and very low power consumption, even two orders of magnitude lower compared to previously mentioned approaches, while employing 1 M neurons and 256 M synapses.

Is power consumption a showstopper for Autonomous Vehicles embedded Machine Learning?
Level 5 autonomy will undoubtedly require huge processing capabilities and looking from the perspective of today’s architectures there are estimates of 300 TFLOPS, i.e. 3*10^14 floating point operations per second to achieve the goal. Let’s assume that such are based on state of the art, classical implementations, e.g. GPU based, this results in at least 50 GPU cards, consuming towards > 6 kW. Not taking into account the space to accommodate such solution. Power consumption will be one of the issues for AV embedded solutions, since the majority of future AVs are going to be powered by electric motors and batteries, where their energy density is required to push the operating radius of electric vehicle, primarily. The good news is that there are specialised solutions on the horizon that will bring the numbers down for a factor 5-10. This is good news, and key enabler to make AVs a ubiquitous reality in the forthcoming decade.

Radovan Sernec,

Director, Innovation and Technology at AV Living Lab