In 2019, Artificial Intelligence (AI) advanced at an astounding pace without many headline-grabbing AI applications. Consider this: during the year the most sophisticated natural language understanding (NLU) models achieved 'superhuman' capabilities and are now able to perform various real-world tasks. This was a major milestone for AI development but was largely overlooked.
Natural language processing
The General Language Understanding Evaluation (GLUE) is a prominent benchmark used to evaluate NLU systems. The advantage of this benchmark over traditional, single dataset tests is the ability to assess a model’s capacity for handling out-of-domain data, how sample-efficient it is, and its level of self-learning and knowledge transfer among different tasks, all of which are core areas of machine learning. Overall, human performance in these sets of tasks has had an average score of 87.1. Since the Google AI Brain team’s NLU model first surpassed human performance with a score of 88.4 in June 2019, seven other teams’ models have done the same. It is no surprise that Baidu, Alibaba, Microsoft and Facebook are among those seven.
With the continuous improvement of training transformer-based models like Microsoft’s, proliferation of the applications built on this technology would be transformational for many industries with profit pools in impacted industries likely to shift.Natural language processing (NLP), a combination of NLU and natural language generation, remains one of the most challenging AI applications. A massive number of parameters are required in NLP models to enable knowledge transfer, just one of multiple necessary tasks, which increases computing intensity dramatically. Microsoft has been working on a transformer-based generative language model; this would allow systems to generate words to complete an unfinished sentence based on context, respond to a question with direct answers and summarise articles in the same way humans can. To do so, its latest Turing Natural Language Generation (T-NLG) model features 17 billion parameters to learn from essentially all the text published on the internet, compared to 26 million parameters in a typical image recognition model. Extremely short latency is also required as the response gap in natural conversation is just 200 milliseconds. With only several months’ training, Microsoft’s T-NLG has successfully surpassed human performance, with 98% grammatical correctness and 96% factual correctness, compared to the average human’s score of 97% and 92% respectively.
The implications of this astonishing progress in NLU are profound. Today, it is prohibitively expensive to collect annotated, supervised data from business activity; this is the only major gating factor for the wider adoption of real-time analytics. With the continuous improvement of training transformer-based models like Microsoft’s, proliferation of the applications built on this technology would be transformational for many industries with profit pools in impacted industries likely to shift. That is why NLP is at the forefront for every major technology giant.
Given its strategic importance, 2019 was unsurprisingly another record year for both AI-related M&A activity and AI fundraising. While 231 AI-related deals were completed, 2,235 transactions raised $26.6bn for AI start-ups despite growth decelerating from 31% in 2018 to 20% in 2019. Of this, $4bn went to the healthcare sector, nearly twice the $2.2bn received by finance, the second largest recipient. Across all sectors, businesses actively embraced AI-infused automation technologies and incorporated them into workflows to streamline operations in a new world informed by data.
Robotic process – automation and limitation
Robotic process automation (RPA) quickly became a hot area for investors last year. Compared to NLP and image recognition, which utilise complex models with millions of parameters and require extensive training processes, RPA is one of the simplest AI applications, performing structured and repetitive digital tasks.
Generative Adversarial Networks (GANs) have attracted a great deal of attention since the director of AI research at Facebook called it “the most interesting idea in machine-learning in the past 10 years”.Despite several advantages, such as including the transparency of decision-making and speed to implement, current uses are limited to data extraction and manipulation according to predefined rules, largely due to the scarcity of datasets that are both relevant and structured. Efforts have been made by the RPA solution providers to integrate image recognition into the process, however this significantly increases computing intensity and is a burden that many businesses cannot afford. While moving certain workflows to the cloud might solve the computing requirement issues, the increased complexity of the necessary integration with existing infrastructure, means the cost might outweigh the benefits. That said, RPA remains an attractive proposition in industries where the cost of obtaining structured datasets is low. For example, US nurses spend, on average, 25% of their working time on regulatory and administrative activities; this could largely be saved if RPA is implemented as a part of standard workflows.
Controversies and the outlook for AI
As with many new technologies, there are often controversies during the early stage of adoption. Generative Adversarial Networks (GANs) have attracted a great deal of attention since the director of AI research at Facebook called it “the most interesting idea in machine-learning in the past 10 years”. The basic framework is to simultaneously train two neural networks with the aim of one tricking the other, like a minimax two-player game. This model proved efficient in information-generated machine training, such as image generation, voice generation and video generation. It soon became apparent that this framework could be trained to generate new data that mimic any distribution of data, opening the floodgates for an exponential increase in AI-based media production.
On the positive side, GANs can greatly reduce the time spent in media creation. A portrait generated by AI using a GANs model was sold for $432k at Christie’s. However, on the negative side, this is the technology underpinning so-called ‘deepfakes’, often seen on social media where the model has been exploited to transpose faces into video clips or transform voices to give the impression that someone did or said something they did not. So many deceptive videos and audio have been generated, causing significant social implications, and the issue is so serious that a few US states, including California and Texas, have passed laws banning the use of deepfakes for fear of interference with elections.
Privacy concerns about AI applications have been present since the technology was born. The European Commission tabled a plan to ban facial recognition in public places for up to five years until new laws could be created to bolster regulation surrounding privacy and data rights are introduced, a move mirrored by San Francisco. Lawmakers are rightly cognisant of the risks and it is imperative that industry participants and regulators establish appropriate governance to facilitate the rapid and healthy development of AI, which has the power to be the most transformative technology of this generation.