Earlier this year I wrote some thoughts on Artificial Intelligence (AI). The concept of AI first originated in the mid-1950s as attempts to develop machines more like humans. Machine learning is a subset of AI and emerged in the mid-1980s. Deep learning techniques have only been refined over the past three or four years, and are a subset of machine learning.

Machine learning is an attempt to design generic algorithms that can make sense of data. So for example, if one has a data set on attributes of used cars such as make, model, mileage, condition, and price, plugging this data into generic machine learning algorithms could work out a formula for the price of a used car based on the inputs of the other data. Machine learning algorithms enable this without having to create specific code for used vehicles – generic algorithms can suffice.

With machine learning, data is fed into generic algorithms to “train” the system how to calculate the result. In the example of used cars above, the training data can be used to “teach” the system how to predict the price of a vehicle if attributes such as make, model, mileage, condition, etc. are known. This is known as supervised machine learning – where the training data includes “example answers”, i.e. car prices are included in the data set. An example of a supervised machine learning application is predictive text on a mobile keypad, or a spam filter where emails are classified as spam or legitimate based on learning from a training data set. SentryBay classify web pages as phishing or legitimate based on supervised machine learning Bayesian algorithms.

Unsupervised machine learning involves feeding data sets into the system without “example answers”, the algorithms are used to figure out relationships in the raw data. With large data sets, machine learning can be employed to discover relationships that are not otherwise apparent.

Deep learning has recently evolved specific techniques of machine learning. Deep learning has fuelled the recent explosion of AI applications, all made possible by expansions in computing processing power and storage.

Artificial Neural Networks (ANN) were initially an attempt to emulate the functioning of the human brain. The concept has evolved into layers of neural networks and convolutional neural networks (CNN) where masses of data is used to train systems. An example is in teaching a system to identify individual objects within an image. The system needs to be able to isolate individual objects within the image and then accurately identify them. Google refined the technique in developing a process of identifying the image of a cat in a video images. The training used 10 million images from YouTube videos.

Deep learning has now made the machine identification of individual objects within an image more accurate than what can be achieved by a human. The technology is used in facial recognition, translation services and in autonomous vehicles. In the medical field the process is used to identify cancer in blood and tumours in MRI scans. Other current applications are in natural language processing and drug discovery. And of course there is exciting potential for digital security.

I’ve written at various times over the years about the short-comings of the dominant end-point security technology – signature-based malware scanning. In the early years, my views were contrary to prevailing industry thinking, but as more and more evidence came to light the failures of traditional antivirus are now widely accepted. The big question is what is the replacement for signature-based scanning? Although minor technologies will successfully solve important peripheral security issues, recent deep learning advancements have now opened the path for AI to become the dominant core of end-point security, network monitoring and IoT security. Opportunities for cloud-based AI also abound in mobile security, a field so far lacking in PC security equivalents.

The latest Symantec internet security threat report highlights that 430 million new unique pieces of malware were identified in 2015. Everyone, including Symantec now agrees that traditional AV cannot cope with this huge volume. Effective solutions lie within AI. Machine learning and greater intelligence from deep learning are central technologies for our future digital security.