Biological intelligence

Given the speed at which computers can cycle through generations of mutated programmes, evolution may be a promising new approach for self-learning

Biological evolution works through random mutation coupled to the paradigm of survival of the fittest. Random mutations occur all the time in the DNA of every living species. Some mutations are harmful. For example, diseases such as Huntington’s Disease or Thalassemia are due to genetically inherited mutations.

But some mutations are beneficial. For example, a mutation that enables resistance to malaria, or lowers the cholesterol, or increases bone density, is beneficial. Individuals who carry these mutations pass down inheritable characteristics to descendants, who are “fitter”. This process is random and it works over generations.

Computer scientists working in the field of artificial intelligence (AI) are now exploring the idea of using random mutation mechanisms to create new algorithms. This is an extension of the “biological borrowing” used to design neural networks.

Neural networks are modelled to work the same way the brain does. In a neural network, different clusters of “neurons” perform different tasks and researchers strengthen the connections between these clusters to make the algos more efficient.  

In a new paper, (https://arxiv.org/abs/2003.03384), a research team from Google, led by Quoc Le, have described how they are developing an upgraded version of the language, “AutoML Zero”, to use the power of evolution. AutoML is a popular machine learning tool, since its open-source. As the name suggests, it is designed to automate the processes of developing algorithms for Machine Learning (ML). The current public version uses manually created algorithms, which are fed into the system and fine-tuned automatically, by fiddling with new parameters.  

The new version, the “Zero”, will develop its own algorithms. The abstract of the paper claims that “it is possible today to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space.

Essentially the new program is fed with the rules of basic high-school mathematics, and left to discover more for itself. It starts by randomly combining mathematical operations to create multiple batches, or populations, with 100 candidate algorithms in every batch. It tests these algorithms on standard ML tasks such as recognising images.

The test results are compared with manually created algorithms performing the same tasks. The top performers are copied and “mutated” by randomly editing their code to introduce variations. Older, less efficient algorithms are deleted, while the mutated new candidates are added in every cycle of iteration. Given Google’s processing power, thousands of these algos are randomly created, tested and mutated every second. AutoML Zero also uses the evolutionary trick of mixing populations to create more robust algos. The paper shows how this approach can actually lead to the discovery of well-known ML methods, including the program learning how to create neural networks.

This takes the concept of self-learning algorithms one significant level deeper. If the program can discover rules we know by using this random evolutionary approach, it may also find new things that we have not yet discovered. It could also reduce human biases, which can lead to racism and gender disparities being coded into AI.

According to the researchers, AutoML Zero already outperforms the precursor and other similar ML tools. They plan to use these methods to develop algos with more specific focus. They will also experiment with using combinations of the self-learning, evolutionary approach with manually created algorithms as the seed population.

Uber has used similar playful methods in its AI lab. Kenneth Stanley at Uber has tried to program curiosity, which is a key driver of biological intelligence. Most ML programs focus on solving specific problems and “reward” the algos that do it efficiently. Stanley did not set a reward structure; his algos are allowed to simply find new things to do. He discovered his curious AI robots could efficiently negotiate mazes — dead-ends left them “bored”.

Uber’s program Ludwig was used for predicting food delivery logistics before it was released publicly. Ludwig trains itself when fed data. It learns to recognise associations, and processes new data to identify images, answer questions, and make numerical estimates, etc. After public release, Ludiwg has been used to analyse images from telescopes by astrophysicists and images from microscopes by bioscientists, as well as performing predictive logistical tasks.

Biological intelligence arose as a function of evolution, through a sequence of random mutations. Mutations that created smarter beings were propagated better because those smarter beings could navigate and manipulate their environments better. This took millions of years. Given the speed at which computers can cycle through generations of mutated programs, evolution may be a promising new approach for self-learning.