Brains, unlike artificial neural nets, use sym- bols to summarise and reason about perceptual input. But unlike symbolic AI, they “ground” the symbols in the data: the symbols have meaning in terms of data, not just meaning imposed by the outside user. If neural nets could be made to grow their own symbols in the way that brains do, there would be a good prospect of combining neural networks and symbolic AI, in such a way as to combine the good features of each.

It is argued that the secret of growing symbols in neural nets lies in cluster analysis. Algorithms for clustering, many of them naturally implementable in neural hardware, would produce clusters, which are discrete entities summarising data that have all the properties of symbols.

The war between symbolic artificial intelligence and its neural net rival continues because each has strengths that the other lacks, and it has proved impossible to combine them successfully. It is agreed that symbolic systems work well on discretely structured problems, like chess, and give a transparent understanding of what they are doing, which allows their use in new situations through adding and deleting rules. But it is difficult to make them adaptive to data, especially in situations where there is only data to go on, and almost no understanding via rules, such as face recognition. Scaling up from toy to real problems is also hard. Neural nets, on the other hand, are strong where symbolic AI is weak, and vice versa. They adapt easily to data, but the black-box nature of their processing makes it very difficult to understand what they do, and hence to improve it, or adapt it to a different problem.

Download pdf How a neural net grows symbols