Emergence is a very interesting concept in the computing world: the idea that complex phenomenon can arise from relatively simple interactions. There are two types of emergence, somewhat controversially: weak emergence and strong emergence. The wikipedia article provides an introduction to the mystery, but the essential difference is that strong emergence suggests that the resultant is more than the sum of its parts; with strong emergence, you end up with something that cannot be explained by dissembling into constituents.
Imagine a simple rule for a termite: “walk around until you see a wood chip, then pick it up — unless you’re already carrying a wood chip, in which case, drop it.” Simple rules like these have been shows to lead to incredibly large and complex termite mounds , one of the oft quoted examples of emergence.
In the world of algorithms, an emergent system is one that exhibits complex results based on such simple rules. A combination of NAND gates can be used to simulate essentially any sophisticated logic function. An artificial neural network (ANN) similarly leads to complex results, and the inherent fuzzy logic allows the system to correctly solve problems it has never encountered before.
Evolutionary algorithms (EAs) utilize simple rules repeatedly until parameters converge on the desired solution. It should be of no surprise to anyone that understands Darwin’s ideas that this can produce almost limitless complexity. The rich history of EAs, leading to what is now known as Evolutionary Computation will be the subject of a future post.
Cellular Automaton models allow complexity to arrive through both time (generations of self-replication) and the connectionist approach.
These concepts are best illustrated by example, which I will dive into in subsequent posts. I hope to explore these different computational themes and show how they are connected by recurring philosophical ideas such as ‘Holism vs. Reductionism’, and along the way we will remind ourselves to question whether our examples exhibit strong emergence or merely weak emergence.