InterJournal Complex Systems, 1937
Status: Accepted
Manuscript Number: [1937]
Submission Date: 2006
Emergence, Intrinsic Structure of Information, and Agenthood
Author(s): Daniel Polani

Subject(s): CX.1



The centrality of the concept of emergence for the understanding of complex systems has been the topic of a numerous number of publications, covering a spectrum from category-theoretic to synergetic approaches (Rasmussen et al. 2001; Haken 2000). A particular interest has been devoted to finding information-theoretic formalizations of the notion of emergence, since the universality of information theory lends itself to a wide area of applicability. Ideally, the notion of emergence would not have to be considered ``in the eye of the beholder" (Harvey 2000), but would arise naturally from the dynamics of the system. Such a notion of emergence in time series has been developed in (Crutchfield 1994); Shalizi 2001), based on the epsilon-machine concept: a process emerges from another one if it has a greater predictive efficiency than the second. Stated informally, this means that, the ratio between prediction information (excess entropy) and the complexity of the predicting epsilon-machine is better in the emerging process. This fits smoothly into the perspective that emergence should represent a higher-level coarse-grained view or simplification of a more intricate fine-grained system dynamics. A related, but different view is taken in the emergent description model from (Polani 2004). Here, as above, the predictivity of a time series is measured by mutual information, but we considered a memoryless system, unlike above work using the epsilon machine modeling maximal causal histories. The main difference, however, is that we considered a decomposition of the total system into individual independent informational submodes, thus providing a more intricate picture about intrinsic structure the informational dynamics of the total system. In (McGregor and Fernando 2005), higher-level prediction models for partial aspects of a systems are suggested, based on entropy measures, which can be interpreted as a simplified version of the model from (Shalizi 2001) as it does not consider causal states, or of our model as it does not require a full partition into independent modes. On the other hand, a decomposition philosophy for dynamical hierarchies into modes similar to ours, but based on smooth mappings instead of information theory is taken in (Jacobi 2005). Armed with above insights, here we wish to suggest some insightful extensions of the picture and interpretation of emergent descriptions. 1. the emergent descriptions model can be sought by e.g. Multiobjective Evolutionary Search algorithms that can optimize for the several criteria of emergent descriptions (completeness, predictivity and independence). In general, the criteria cannot be simultaneously fulfilled, so a solution realizes a tradeoff which can be picked at the Pareto front of the search. 2. Instead of considering a memoryless process, the individual ``coordinates" or modes can be equipped by an epsilon-machine like extension, as to accomodate possible memory effects. 3. Instead of point 2, on the other hand, it is possible to search for suitable inputs from other modes that would help in the prediction of the next state. That induces a natural hierarchy in the different modes, not unlike the algebraic model of semigroup decomposition into ``coordinates" suggested in (Nehaniv 1997) where subordinate modes feed into higher-level modes. 4. In (Klyubin et al. 2004) it has been shown that the perception-action loop of an agent acting in an environment can be modeled in the language of information. This is particularly interesting for above considerations, as the agent/environment system is a generalization of a time series (a time series can be considered an agent without without the ability to select an action, i.e. without the capacity for ``free will"). Using infomax principles, above agent/environment system can be shown to structure the information flows into partly decomposable information flows, a process that can be interpreted as a form of concept formation. This gives a new interpretation for the importance of emergence as the archetypical mechanism that allows the formation of concept in intelligent agents and is thus perhaps a key driving the creation of complexity in living systems. Bibliography Crutchfield, J. P. (1994). The calculi of emergence: Computation, dynamics, and induction. Physica D, pages 11-54. Haken, H. (2000). Information and Self-Organization. Springer Series in Synergetics. Springer. Harvey, I. (2000). The 3 es of artificial life: Emergence, embodiment and evolution. Invited talk at Artificial Life VII, 1.-6. August, Portland. Jacobi, M. N. (2005). Hierarchical organization in smooth dynamical systems. Artificial Life, 11(4):493-512. Klyubin, A. S., Polani, D., and Nehaniv, C. L. (2004). Organization of the information flow in the perception-action loop of evolved agents. In Proceedings of 2004 NASA/DoD Conference on Evolvable Hardware, pages 177-180. IEEE Computer Society. McGregor, S. and Fernando, C. (2005). Levels of description: A novel approach to dynamical hierarchies. Artificial Life, 11(4):459-472. Nehaniv, C. L. (1997). Algebraic models for understanding: Coordinate systems and cognitive empowerment. In J. P. Marsh, C. L. Nehaniv, B. G., editor, Proceedings of the Second International Conference on Cognitive Technology: Humanizing the Information Age, pages 147-162. IEEE Computer Society Press. Polani, D. (2004). Defining emergent descriptions by information preservation. In Proc. of the International Conference on Complex Systems. NECSI. Long abstract, full paper under review in InterJournal. Rasmussen, S., Baas, N., Mayer, B., Nilsson, M., and Olesen, M. W. (2001). Ansatz for dynamical hierarchies. Artificial Life, 7:329-353. Shalizi, C. R. (2001). Causal Architecture, Complexity and Self-Organization in Time Series and Cellular Automata. PhD thesis, University of Wisconsin-Madison.

Retrieve Manuscript
Submit referee report/comment

Public Comments: