FP7
Roadmap
for the area of
"Nature-Inspired
Smart Information Systems
(NiSIS)"

(Version 2.1 of November 2006, DRAFT)

Executive Summary

(This chapter is yet to be provided)

Notes

Last version saved by:
Aleksandar Jovanovic
Contributions:

Derek Liekens

Version: 2.1

Date: November 2006, DRAFT

1. Introduction

The topics and focus of the European Project “Nature-inspired Smart Information Systems” (NiSIS), as specified in FP6-2002 IST-1, Co-ordination Action NiSIS, Proposal/Contract no.: 13569, have the following strategic objectives

• Co-ordination of multi-disciplinary studies and research endeavours into the development and utilization of intelligent paradigms in advanced information system design
• Extension of investigation into emerging new areas inspired by nature both at biological (i.e. micro) and behavioural (i.e. macro) levels for visionary concepts of information processing and architectures

The NiSIS programme comprises an extensive scheme of activities both in research and organization. This scheme consists of four parts:

• A set of three Focus Groups to deliver scientific and technological contributions to the programme:

1. Nature-inspired Data Technology (NiDT)
2. Nature-inspired Networks (NiN)
3. Nature-inspired Modelling, Optimization and Control (NiMOC)

• A Technology Transfer, Training and Education (TTE) Committee ;Public Relations
• A Roadmap designed by an Integrated Technology Board (ITB)
• Project Management and Co-ordination by the European Laboratory for Intelligent Techniques Engineering (ELITE) Foundation Service Center, Aachen providing support and infrastructure

Inspiration from nature has long been the source of many successful, ingenious and innovative ideas in many fields of science and technology, despite the fact that the final outcome of the creative process, which transfers these ideas to actual scientific and technological progress, has often discarded any natural plausibility to favor the scientific and technological advance. A simple example is the technological field of artificial neural networks, which has flourished in the last decades and, despite the inspiration derived from biological neurons, has reached a high level of maturity and a vast amount of successful application in the ICT field, sacrificing any close similarity to biological neural networks.

The above point of view is the main basis for this Roadmap. Obviously, this is not the only possible vision: an alternative approach is the understanding of nature for directly interfering with its processes and forcing it to perform desired tasks. This duality is almost always present in the fertilization between nature and ICT: a simple example is DNA computing, where biological DNA strands are used to efficiently solve difficult optimization problems, by appropriately coding the problem on the biological material and exploiting the massively parallel “computation” performed naturally. At the same time, from a dual perspective, this kind of Natural Computation can suggest new algorithms in the field of, for example, Genetic Programming, which is a branch of Computer Science and has become a widely used technology with a large number of successful applications.

The “inspiration from nature” described above can be roughly represented as in Fig. 1. The disciplines studying the natural world, its structure, rules and phenomena are the general basis for inspiration. Two main aspects of the study of nature have been identified: a modeling aspect, which deals mainly with the structural properties of nature by addressing local or global properties, discrete or continuous models, multi-layer or hierarchical structures, etc.; the second aspect deals with strategies adopted by nature like self-assembling, learning and adaptation, etc. The research in these two areas is the starting point for the design and creation of artifacts in the ICT world. Three kinds of artifacts can be identified: virtual artifacts, artificial (or synthetic) artifacts and hybrid artifacts. The first class comprehends all the nature-inspired algorithmic field, where strategies and models inferred by nature are translated into algorithms, software, mathematical descriptions, etc.; the second kind of artifacts realize the next generation of ICT devices and systems, which inherit the desired characteristics of the natural world, like self-repairing, autonomous behavior, self-assembling, etc.; the hybrid artifacts are the connection between the natural and synthetic world including artificial organs, sensing aids, etc.

The thin-line arrows represent the knowledge flow from the ICT world to the natural world. This is a research field of paramount importance because it comprehends all the ICT activities that can help in understanding the natural world (e.g. bioinformatics, chemical informatics, etc.) but are outside the scope of our target.

Fig. 1 - Nature-inspired ICT (see also http://www.beyond-the-horizon.net

Fig. 1 – Nature-inspired ICT

2. State-of-the-art

Contributions regarding the present situation of nature-inspired ICT have been gathered from the 3 Focus Groups and are utilized in the following sections under the themes of data technologies, networks and modelling systems.

2.1 Data Technology (NiDT)

The term “Data Technologies” (DT) must be interpreted in a broad sense, including not only the field usually referred to as “Data Base Technology” but all the aspects of data or, better, information processing, storing and transfer.

Fig. 2 – Artifact information flow

Fig. 2 sketches the DT related issues of a generic artifact, as described in the Introduction, through a coarse-grain representation of the internal information flow (note the similarity to the sensing-processing-actuating framework of living and life-like systems). This scheme is quite general and can be applied to virtual, synthetic and hybrid artifacts as well. For example, in the case of virtual artifacts, e.g. a software module, the environment is simply the computing systems on which the module has been implemented and is executed.

Further descriptions follow on different aspects of the requirements for acquisition, storage and utilisation of massive amounts of data.

2.1.1 Data acquisition

The “data acquisition” refers to the input of data from the environment surrounding the artifact. In the case of “real-world” environments the acquisition process is performed by “sensing” and appropriately translating actual physical measures to information that can be processed by a processing device or system. Inspiration from nature is derived in many ways, for example by building auditory and vision models, attention and perception models [Car04] and studying neuromorphic architectures [Ind01] that can be translated in methodologies and devices for acquiring signals and images in a way similar to the human or, more generally, biological perception system [Car04]. The paradigm of Amorphous Computing (AC) is also of some interest in the field of data acquisition: AC is the development of organizational principles and programming languages for obtaining coherent behavior from the cooperation of myriads of unreliable parts that are interconnected in unknown, irregular, and time-varying ways [Abe01]. A possibility is to produce particles that could be mixed with bulk materials, such as paints, gels, and concrete to obtain, for example, a smart paint which applied to bridges or buildings could sense and report on traffic and wind loads and monitor structural integrity.

The complexity of the sensing task can be overwhelming, so the “natural” self-assembly and self-evolution of devices for sensing is attracting the attention of many researchers [Dau01].

In the field of “virtual” environments, the sensing process is not related to the actual measure of physical variables but mainly to the intelligent acquisition of data in a computing system. NiSIS-NiDT has launched a Task Force in this field, applied to industrial processing problems and focusing on “Nature Inspired Self healing Soft Sensors for Process Industry”.

[Dau01] K.Dautenhahn et al., Special Issue on Sensor Evolution, Artificial Life, Vol. 7, N. 2, 2001.

[Abe01] Abelson et al. Amorphous Computing, Communications of the ACM, Vol. 43, N. 5, May 2001.

[Car04] L.Carota, G.Indiveri, V.Dante. A software–hardware selective attention system, Neurocomputing, N. 58–60, pp. 647 – 653, 2004.

[Ind01] G.Indiveri, A Neuromorphic VLSI device for implementing 2D selective attention systems, IEEE Transactions on Neural Networks, Vol.12, N.6, pp.1455-1463, 2001.

2.1.2 Data processing, Data storage, Data mining

The problems of data (pre)processing, storage and mining are probably the core target of data technologies. Nature has long been a source of inspiration: strategies like Learning and Adaptation, which are concerned with entities improving their performance by experience have their roots in the early 1950s with attempts to mathematically model biological neural networks in order to ultimately simulate the operation of the brain. Beginning with the 1970s, various views of learning and adaptation came in the focus of adaptive control [Goo84] and pattern recognition [Dud73] research. For the past several decades, the machine learning [Mit97] research field has become established in its own right, encompassing the entire range of artificial learning systems investigation, focusing mainly on devising computer algorithms able to learn. At the same time, research on Cognitive Models has become a powerful tool to manipulate very large and complex data information processes [Cli03]. These models have been created thanks to the development of artificial systems that were inspired by the study of natural-occurring behaviour found in areas like: developmental biology, the human immune system, gene-pools and the interaction of groups of autonomous agents.

Evolvable systems play a particularly important role in the fields of data processing, interpretation, and knowledge extraction. They tend to use a population based optimisation algorithm that uses mechanisms inspired by biological evolution such as selection, mutation and crossover. Candidate solutions play the role of individuals in a population and evolution of the population takes place by repeating the application of the above mechanisms until a satisfactory solution has evolved. Specific examples of algorithms used to evolve solutions/systems are: Genetic Algorithms [Mit96], Evolution strategies [Sch05], Learning classifier systems [Gaf95], Genetic programming [Neg05], and Evolutionary programming [Del05].
Artificial Immune Systems are inspired by the biological immune system, and use learning, memory, and associative retrieval to solve recognition and classification tasks. In particular, it learns to recognize relevant patterns, remember patterns that have been seen previously, and use combinatorics to construct pattern detectors efficiently. These remarkable information-processing abilities of the immune system provide important aspects in the field of computation. This emerging field is sometime referred as Immunological Computation, Immuno-computing or Artificial Immune Systems (AIS) [Tim04].

The nature-inspired approach to data processing is also developing through the use of unconventional computing frameworks, directly inspired from Natural Computation: examples are Membrane Computing (MC) [Pau00], DNA Computing (DC) [Adl94], and Quorum Computing (QC) [Tho79]. MC proposes to abstract from the architecture of the cell and the way biological substances are both modified and moved among internal compartments, and to interpret the phenomena occurring inside the cell as computing processes [Cio05]. DC exploits molecular biology laboratory techniques to manipulates DNA strings so that the strings encode information related to the problem to solve, by using the DNA structure. The major idea is to take advantage of the huge parallelism provided by the biochemical processes occurring in a DNA solution, which also turn out to be favourable, from the point of view of the required energy and the size needed to store information. This approach is promising in solving, for example, difficult combinatorial optimization problems [Yan05]. Quorum systems are tools for building highly available replicated data services. A quorum system can be defined as a collection of sets (called quorums) with certain intersection properties. These properties allow read and write operations to be performed only at a quorum of servers, since they ensure that any read operation will have access to the most recent value that was written on a shared element (variable, document, etc.) [Nao03].
Several other models and strategies of data coding, storage and processing can be inspired by the way nature deals with modularization, decomposition and hierarchies of data and its temporal and spatial structure, but the research in these fields is far from being well established.

New approaches to bio-inspired data processing and mining must also be derived for the new generation of computing devices that are the result of the advances in (nano)electronics. While nature-inspired devices implemented in microelectronics could take advantage of current generation analog (continuous state) electronics to perform biological-like computations [Mea89], the emerging nanoelectronic devices appear to be inherently digital (discrete state) and error-prone [Lik05] so new data learning and processing paradigms, based on discrete (combinatorial) methodologies, must be found. This approach is targeted by one of the NiDT Task Forces: “NiCOLE: Nature-inspired COmbinatorial machine LEarning”.

[Adl94] L.Adleman, Molecular computation of solution to combinatorial problems. Science, 266, pp. 1021-1024, 1994.

[Cio05] G.Ciobanu, M.Perez-Jimenez, G.Paun, Applications of Membrane Computing. Springer-Verlag, 2005.

[Cli03] D.Cliff, Biologically-Inspired Computing Approaches To Cognitive Systems: a partial tour of the literature, Digital Media System Laboratory. HP Laboratories Bristol. HPL-2003-11 April 14, 2003

[Del05] K.Delisle, Decision Trees and Evolutionary Programming. Available at: http://ai-depot.com/Tutorial/DecisionTrees.html

[Dud73] R.O.Duda, P.E.Hart, and D. G. Stork. Pattern Classification and Scene Analysis. John Wiley, 1973.

[Gaf95] D.Gaff, Parameter Optimisation for the Learning Classifier System Available at: http://www.sv.vt.edu/classes/ESM4714/Student_Proj/class95/gaff/gaff.html

[Goo84] G.C.Goodwin and K.S.Sin. Adaptive filtering, prediction and control. Prentice-Hall, 1984.

[Lik05] K.Likharev et al., Architectures for Nanoelectronic Implementation of Artificial Neural Networks: New Results. Neurocomputing, Vol. 64, No. 1, pp. 271-283 (2005).

[Mea89] C.Mead, Analog VLSI and neural systems, Addison Wesley, 1989.

[Mit96] M.Mitchell, An Introduction to Genetic Algorithms. MIT Press, 1996.

[Mit97] T.Mitchell. Machine Learning. McGraw Hill, 1997.

[Nao03] M.Naor and U.Wieder. Scalable and dynamic quorum systems. In ACM Conf. on Principles of Distributed Computing (PODC), 2003.

[Neg05] M.Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems. 2nd ed., Addison-Wesley, 2005.

[Pau00] G.Paun, Computing with Membranes. Journal of Computer and System Sciences, 61, 1, pp. 108-143, 2000.

[Sch05] S.Schulze-Kremer, Genetic Algorithms and Protein Folding, Available at: http://www.techfak.uni-bielefeld.de/bcd/Curric/ProtEn/112.html

[Tho79] R.H.Thomas. A majority consensus approach to concurrency control for multiple copy databases. ACM Transactions on Database Systems, Vol. 4, N. 2 , pp.180-209, 1979.

[Tim04] J Timmis, T Knight, L N De Castro, and E Hart, An overview of artificial immune systems.. In R Paton, H Bolouri, M Holcombe, J H Parish, and R Tateson, Eds., "Computation in Cells and Tissues: Perspectives and Tools for Thought", Natural Computation Series, pp. 51-86. Springer, 2004.

[Yan05] C.N.Yang, C.B.Yang, A DNA solution of SAT problem by a modified sticker model. Biosystems, vol. 81, pp. 1-9, 2005

2.1.3 Data transfer toward Humans - Biological/Natural environment

This area deals with the transfer of data and information from the artifacts to the external world, which can be accomplished through nature-inspired visualization concepts [Shen] and evolvable interfaces.
More advanced approaches involve a tighter connection between artifacts and the biological world like, for example, artificial prostheses, artificial retinas, etc. which are at the interface between the ICT and medical/biological research field.

[Shen] W.Shen, A.D’Angelo, A.Pang, Nature Inspired Flow Visualization”, to appear in Computer Graphics.

2.1.4 Data transfer for interaction and cooperation

The interaction and cooperation among the artifacts is the basis for the emergence of intelligent and complex behavior. This research field is addressed by several nature-inspired approaches, which can be considered as the evolution of the traditional agent-based approach. Examples are Swarm Intelligence and Ants Colony [Dor99] research topics, which focus on the collective behavior of unsophisticated agents interacting locally with their environment and causing coherent functional global patterns to emerge. Characteristics of a swarm are distributed, no central control or data source, no (explicit) model of the environment, perception of the environment, limited time to act and strong emphasis on reaction and adaptation. Data Mining with Swarm/Ants Colony Intelligence is a new and active field of research: from data clustering [Par02] to web [Abr03], data mining [Abr06] and knowledge extraction [Gal02].

In nature, emergent behavior is observed, for example, in bacteria, ants and bees, but social networks are becoming of great interest too. The social network is a map of the individuals, and the ways how they are related to each other: a single person is the node of the network while edges which link nodes, and are called also "connections", "links" or "ties", correspond to relationships between people. Research in a number of scientific fields has demonstrated that social networks operate on many levels, from families up to the level of nations, and play a critical role in determining the way problems are solved, organizations are run, and the degree to which individuals succeed in achieving their goals. The study of social networks can help to solve problems in the ICT field: for example, Kumar et al. observed great correspondence between the nature of social networks and the structure and content of the Web that can result in more precise Web search mechanisms and better understanding of the sociological aspects of the Web content management [Kum02].

[Abr03] A.Abraham, V.Ramos, Web usage mining using artificial ant colony clustering and Genetic Programming, CEC 03 – Congress on Evolutionary Computation, Canberra, Australia, 8-12 Dec. 2003, pp. 1384-1391.

[Abr06] A.Abraham, C.Grosan, V.Ramos, Swarm Intelligence and Data Mining, Springer, 2006.

[Dor99] M.Dorigo, G.Di Caro, L.M.Gambardella, Ant Algorithms for Discrete Optimization. Artificial Life, Vol.5, No.3, pp. 137-172, 1999.

[Gal02] M.Galea, Applying Swarm Intelligence to Rule Induction, MSc thesis, University of Edinburgh, 2002.

[Kum02] R.Kumar, P.Raghavan, S.Rajagopalan, A.Tomkins, The Web and Social Networks. IEEE Computer, Vol. 35, N. 11, pp.32-36, 2002.

2.2 Networks (NiN)

According to WordNet, a network is “an interconnected system of things or people”. For the purpose of this document we define a network as anything that can be modelled by a graph, where the modelling may be qualitative or quantitative. We do not assume that the model should be predictive or explanatory, merely that the underlying structure of the system should be naturally expressible as a collection of nodes and edges. This can encompass a range of systems such as a market (where nodes might represent producers and consumers, and edges might represent contractual relationships between producers and consumers); a group of computers connected by fixed cables; a set of documents and cross references between them; etc.

Examples of networks can be found in many different areas. Social networks were among the first scale-free networks to be investigated. Citation networks [1], actor networks [2] and scientific collaboration networks [3-5] belong into this category. Two of the technological networks examined are the World Wide Web [6, 7] and the Internet [8, 9]. Biological networks have aroused a lot of interest lately, they include metabolic networks [10-12], protein-protein interaction networks [13, 14], protein domain networks [15] and gene expression networks [16, 17]. Other networks where one would not readily suspect a (scale-free) network structure are the web of words [18], the network of sun corona explosions [19], the network of earthquakes [20] and the medieval inquisition [21]. This list is only a small excert, but it hopefully shows that scale-free networks appear in manifold domains.

Discovering a scale-free network in real-world data is not the exception anymore. On the contrary, it seems that a scale-free network structure can be extracted from nearly every data set available as even comic book characters are not safe from being investigated [22]. Having said that, it has to be pointed out that there are different opinions within the community as to when a data set exhibits scale-free behaviour. A very simple method is to plot the degree distribution on a log-log plot and if the distribution follows a straight line a power law distribution is present. A given empirically-estimated degree distribution might be classified by one research group as a power law whilst another classifies it as a stretched exponential degree distribution. In several cases an examination of a single data set by different research groups yielded widely different estimations of the degree exponent. A major problem is the lack of standardized methods for the detection of a scale-free behaviour and the subsequent calculation of the degree exponent ! . In [23, 24] for example it is claimed that the biological networks whose degree distributions have been investigated so far often do not exhibit pure power-laws, although some are found to follow a power-law with an exponential cut-off. The authors attribute the different findings to the simple mathematical methods used until now to calculate the degree exponent such as linear fit of the data on a log-log plot. They argue that the more accurate maximum likelihood method produces different results. A similar argument is presented in [25]. In the next section, the metabolic network is presented in more detail as a network that was found to be scale-free.

2.2.1 Metabolic Networks

A metabolic network describes the chemical changes within an organism or cell that occur in order to produce energy and materials needed to sustain the organism's or cell's life processes. In general, such a network is developed as follows: the substances on which enzymes act (the so-called substrates) form the nodes of the network and a connection is established between two nodes if a metabolic reaction occurs with both of the node's corresponding substrates taking part. Fell et al. [10] did not distinguish between reaction educts and reaction products and therefore applied undirected links. They analyzed the structure of the core metabolic network (275 substrates) of the bacterium Escherichia coli and reported that the probability of a substrate to take part in k reactions follows a power-law. While they discarded common i.e. co-enzymes such as ATP due to their ubiquitous nature, Jeong et al [11] included them in their investigation into the metabolic networks of 43 different organisms from all three domains of life: archaea, bacteria and eukaryotes. They constructed a directed bipartite graph for each organism from data provided by the WIT database, a pathway-genome database which allows the prediction of a given metabolic pathway based on the annotated genome and biochemical knowledge.

The striking priority of network structures in living systems, gene regulatory networks, metabolic networks, and signal transduction networks in cells, networks of neurons in the brain, the network of immune system components, the network of blood circulation, the network of respiration paths, in contrast to the unbranched digestive tract must have their reason for their existence. It is evolution that is the key to understand the existence of systems, processes, phenomena in living systems. Networks, in contrast to hierarchies, seem to have advantages in data, signal or information processing and decision-making since there is enough flexibility to find some way out if the path used previously gets blocked. As a consequence, networks seem to be more fault-tolerant than hierarchies. However, there is also a danger in high flexibility concerning misguidance or dysfunction. There has to be a delicate balance between regulatory capacity and misregulation. How to model such a delicate balance and how to derive computational algorithms of data or information processing from network behaviour are important questions. Can neuronal networks mirror the behaviour of cellular networks, immune networks, blood networks, respiratory networks, too ? What about models of signal transduction and information transfer by biochemical receptor binding ? Networks, at least complex ones, seem to be capable of emergent phenomena due to multiple interactions among the network’s constituent parts. Emergent phenomena are never seen at single elements of the network but are only associated with the network as a whole.

2.2.2 Financial Networks

Managerial and economic systems can be considered as networks of information processing entities that pursue their own goals and interact with one another in a multitude of flexible ways. Economic theory has developed a number of methods for modelling such networks and analysing the desirable properties of their operation. Theories regarding the workings of markets, microeconomic systems, influence of adverse information effects and the influence of transaction cost effects can be given as examples. These results could be used to allocate resources efficiently and effectively in complex information systems, to design frameworks in which such information systems can operate and to guide the emergent properties of complex information systems. Already artificial markets are being designed, investigated for optimisation in decentralised network systems (e.g. optimal scheduling in IT systems for supply chain management) and information sharing (artificial information markets for autonomous agents). Furthermore, quantitative management approaches offer a variety of optimization problems, suitable to be handled by nature-inspired smart information approaches. A number of these problems can usually be represented in the form of a network (the so-called orienteering problem being a representative one).

11. Jeong, H., et al., The large-scale organization of metabolic networks. Nature, 2000(6804): p. 651-653.

12. Ma, H.W. and A.P. Zeng, The connectivity structure, giant strong component and centrality of metabolic networks. Bioinformatics, 2003. 19(11): p. 1423-1430.

13. Jeong, H., et al., Lethality and centrality in protein networks. Nature, 2001(6833): p. 41.

14. Wagner, A., The Yeast Protein Interaction Network Evolves Rapidly and Contains Few Redundant Duplicate Genes. Molecular Biology and Evolution, 2001. 18(7): p. 1283-1292.

15. Wuchty, S., Scale-Free Behavior in Protein Domain Networks. Molecular Biology and Evolution, 2001. 18(9): p. 1694-1702.

16. Agrawal, H., Extreme Self-Organization in Networks Constructed from Gene Expression Data. Physical Review Letters, 2002. 89(26): p. 268702.

17. Bergmann, S., J. Ihmels, and N. Barkai, Similarities and Differences in Genome-Wide Expression Data of Six Organisms. PLoS biology, 2004. 2(1): p. 85-93.

18. Cancho, R.F.i. and R.V. Sole, The small world of human language. Proceedings- Royal Society of London B, 2001(1482): p. 2261-2266.

19. Paczuski, M. and D. Hughes, A heavenly example of scale-free networks and self-organized criticality. Physica A, 2004. 342(1-2): p. 158-163.

20. Abe, S. and N. Suzuki, Scale-free network of earthquakes. Europhys. Lett., 2004. 65(4): p. 581-586.

21. Ormerod, P. and A. Roach, The Medieval inquisition: scale-free networks and the surpression of heresy. eprint arXiv:condmat/ 0306031, 2004. 339: p. 645-652.

22. Alberich, R., J. Miro-Julia, and F. Rosselló, Marvel Universe looks almost like a real social network,. preprint cond-mat/0202174, 2002.

23. Khanin, R. and E. Wit, How Scale-Free are Biological Networks. preprint, 2005.

24. Stumpf, M.P.H., et al., Statistical model selection methods applied to biological networks. Transactions in Computational Systems Biolog, 2005.

25. Goldstein, M.L., S.A. Morris, and G.G. Yen, Problems with fitting to the power-law distribution. European Physical Journal B Condensed Matter Physics, 2004. 41: p. 255-258.

2.3 Modelling and Systems

Model simulation of complex systems is the only means to detect them in particular regions of the multi-dimensional space of model parameters. An example is learning capacity and consciousness of the brain which must have evolved as an emergent phenomenon in neuronal networks of the brain. Existing learning algorithms and network algorithms are obviously Nature-inspired algorithms of information processing. However, the question is open on whether emergent phenomena of thought processing in brain can be modelled or can give rise to development of new numerical or logical algorithms as Nature’s information processing principles. The immune network is obviously different in type. Like neuronal networks of the brain it is cell-based in contrast to cellular networks of gene regulation, signal transduction and metabolism that are molecule-based. Moreover, the immune network is mobile and embedded in fluid medium. Nevertheless, it may be considered a network since, as revealed by immunoassay experiments, there exist interactions among its elements that give rise to possibly emergent phenomena, such as recognition of and protection against pathogens.

Models of the immune system are based on heterogeneous Cellular Automata, rather than on network models, and are called virtual immune systems. Artificial Immune Systems is the name for immune system-inspired information processing algorithms. As well known, these algorithms have already been used in protection of computers against viruses and worms.

Inspiration from Nature, with respect to modeling, optimization and control, in order to design smart information systems begins with the question: What renders living systems viable? and ends with the question: What renders living systems smart?

Well known characteristic properties of living systems are:

• Metabolism (Flow, Exchange and Conversion) of matter and energy
• Propagation, Self-Reproduction
• Mutability
• Evolvability

With some caution, this can be seen to be the definition of viability. To understand how a living system can be smart in the sense of clever or intelligent, this definition is insufficient and needs modification and complementation. One possibility leads to this:

• Metabolism of matter and energy and information
• Propagation, Self-Reproduction, Self-Information, Heredity
• Communication inside and outside the living system with its environment
• Mutability in structure and information processing
• Evolvability in function and behaviour

With this definition in mind, one can imagine how a living system can be smart.

• A living system is Smart if it manages to survive by appropriately changing its behaviour to adapt to changing environmental conditions.
• A living system is Smart if it manages to survive, in spite of competition pressure, by adaptation to competitive stress conditions.
• A living system is Smart if it manages to keep alive by maintaining its function.
• A living system is Smart if it manages to achieve adapted function and behaviour.

There are further versions of cleverness or intelligence of living systems,but already among those mentioned above there is a striking difference between the first two and the second two points. In the first two points, mutation and selection of adapted species is a matter of population rather than of individuals and concerns the offspring of living systems resulting from mutative propagation, rather than the parent individuals. So, subsequent generations learn from their parents how to lead a better life. In contrast, in the second two points the individual living system itself is concerned. As long as self-reproductive artificial living systems remain outside the scope of reasoning about Nature-inspired smart information systems and Nature-inspired modelling, optimization and control, the second two points are the only ones of importance and in particular the last one.

How can an artificial Nature-inspired smart information system achieve adapted function and behaviour, i.e. function and behaviour predefined or adapted to human desire ? The answer is: by modelling, optimization and control, more precisely by evolutionary modelling, optimization and control. What does this mean ? Modelling, simulation, optimization and control mark the roadmap strategy for inspiration from Nature in order to reveal its information processing principles that will open new perspectives of information and communication technologies. Modelling, simulation, optimization and control are steps in the design of Nature-inspired smart information systems that possess pre-defined properties. In view of lacking knowledge about information processing as well as about structure-function relation or genotype-phenotype relation of the living system such a design cannot be done at once. Rather, it will require many repeated model modifications, subsequent simulation runs to check the model behaviour, and optimisation to adjust model parameters or even model modules in order to correct the behaviour, and control to guarantee the stability of the behaviour. This design has very much the character of a trial and error procedure whose convergence is not certain but can be achieved by operating carefully and as systematically as possible by drawing the right conclusions from both successful and failed attempts.

The repeated attempts of model modification generate a whole quasi-population of model mutants evolving under the selection pressure to achieve the pre-defined function of the Nature-inspired Smart Information System. During the design process those mutants that show desired properties or come close will propagate fastest and therefore survive whereas failing model mutants will die out. So, the difference between the versions of cleverness or intelligence of living systems mentioned above, is not as striking as originally suspected. Although self-reproductive artificial living systems are not taken into account, there is nevertheless a situation of man-made reproduction, similar to evolution of a particular species of Nature-inspired smart information systems in a population of competing species. Evolutionary design of biological macromolecules, as opposed to rational design, has been known for a decade or so. While rational design and preparation of simple biochemical molecules of given properties can be done according to Quantitative Structure Activity Relations, this approach is inappropriate for chain-like biological macromolecules like nucleic acids due to the huge amount of sequence variants at even moderate sequence length. Even less appropriate is rational design for more complex living systems. Nucleic acids are,in terms of reproducible information carriers,the simplest living systems. The most prominent example is evolutionary design of particular species of RNA. RNA is simultaneously a carrier of genotype by its sequence and a carrier of phenotype by its secondary structure. RNA can propagate by mutative replication in an RNA population when fed by a nucleotide resource for which several RNA mutants will compete. Those species that will survive replicate fastest since they have the fastest access to limited resources of food and space. The velocity of propagation of any species essentially depends on its capability to replicate and,therefore,on its secondary structure. While mutation affects the genotype, selection is exerted on the phenotype. In a different variant of evolutionary design of RNA species, the RNA propagation velocity of the winning species is fastest due to retention of its particular secondary structure by specific binding to the inner wall of the device, while competing species are washed out again and again by some dilution flow in numerous cycles of the RNA population through the device.

Inspiration from Nature means to understand the analogy or metaphor between evolutionary preparation and evolutionary design. The device for evolutionary preparation is replaced with the designer’s laboratory. The mutations are replaced with the model variants (genotype, structure), while selection of species function is replaced with selection of simulation behaviour (phenotype, function). Food resources are replaced with designer’s ideas. Selection pressure is replaced with limited computer time and limited storage space or any other resource limitation for the design process. Selection goal of species function is replaced with selection goal of simulation behaviour, and a true experimental procedure is replaced with computer experiments. Selection pressure may also originate from qualitative targets of the design process. Finally, evolutionary preparation is replaced with evolutionary design through Modelling, Simulation, Optimization and Control.

Teaching & Education

Technology Transfer

Standardization

Dissemination & Publication

Competitions & Patents

Case studies

3 Applications and Existing Challenges

3.1 Data Technologies

Nature-inspired algorithms (Genetic Algorithms, Particle Swarm Optimization, Foraging, Ant colony …) are the state-of-the-art solution techniques for some problems in emerging computing environments such as autonomic computing, ubiquitous computing, P2P systems, the Grid and the Semantic Web, where the interaction of large numbers of decentralised, parallel, asynchronous, and distributed components (software or hardware) is demanded.
Essential for tackling the scalability problem is the introduction of modularity into the system. This requires defining the global goal, designing the activity of the local small entities, defining the interactions among the entities, and achieving the emergence of robust global behaviour. The global goal is not the sum of the local goals, but beyond that. Inspiration from biology, such as the concept of stigmergy (i.e., indirect communication via modifications of the environment), is particularly useful in the design of information systems that can adapt to unexpected environmental changes without pre-programmed system behaviour.

Autonomic Computing

Autonomic computing has been conceived as a holistic approach to computing. The computing paradigm change from one based on computational power to one driven by data [1]. The term "autonomic" comes from the autonomic nervous system found in mammals and other higher order creatures - it refers to things like the heart beating and the function of the sweat glands - in other words, to those necessary body functions that we don't have to think about in order to perform. Some examples of vendor-driven autonomic systems include IBM's eLiza (now formally known as the IBM Autonomic Computing Initiative), Sun's N1 [2] and Hewlett-Packard's Utility Data Center [3,4].

Peer-to-Peer Computing and Web

The evolution of the Internet has led us to the new era of the information infrastructure. As the information systems operating on the Internet are getting larger and more complicated, it is clear that the traditional approaches based on centralized mechanisms are no longer meaningful. One typical example can be found in the recent growing interest in a P2P (peer-to-peer) computing paradigm. It is quite different from the Web-based client-server systems, which adopt essentially centralized management mechanisms [5,6]. (Foreword of Proceedings of BioADIT 2004 (Lausanne, Switzerland) – and 2006 (Osaka, Japan) – a Conference devoted to “Biologically-Inspired Approaches to Advanced Information Technology”)

Storage Area Networks (Example)

Hewlett-Packard. BICAS: Biologically Inspired Complex Adaptive Systems. BICAS is a research group centred at HP Labs, Bristol.
Designing storage area networks is an NP-hard problem. Previous work has focused on traditional algorithmic techniques to automatically determine fabric requirements, network topology, and flow routes. New nature-inspired approaches look at the ability of an ant colony optimisation algorithm to evolve new architectures. For some small networks (10 hosts, 10 devices, and single-layered) it can create networks which result in savings of several thousand Euros over previously established methods [7].

Sensor Networks (Example)

In wireless sensor networks, hundreds or thousands of micro-sensors are deployed in an uncontrolled way to monitor and gather information of environments. Sensor nodes have limited power, computational capacities, memory, and communication capability. Novel nature-inspired schemes for data gathering are proposed where sensor information periodically propagates without any centralized control from the edge of a sensor network to a base station as the propagation forms a concentric circle. By observing the radio signals emitted by sensor nodes in its vicinity, a sensor node independently determines the cycle and the timing at which it emits sensor information in synchrony. For this purpose, a pulse-coupled oscillator model is adopted based on biological mutual synchronization such as that used by flashing fireflies, chirping crickets, and pacemaker cells.

Embodied, situated and Distributed Cognition (Example)

Distributed neural architecture can be designed for the general control of robots, being applied in an example for the generation of a walking behaviour in the Aibo robotic dog. Bio-inspiration is used twice: first it is used in the concept of central pattern generators in animals to obtain the desired walking robot; second it is applied to evolutionary processes to obtain the neural controllers.

Digital Nervous Systems

The phrase “digital nervous system” occurred to Bill Gates during preparations for Microsoft’s first annual CEO summit. Gates predicted that every sizeable corporation will soon have such a system, and that no two will be identical. Vast quantities of information exist in the information system of almost every commercial establishment, already in electronic form but scattered about in different locations and formats, for use on different machines by different groups of people for different purposes. Designing, installing, and updating digital nervous systems will take decades, creating endless opportunities for software firms and professionals (from [8]). Nature-inspired technologies will be the basis for the management and the exploitation of this complex ICT environment.

Unsupervised Connectionist models

Unsupervised Connectionist models based on Statistical Methods like Principal Component Analysis, Exploratory Projections Pursuit and so on. These models are called projectionist models and are applied in several fields as:

Food industry: it is used to analyse the response of different properties (as flavour, texture, etc) of some component of final product in different conditions (temperature- response after freezing-, humidity and so on.)

Spectroscopy Data sets: these models are applied to this type of data set in order to cluster the different components which made up the materials. From a restoration, historical and archaeological point of view it may help for classifying the origin and date of each segment, which will help in the restoration of historical places or monument (stones, stain glass windows, etc ).

Security: these methods can be applied in several areas of security. In the field of Intrusion Detection Systems (IDS), they are applied in order to identify any type of attack (intrusion) to a computer network. The main advantages of these projectionist models are:

• it does not require any previous knowledge in the form of rules
• and it is able to detect unknown attacks, day-0 ones.

Knowledge Generation from Data

Examples of application areas of real-time nature-inspired Knowledge Generation from Data include, e.g., autonomous robotic systems (automatic adaptive target recognition, moving obstacle avoidance, cooperative multi-model real-time classifiers, etc.), bio-informatics (knowledge extraction from genomic data, reverse engineering, proteomics etc.), customers behaviour analysis (marketing and Internet user data, communication, mobile workforce related problems, etc.). In customer relationship management, for example, large amounts of data about customers are collected every day. The analysis of such data can inform about changes in customer behaviour, for instance the conditions (special offers, advertisements, etc.) under which a customer tends to buy certain products. Another important issue concerns the prediction of customer preferences. Such predictions may have an immediate influence on the production, especially for products like cars that offer a great variety of extra equipment and devices. Generating interpretable knowledge that evolves over time by incremental learning from continuously arriving data with adaptive dimensionality reduction, feature selection is very attractive and desirable. Even though most of these desiderata have already been addressed individually, learning systems satisfying all of them are not yet available.
A brief summary of the open problems in this area and some examples of on-going work in collaboration between industrial or academic partners are:

Changing concepts and patterns. This is an acute problem in fields like process industries and robotics. Examples: non-stationary signal processing, moving images, dynamic non-stationary processes (Ford, BAE Systems).

Very large and heterogeneous data sets. Even if the data set to be analysed is static, it may be too large to process in batch mode. Moreover, analysing a data set as a whole might be problematic if the data is heterogeneous. In this case, good models might be found for different subsets of the data (locally valid sub-models) but not for the complete data set.
Examples: market-basket analysis (Retail Analytics Ltd.) , mass-spectrometry data analysis in proteomics

Scalability and model adaptation. In many applications, the knowledge model may soon become outdated. Instead of designing a completely new system, a viable alternative is to adapt the existing one. In fact, a gradual evolution of an existing system has several advantages. For example, quickly adapting the rule-base to new operating conditions will improve process security. This legacy problem has implications, e.g., to decision support in the bio-medical area, speech processing, or hand-written character recognition. Moreover, if a solution has to be designed for a new but similar application, costs can be saved by taking over large parts of an existing system.
Examples: Error concealment in VoIP communication (Nokia)

Hybrid systems. In many applications, one disposes not only of empirical data, but also of knowledge coming from human experts.

Speed of learning and computational simplicity. In many applications, for example in the field of robotics or traffic management, measurements are recorded in real-time. Correspondingly, these applications require real-time model evolution and, hence, incremental learning techniques with especially high computational performance (Ford, BAE Systems).

Adaptive feature selection. In industrial applications, the number of features extracted from objects like images or signals is typically large. As learning in high-dimensional input spaces is difficult and irrelevant features may impede the learning, the problems of dimensionality reduction and feature selection become crucial. The data space dimensionality may change over time, these techniques have to be used in an on-line mode. The questions of how to perform feature selection incrementally and how to adapt a model to a modified input space pose challenging problems.

Social Networks and Online Communities

A vast amount of research that concerns social networks has been conducted in the real world, e.g. for families or student and employee communities. The challenge is to investigate whether the behaviour and relationships within conventional social networks can be mapped into the online communities.

Creation of the taxonomy. In order to classify the existing online social networks, their taxonomy ought to be created. According to the approach proposed by the authors of this study, the following types of social networks can be distinguished: dedicated (e.g. dating or business networks, networks of friends, graduates, fun clubs), indirect (online communicators, address books, emails), common activities (e.g. co-authors of scientific papers, co-organizers of events), local networks (e.g. people living in the neighbourhood), families, hyperlink networks (links between home pages). However, they are not well defined and described in terms of the social network science.

The measurements and characteristics of the social networks. The problem of the social capital and the possibility of its management. The relevant measurements as well as their automatic estimation methods, which enable to characterize the social capital of members and the entire online social networks, should be developed. The measurements, known from conventional networks, need to be verified whether they can be applied in the online social networks. New measurements that enable one to classify the relationships are necessary to define.

References

[1] http://www.research.ibm.com/autonomic/

[2] P.Strong, Enterprise Grid Computing, ACM Queue, vol. 3, no. 6 - July/August 2005.

[3] HP Unveils First Intelligent IT Infrastructure to Reduce Costs and Enhance Agility for Large Computing Environments, News release, Nov. 5, 2001.

[4] D.Farber, Utility computing: What killed HP's UDC?, ZDNet, Sept. 2004.

[5] A.J.Ijspeert, M.Murata, N.Wakamiya (Eds.), Biologically Inspired Approaches to Advanced Information Technology: First International Workshop, BioADIT 2004, Lausanne, Switzerland, January 29-30, 2004, Lecture Notes in Computer Science Vol. 3141.
http://lslwww.epfl.ch/bio-adit2004/

[6] The Second International Workshop on Biologically Inspired Approaches to Advanced Information, January 26-27, 2006. Senri Life Science Center, Osaka, Japan.
http://www.ist.osaka-u.ac.jp/bio-adit2006/

[7] http://www.hpl.hp.com/research/bicas/pubs/abstract-29.html

[8] J.Case, Digital Nervous Systems: Making Sense of Shared Information, SIAM News, Vol. 32, No. 10, 1999.

3.2 Networks

3.2.1 The telecoms context

Within telecommunications companies, billions of pounds are being invested in new networks, new services and new technologies – not just to provide telephone services but internet, data, broadband and mobile services too. They are pushing back the frontiers of technology to offer services like virtual markets, electronic commerce, broadband, mobility and security. In fact, in the last years the leading companies in the telecommunication business have shifted their interest from the traditional management of the physical telephony network (i.e. their infrastructures) to the management of all sort of new networks: ranging from sensor networks to social networks. These two extremes illustrate the two new strategic objectives of any telecommunication company today. All sorts of networks are going to invade our space: home networks, traffic information networks, medical knowledge networks, etc. And if we want to avoid spending our lives administrating these networks self-configuring, self-managing, self-healing technologies for networks will be needed. In other words we could say that these “networks” should be alive - one reason for looking for inspiration from nature. On the other hand, what is transported and managed in these networks is no longer just an electric signal. Today, the telecoms industry is interested in managing knowledge and people. This is a completely new challenge for this industry to address. It is plausible that nature inspired systems are especially suitable for this type of complex, human-related networks.

3.2.2 Existing challenges

Even if the telecommunication industry is changing into more global information technologies, it still has to manage and provide a quality service when exploiting its telecommunications network. What we call here the old challenges, are problems oriented around the physical operations on the infrastructure. The general idea behind it, is to find fast and efficient solutions for the continuous dynamic changes of the networks in terms of issues ranging from topology to demands for services. As an example, in 2000 British Telecom research had a large number of projects active in this area. Most of them focused on the optimisation of some particular aspect of their networks. Unsurprisingly most of the nature-inspired solutions were of the evolutionary type, since it was known that these techniques are quite good for optimisation purposes. In the following, we present some of the challenges addressed by these projects and the proposed solutions:

(i) Network design and planning

The optimal design of telecommunications networks infrastructures demands considerations of many complex factors such as type, number and position of components and cable paths – consistent and cost-effective solution. Genetic Algorithms can generate different network configurations and evaluate them rapidly to arrive to an optimal or near-optimal solution [35].

Another problem tackled by the industry is how to respond to rapid network growth, as for instance for the dial-up IP access network. In this framework BT developed an approach in which a Genetic Algorithm is employed to produce optimised sets of planning rules that can be used by network planners and designers to aid in the task of growing telecommunications networks. The optimisation takes into account basically constraints of cost and quality of service. [33, 36].

(iI) Network traffic management

Telecommunication networks are interconnected by routers such that networks protocols must find a path to connect them. This process is known as routing. The usual approach is to estimate the shortest path using a heuristic. BT proposed to combine this with a Genetic Algorithm in order to find solutions, which lead to less congested nodes and links and a better utilisation of network resources. [37].

(iii) Discontinuous communication or limited bandwidth networks

When facing discontinuous communication or a limited bandwidth network then it is not possible to have heavy and/or continuous exchange of data. In this case, a natural idea is to move the software instead of moving the data around. For this problem BT proposed an insect-inspired approach. The idea is that mobile software agents will perform local processing of data as opposed to remote access. But to execute remotely, mobile agents move (as insects) to a machine running a mobile agent server, which provides an interface to the underlying host machine. [38]

(iv) Perspectives for nature inspired solutions of the “old challenges”

British Telecom research laboratories are not only interested in adapting solutions (e.g. existing algorithms) to the industrial problems, they also look to improve them using their knowledge of the limitation based on confrontation with real world applications. In this line BT has proposed several fundamental research perspectives for genetic (evolutionary) computation, among them the genotype-phenotype mappings and the introduction of economics into the evolution.

The idea behind the genotype-phenotype mappings is to separate the coding (genes - genotype) and the actual resulting organism (phenotype), which is actually the result of a complex developmental process played out as the genetic information is interpreted. The idea of introducing economics into evolution is to complete the genetic-inspired view in which interaction occurs only via exchange of agent characteristics encoded as genes, plus measures of success. And an economic-inspired view has agent adaptation driven by changes in prices and supply and demand.

33. Hoile, C. and R. Tateson, Design by morphogenesis. Bt Technology Journal, 2000. 18(4): p. 112-121.

34. Miller, J.F. and W. Banzhaf, Evolving the Program for a Cell: from French Flags to Boolean Circuits, in On Growth, Form and Computers, P. Bentley and S. Kumar, Editors. 2003, Academic Press: New York. p. 278 - 302.

35. Poon, K.F., Successful application of genetic algorithms to network design and planning. Bt Technology Journal, 2000. 18(4): p. 32-41.

36. Shipman, R., Coupling developmental rules and evolution to aid in planning network growth. BT Technology Journal, 2000. 18(4): p. 95-102.

37. He, L. and N. Mort, Hybrid genetic algorithms for telecommunications network back-up routeing. Bt Technology Journal, 2000. 18(4): p. 42-50.

38. Ghanea-Hercock, R., J. Collis, and D. Ndumu. Co-operating Mobile Agents for Distributed Parallel Processing. in Third International Conference on Autonomous Agents (Agents ‘99). 1999. Seattle, Washington.

3.2.3 Applications in finance

(i) International financial markets as nature-inspired networks

The international financial markets form an interesting nature-inspired network (NIN) on which information is continuously exchanged and discounted. What do nodes of the network represent?

1. Homogeneous markets located in different geographical regions. For example, the New York Stock Exchange, the Frankfurt Stock Exchange, etc.
2. Heterogeneous markets, possibly within the same national border.

Fixed-income (bond) market, foreign exchange market, derivatives market, etc.

Financial markets are not autonomous entities, as “shocks” to one market have an effect on the behaviour of others. Such cross-influences, however, can manifest themselves in various ways. For example, we have “leading” and “laging” nodes (infuential US equity markets versus emerging South-eastern Europe, Asia stock markets). Markets resemble natural systems in many social/behavioural characteristics:

• adaptivity: just as natural systems adapt to environmental changes (temperature changes, harsh weather conditions, etc), markets respond accordingly to incoming economic and political information.
• idiosyncracy: markets do not respond in a mechanistic way. Although certain policies and economic decisions are intended to favour certain market reactions, the expectations and predispositions of the market participants can drive prices in the opposite direction! The prevailing market “climate” or “mood” is often a more important determinant of market prices than the fundamental news themselves.
• irrationality: Agents’ expectations and reactions are not solely based on economic fundamentals but can also be the result of irrational exaggeration, herding, etc.

(ii) Three important components of the markets cross-dynamics

1. 1. Short-term mean dependencies: it is often the case that past returns on one market can on average predict future returns on others (i.e. this market “causes” the others). Analysts are often interested to know how an unexpected decline in a major U.S stock market will affect continental ones. Causality may or may not be bi-directional and can vary substantially in time.
2. Long-run relationships: Many markets tend to follow each other in the long-run. Empirical studies show that this tendency is stronger in the “neighbouring” nodes of the network (e.g. markets that are members of a broader economic union as is the EU). Equilibriums are not static but always perturbed by random shocks (statistical equilibriums). One can think of long-run relationships as attractors to individual series, in the sense that whenever short-run deviations are observed, markets are expected to react in a way so that equilibrium is restored.
3. Volatility spill-over: Market dependencies are not only observed among mean returns but also among the second moment of the distribution of returns, i.e. the volatility. A sudden increase in the short-term volatility of a market may have an affect on the volatility of other markets. It is very important therefore to note that market dependencies are not only observed among mean returns but also among the second moment of the distribution of returns, i.e. the volatility. This fact adds additional levels or dimensions into the analysis of the cross-dynamics between the international markets. These dimensions are not independent and empirical work has shown that past unexpected returns cause a volatility increase which in turn affects future returns. Such findings are also supported by financial theory.

(iii) Several special features of the application domain

Empirical studies have shown that price series (levels) follow unpredictable or stochastic trends, exhibiting a “random walk” behaviour. Returns series on the other hand are mean-reverting. In estimating models with trending time-series there is a high possibility of getting “spurious” or phenomenal relations where none in fact exists. Granger and Newbold [44] first demonstrated the spurious regression problem by means of a simulation study. They showed that “unit roots” invalidate standard statistical inference on regression models. Testing hypotheses about the coefficients of the regression using standard statistics, like t-ratios, Lagrange Multiplier or Wald, may wrongly indicate significance of parameters leading to false statistical inference. It is very likely that the problem of spurious relations carries over to nature-inspired methodologies. It is therefore always safe to build predictive models based on mean-reverting returns series. There is still the issue of how to discriminate a trending from a mean-reverting series. We are mostly interested in detecting unpredictable (stochastic) trending series, i.e a time series that cannot be turned into a stationary one by simply subtracting a deterministic function of time, but has to be differenced one or many times. In econometrics there are formal statistical procedures to test the trending or mean reverting behaviour of a time series. These are the so-called unit-root tests, such as the (Augmented) Dickey-Fuller (ADF), Phllips-Perron (PP).

A statistical analysis of financial time series shows that simple regression models do not get the whole picture of cross-dependencies, as those are intended to capture relations in mean. Returns series follow no trivial statistical laws that highly deviate from the “normal white noise” prototype. However, other features of the empirical distribution are of interest to the financial analysts. In addition, the probability-generating process is not invariant but changes with time. This particularly applies to the second moment of the conditional returns distribution, i.e. the variance. The above features should be of concern in any application of nature-inspired methodologies.

3.3 Modelling, Optimisation and Control (NiMOC)

Biological Computation includes efforts to determine how biology processes information technology from the sub-cellular level to the systems and population level. The three main categories are:

• in-silico systems for fundamental understanding
• hybrid systems to reverse engineer the biology
• systems biology at the multi-cellular level and beyond

Nature Inspired Algorithms are most appropriate for problems of optimization, scheduling, chemometrics,
routing, and assignment, management, organization, and logistics. They are sometimes associated and
combined with meta-heuristics and should at best be considered in common, in theory and foundation as well as in application. Meta-heuristics/nature-inspired computing include local search, tabu search,
simulated annealing, adaptive memory procedures, scatter search, ant systems, artificial immune
systems, bacterial chemo-taxis, fuzzy logic, soft computing, evolutionary methods, particle swarm
optimization and their hybrids. Meta-heuristics/nature- inspired computing are helpful in data mining and integrating qualitative knowledge in optimization. Meta-heuristics/nature-inspired computing may find
applications in manufacturing, drug design, medicine, banking, financing, geography, and all other
practical disciplines.

Application fields of the nature-inspired modelling are also in microbial, human, plant cell and social
systems, such as:

• Intra- and inter-cellular communication
• mixed pathogen – host interactions (infections), vaccination strategies
• wound healing
• morphogenesis
• cancer cell metabolism
• vision
• metapopulation and regional dynamics in plants

Artificial Intelligence Algorithms include Genetic Algorithms and Neural Networks, both in several
variants. Recently, there are being considered algorithms based on morphological development and on the theory of cognitivity. All of them play a role in Game theory: Neural networks, genetic algorithms,
reinforcement learning. Genetic Algorithms are a major component of Artificial Life, too. In addition,
evolutionary algorithms and cellular automata are largely used in Artificial Life models and simulations. Real world applications still seem to be rare. However, Artificial Life models and simulations have good
perspectives to understand evolution and cognition. A particular field is application to molecular evolution guided by environmental conditions, in Preparative Evolutionary Biotechnology.

Further examples of nature-inspired systems are Acoustic Sensor Systems and Peer to Peer applications.

The University of Birmingham, School of Computer Science [4], offers a course on Natural Computation 2005/2006 which opens a wider perspective on nature-inspired algorithms than just on data processing.

Natural Computation is the study of computational systems that use ideas and gets inspiration from natural systems including biological, ecological and physical systems. It is an emerging interdisciplinary area in which techniques and methods are studied for dealing with large complex and dynamic problems. It covers a number of topics, such as evolutionary algorithms, co-evolution, evolutionary design, nature-inspired optimisation techniques, evolutionary games, novel learning algorithms, artificial neural networks, theory of natural computation, molecular computation and quantum computation.

nature-inspired Modelling, Optimization and Control are dedicated to the investigation of intelligent paradigms existing in Nature and studied by systems approaches, such as Systems Biology, in order to learn from them how to better design smart, i.e. intelligent, adaptive and advanced information systems.

Regulatory gene expression and cellular signal transduction may be considered as some kind of data processing or information processing. Together with motif recognition on promoters and enhancers they seem to have the potential for the design of new Nature-inspired algorithms of data and information processing.

Reconstruction of networks, such as gene regulatory networks, signal transduction networks and metabolic networks, by Reverse Engineering in Systems Biology, provides hypotheses about the mechanisms that underlie the observed behaviour of living matter investigated, e.g. cells, organisms or populations.

Within NiSIS, Reverse Engineering in Systems Biology may also be considered an essential first step to elucidate/reconstruct some of Nature’s information processing principles in order to proceed towards design of more advanced artificial information systems.
Existing nature-inspired methods of modelling, optimization and control were presented in the recent NiSIS Spring School and Workshop on Reverse Engineering in Systems Biology (Jena, June 09/10, 2005):

• Reverse Engineering of biological networks
• Analysis of high-dimensionality data
• Data-based interaction analysis
• Knowledge-based network analysis

Network reconstruction methods may be subdivided into:

• Non-directed models such as correlation networks
• Directed deterministic networks such as Boolean networks
• Directed probabilistic networks such as Bayesian networks
• Rule-based networks

Linear Algebra Methods for transformation of the N*M gene expression matrix:

• To reduce the number N of genes since N>>M, the number of measurements
• To restrict the variety of models by enhancing the sparsity of the gene expression matrix

These include singular value deccomposition (SVD), Principal Component Analysis (PCA), Maximum a posteriory (MAP) estimation in Bayesian Inference.

Other approaches to Nature-inspired Modeling, Optimization and Control represented by NiSIS members are:

• Fuzzy rule-based modeling
• Linguistic reasoning and fuzzy modeling and inference,
• Fuzzification as coarse graining in multiscale modeling and simulation
• Cellular automata, Turing models
• Neuro-fuzzy hybrid models,
• Coupling intragranular dynamics with extragranular dynamics
• Modeling from sparse data
• Hierarchical decomposition of decision-making and control
• Modeling self-organizing adaptive behaviour
• Multi-objective optimisation/goal seeking using Particle Swarm Intelligence,
• Modelling Coupled moduls
• Artificial Immune System
• Piecewise Linear Dynamic Modeling
• Network Modelling and Simulation from sparse, incomplete and uncertain data
• Qualitative Network Modeling

Examples and Applications

Application fields of the nature-inspired modelling are also in microbial, human, plant cell and social
systems, such as:

• Intra- and inter-cellular communication
• mixed pathogen – host interactions (infections), vaccination strategies
• wound healing
• morphogenesis
• cancer cell metabolism
• Vision
• metapopulation and regional dynamics in plants

Artificial Intelligence Algorithms are mainly Genetic Algorithms and Neural Networks, both in several
variants. Recently there are being considered algorithms based on morphological development and on the theory of cognitivity. All of them play a role in Game design: Neural networks, genetic algorithms,
reinforcement learning. Genetic Algorithms are a major component of Artificial Life, too. In addition,
evolutionary algorithms and cellular automata are largely used in Artificial Life models and simulations. Real world applications still seem to be rare. However, Artificial Life models and simulations have good
perspectives to understand evolution and cognition. A particular field is application to molecular evolution guided by environmental conditions, in Preparative Evolutionary Biotechnology.

Further, examples of Nature Inspired Systems are Acoustic Sensor Systems and Peer to Peer applications.

Existing Nature-inspired methods of modelling, optimization and control are numerous but scattered and
isolated. Urgently needed is their integration into some repository, arsenal or platform where they are
available from for flexible incorporation into the strategic pipeline of modelling, optimization and control.
Future work in the field of Nature-inspired smart information systems must furnish the transition from
recovering Nature’s principles to using this knowledge for the design of advanced information systems. This transition is often called the transition from Systems Biology to Synthetic Biology (or Constructive Biology).

Already Kitano’s Symbiotic Systems Project http://www.symbio.jst.go.jp/) contains this transition. Kitano
refers from the very beginning to total genomic information from genes and proteins, in contrast to former attempts of Systems Biology at physiological level. He calls four research fields of importance for Systems Biology

• Genomics and related molecular biology
• Simulation, Bioinformatics, Software Development
• Analysis of dynamic systems
• High technology for reliable measurements

The tasks of Systems Biology are

• System Structure Identification
  (The structure of a cell is not restricted to its physical structure of membranes and organelles but
  concerns also gene regulatory networks and metabolic networks, their topology and interaction. In multicellular tissues, organs and organisms structure concerns additionally the spatial arrangement of cells, their interaction in immediate contact or via signal transduction networks. Finally identification means determination of structural model parameters.)
• System Behaviour Analysis
  (Analysis of system behaviour is passive quantitative or qualitative monitoring of the system for
  investigation of input/output behaviour without any active enforcement of a particular output behaviour)
• System Control
  (System Control, in contrast, is active reinforcement of a particular output behaviour by appropriately chosen input signals)
• System Design
  (System design is purposeful modification of a system or modular construction of a system with
  induction of a pre-defined input/output behaviour in mind.)
  Current and future Nature-inspired Modelling, Optimization and Control Algorithms are and will be in the following disciplines:
  • Artificial Intelligence
  • Swarm Intelligence
  • Ants Intelligence
  • Bees Intelligence
  • Agent Intelligence
  • Immune Intelligence
  • Biotop Intelligence
  • Ecology Intelligence
  • Economy Intelligence
  • Artificial Life
  • Neuroinformatics
  • Socioinformatics
  • Computational Biology
  • DNA & Molecular Computing

These Algorithms are, respectively, will be evolutionary, adaptive, developmental, self-learning, genomic,
genetic, metabolic, proteomic, transcriptomic, enzymatic, allosteric ones.

As explained in preceding sections, Modelling, Simulation, Optimization and Control are the steps on the strategic roadmap of design of Nature-inspired Smart Information Systems that possess pre-defined properties. The description of the design process there refers to manual operations of some designer scientist who repeatedly modifies models, assesses simulation behaviour, adjusts model parameters by optimisation, and controls stability and long-term performance of desired simulation behaviour.

It is clear that such a time-consuming (and error-prone) design procedure has to be automated to large an extent although nevertheless man-made options and decisions must remain allowed at places where human intelligence is superior to machine intelligence. It deserves some reasoning to decide on the right balance between interactivity and autonomy of the design procedure.

4 Grand Challenges

The conceptual Grand Challenge for building smart information systems is the mimicking of many of the desirable qualities, features and capabilities of the natural systems showing intelligent behaviour, both in their distinct functionalities and in their aggregated actions.

We can identify the collection of these characteristics under the common umbrella of Bio-mimetic Intelligence (BmI) or, in other words, the ability of an information system to mimic nature-inspired adaptive and intelligent behaviour to better pursue its goals, to improve the robustness, efficiency and usefulness of its functionalities and enhance its interfacing capabilities to the external world.

The word Bio-Mimetic expands and, at the same time, specifies the target research field. While terms like “Artificial Intelligence”, or the more recent “Computational Intelligence”, have been identifying a precise notion of intelligent behaviour by mimicking human intelligence through symbolic or subsymbolic computation , BmI tries to grasp the core of intelligent behaviour from a broader perspective, through technologies inspired by natural systems behaviour. In fact, the idea of BmI is to extend the inspiration from the micro to the macro-level of natural intelligent behaviour, that is from organic or inorganic molecules (e.g. DNA computing, evolutionary computation, etc.), through unicellular organisms and tissues (e.g. membrane computing, quorum sensing, amorphous computing, neural computation, etc.), up to complex organisms and their aggregations (e.g. swarm intelligence, ants colony, intelligent agents, social networks, etc.).

It should be noted that, even if some of the biological details giving rise to intelligent behaviour are unknown and will be obscure even in the near future, nevertheless information systems can benefit from macro-level BmI as a source of inspiration. Already successful cases along this line are, for example, the evolutionary computation and swarm intelligence research fields, that have shown exceptional advances in recent years for information systems despite the corresponding biological mechanisms are not clearly understood.

Thus, the development of new and advanced Smart Information Systems relies both in the understanding and mainly on the imitation of intelligent behaviour. Several of the Grand Challenges for Information Systems are still the same as many decades ago, despite the vast amount of technological advances, but having Bio-mimetic Intelligence as the background scenario makes it possible to see those challenges under a new light and stimulate new research methodologies.

4.1 Grand Challenge 1: Computational Nervous System

The rationale behind the design of a Computational Nervous System (CNS) is the development of “sensing” capabilities in information systems. These capabilities, when addressing an information system, are related both to the acquisition of information from the external world (e.g. artificial vision, speech understanding, etc.) and the understanding of its internal functioning and performance (e.g. autonomic computing).

The ultimate goal is to provide an information system with a CNS able to acquire data and information in a robust way and, at the same time, able to manage uncertainty and as in biological systems, self-adapt and self-repair.

The expected benefits can be: the increased ability to acquire information; the additional safety provided by self-assessing mechanisms and, therefore, the better quality of the acquired information; the robustness aspect to systems failure or performance degradation; a better, fitter and efficient representation of the external world and the system inner status.

Technological and scientific improvements are required in the field of sensors, sensor networks, nonlinear control systems (e.g. robustness, stability, adaptivity, etc.), evolutionary computation, computational intelligence and machine learning, signal processing, etc.

4.2 Grand Challenge 2: Brain-like Computing

The rationale behind the design of Brain-like Computing (BlC) is the development of an information system with brain-like computational capabilities. Two main methodologies appear to be promising: the first one tries to understand and then map brain functionalities on conventional computer systems , the second one builds on next-generation computational devices for building computing systems having a complexity comparable to biological systems.

The ultimate goal is to provide an information system able to take real-time decisions, extract, maintain, manage and memorize valuable spatial-temporal information and knowledge in efficient ways, perform associations between problems and solutions and, eventually, build new solutions to unknown problems (e.g. through creativity processes).

The expected benefits can be: real-time information processing and understanding, better “natural” data processing and understanding, increased efficiency of data processing.

Technological and scientific improvements are required in the field of: heterogeneous and hierarchical data representation and processing, continuous vs. discrete representation and processing, bio-inspired and physics-inspired computational algorithms, self-assembly and self-configuration processes, computational intelligence and machine learning, etc.

4.3. Grand Challenge 3: Distributed Cooperative Intelligence

The rationale behind the design of Distributed Cooperative Intelligence (DCI) is the necessity of dealing with the increasing complexity of information systems through non-centralized mechanisms, as in biological systems.

The ultimate goal is to develop information systems able to survive and improve their fitness through time by the distributed optimization and adaptation of all its components.

The expected benefits can be: improved robustness to failure, computation errors and run time, improved computational efficiency, improved robustness to network communication faults, improved control of distributed systems, plants or organizations.

Technological and scientific improvements are required in the field of: networks, swarm intelligence, ants/bacteria inspired algorithms, intelligent agents, computational intelligence and machine learning.

4.4. Grand Challenge 4: Device and Communication Networks

Advances in micro-electronics have led to a vast increase in the processing power and/or sensing abilities of many everyday objects. At the same time, advances in communication technologies (both wireless and wired) mean that it is possible to transfer data much more quickly and efficiently than before. Taken together, these advances are leading to complex networks involving increasing scale and heterogeneity. Examples include next-generation telecommunications networks and pervasive computing systems.

As a historical parallel, the early days of telephone networks required manual switching of calls by operators. Without automation, either telephone calls would be a rarity or the number of telephone operators would be several orders of magnitude larger. The equivalent problem today is the number of highly skilled technical personnel to configure and maintain complex networks. The challenge (identified by IBM, amongst others, in its autonomic computing initiative) is to automate these processes by emulating the way the body regulates and controls itself. This entails subsidiary goals of self-configuration, self-monitoring, self-optimisation, self-maintenance and self-protection.

A further challenge arises from the need to understand dynamic changes in networks - without the tools to aggregate and visualise changes in networks, it will be impossible to control and trust such systems.

4.5 Grand Challenge 5: Advanced Information Networks - Heterogeneous Information Management

The “Information Society” envisaged by the EU is underpinned by widespread access to, and processing of, the enormous amounts of information available through advanced information and communication technologies. There are essentially no limits to the quantity of data that can be stored. There are however very substantial problems in our interaction with these huge “information spaces” - how can users find the information they need, be sure that they have take into account all relevant pieces of information that relate to their problem, how can they quickly understand the retrieved information and how can they cope with the continual updating of the contributing sources.

The success of Google as a de facto standard for searching is an indication of the need for new approaches which go beyond the simple keyword indexing approach. A subject expert approached for advice is not likely to adopt the Google presentation of answers - “the solution may be somewhere in these 5 million documents” - but will instead seek to clarify requirements and present information in a way suited to the person asking the questions.

The brain can be seen as a source of inspiration for the solution of these challenges. The notion that “I don’t know what I’m looking for but I’ll know it when I see it” cannot be captured logically but is a powerful motivating idea. Associative memory - the linking of concepts related in various (and often subtle) ways - is likely to be an important component, as is the way in which the human mind can re-organise concepts. Use of visual and metaphorical links as well as more traditional “logical” reasoning are also likely to be relevant.

Future information systems will find previously unknown connections, and be able to filter out the meaningless from the meaningful with little human assistance.

4.6 Grand Challenge 6: Nature-inspired Modelling

The modelling of high-dimensional non-linear processes in space and time is a massive challenge. Rational reduction of the dimensionality (e.g. identification of key variables, feature selection) is one of the main tasks. The so-called curse of dimensionality is the main obstacle that renders modelling of complex systems difficult. There exist methods of both global and local dimensionality reduction. There exist several methods of feature selection. The analysis of genetic data poses statistical problems in the form of high dimensionality with small sample sizes. The construction of a composite gene region (sequence pair) heterogeneity measure is one technique for reducing the dimensionality of the problem. A different method is clustering of co-expressed genes. Efficient methods for network inference including non-linearities (learned from multi-stability and instabilities in living systems) are needed. The dynamic theory of Hypercycles by Eigen and Schuster describes gene expression as a non-linear phenomenon with multi- and instabilities by stationarity and stability analysis, however, without any reference to the sequences of bio-molecules and the genetic code. To achieve the given goal, a combination of these approaches is required by some kind of hybrid modelling.

Context-dependent network models can be learned from protein-protein interaction networks. The selective values of alleles in a molecular network model are context-dependent. For mapping genotypic to phenotypic variation, the current understanding of the mechanisms of gene expression indicates the importance of non-linear effects resulting from gene interactions.

Modelling of growing, maturation and aging networks have to be learned from biological and social networks. Large scale systems like communication networks, large databases and software systems, the Internet, large distributed control systems, businesses and the global economy, are examples of huge, interdependent open information-processing systems with behaviour that is increasingly difficult to predict and control. Modelling, simulation, design and control of such large scale systems in technology, business and the sciences are major issues to address in the coming years.

Model inference from incomplete, uncertain and heterogeneous data will be learned e.g. from metabolic network interference from genome, proteome, metabolome data. As is well known, methods for uncertain data analysis with application to knowledge extraction from data may roughly be subdivided into probabilistic modelling (Bayesian Networks) and fuzzy logic methods (Fuzzy Clustering, Rule-based Network Models).

Modelling of complex modular systems will be a challenging task that can be learned from cell-cell interaction, as e.g. in a heterogeneous liver cell bioreactor or in the liver as such; quorum sensing; cross-talk of cells. The types of cell-cell communication can be subdivided into: a) communication by cell-cell contact and b) communication by secretion of cytokines. The cytokine network is very complex. The cytokine system demonstrates great redundancy and great pleiotropism. Redundancy in this context means that most functions of cytokines can be performed by many different cytokines. Morphogenesis is governed by the interplay between differential cell adhesion, gene-regulation, and intercellular signaling. All cells in multicellular organisms experience cell-cell interactions in a range of different contexts such as tissue construction, intercellular communication, information transfer and spatial awareness. Quorum Sensing (QS) describes the phenomenon whereby the accumulation of signalling molecules enables a single cell to sense the number of bacteria (cell density). QS is chemotaxis control.

Model validation is a crucial task which can be done either by experimental verification of model-predicted process states (e.g., optimal states) or by successful application, e.g. in soft sensors for optimal control. For this task standardized tools have to be developed. 

4.7 Grand Challenge 7: Nature-inspired Optimization and Control

Bio-mimetic solutions in evolutionary algorithms that mimic not only micro-evolution but also macro-evolutionary strategies (learning from the ‘big steps’ in biological evolution) are grand challenges in nature-inspired optimization and control. Micro-evolution is defined as the change of allele frequencies (that is, genetic variation due to processes such as selection, mutation, genetic drift, or even migration) within a population. Macroevolution is defined as evolutionary change at the species level or higher, that is, the formation of new species, new genera, and so forth. Evolutionary novelties at macroevolutionary level appear through manifestation of genetic mutations at microevolutionary level, in a new metabolic network, leading to different phenotype in both protein equipment and behaviour. The biomimetic translation means emergent changes in simulation behaviour (phenotype) by minor changes of the model (genotype).

In computing, multitasking is a method by which multiple tasks, also known as processes, share common processing resources such as a CPU. Its optimization concerns CPU usage by the scheduling strategy. It is not known if biological multitasking in the liver can be a model for multitasking in computing although presumably in biology, too, multitasking operates through ressource sharing or by division of labour among different cell types. Optimization by permanently renewed (recyled) material can be learned e.g. from the regeneration of liver cells/hepatocytes, based on modelling of spatio-temporal organization.

Self-configurability with respect to network services is the capability of a system to configure its own network-based services and applications in response to the needs of the user and the environment the system finds itself in. As the needs of the user change or the environment the system is in changes, the end-user system would recognize the change, understand the impact, and respond by reconfiguring itself accordingly. Flexibility of location, a wide range of administrative control, and the need for varied rates of dispersal of changed configuration information across the environment, all complicate the task of autonomic network services configuration behaviour.

Control of interconnected processes can be learned e.g. from co-ordinated action of E. coli’s catabolism and anabolism. Anabolism is the building up of complex molecules, while catabolism is their breakdown. To build molecules and sustain life, the body needs energy. So, for molecular construction to occur, molecular destruction must go on at the same time to release the energy required to drive the biochemical reactions. There are many interconnected processes in tumorigenesis, involving tumor cell signaling and information processing.

Sensing of multiple signals and information processing toward multiple targets should be learned e.g. from signalling pathways into and within cells including cross-talk of such pathways, signal transduction via membranes, second messengers. There exist information-based approaches to processing and organizing spatially distributed, multi-modal sensor data in a sensor network. Energy constrained networked sensing systems must rely on collaborative signal and information processing (CSIP) to dynamically allocate resources, maintain multiple sensing foci, and attend to new stimuli of interest, all based on task requirements and resource constraints. Target tracking is an essential capability for sensor networks and is used as a canonical problem for studying information organization problems in CSIP.

Data-Filtering by improved Artificial Immune Systems will be learned e.g. from the defence of internal but harmful peptides/proteins e.g. coming from tumour cells, and considering also dysregulation during autoimmune diseases and its therapy. Immune System-inspired Algorithms of Artificial Intelligence mimic its capability of pattern recognition, memory and clonal selection for anomaly detection, data filtering, feature extraction, and feature selection in Data Mining and Knowledge discovery or for function optimisation.

Multi-agents systems (MaS) can manifest self-organization and complex behaviours even when the individual strategies of all their agents are simple. Topics of research in MaS include: beliefs, desires, and intentions (BDI), cooperation and coordination, organisation, communication, negotiation, distributed problem solving, multi-agent learning, scientific communities, dependability and fault-tolerance.

Self-healing, i.e. diagnosis and reaction to system malfunctions, including regeneration of system components may be learned e.g. from the regeneration capacity of the liver. Several different self-healing mechanisms are subject to research.

4.8 Grand Challenge 8: Nature-inspired Devices and Industrial Plants

Models of biodiversity (many species) and (bio)chemical diversity (many natural products) e.g. with respect to pharmaceutical research and development, lead structure search etc., reflect the result of evolution and fit the changing environmental conditions in bioreactors of artificial evolution to obtain desired functions in growing biomolecule populations by evolutionary design. Adaptation issues with respect to artificial immune systems, may be similar to issues of biodiversity.