|
ECONOMIC THEORY, ANTICIPATORY SYSTEMS AND ARTIFICIAL ADAPTATIVE AGENTS Sérgio Luiz de Medeiros Rivero
Bernd Heinrich Storb Raul Sidnei Wazlawick ABSTRACT In this paper we propose an artificial intelligence approach to simulation in economics based on a multiagent system. The multiagent approach is based on the seminal work of Holland & Miller [6], in which the authors propose that the economic system may be viewed as a complex dynamic adaptive system with a large number of different kinds of agents and that these agents can be simulated using classifier systems. In the model developed in this article the agents take its decisions based on the anticipation of the future state of the world. The concept of anticipation is developed from the work of Davidsson [4] [2]. These agents are heterogeneous, autonomous, adaptive and anticipatory. This model is compared with the one developed by Arthur et al. [1]. In this paper are developed similarity measures between situations, actions, and changes in the world. These measures are useful for a computationally simulated economic agent to compare previous situations, actions and results, and decide which action could lead to a situation with the best utility or satisfaction degree.
This article is a modified version of a
work presented in the 4th Conference on Information Systems, Analysis and
Synthesis - ISAS'98, held at Orlando, FL, in July 12-16, 1998 and first published
in its proceedings at pp. 64-69. Contents · Introduction · Computational Simulation and Economic Theory · Causal Systems and Anticipatory Systems · The Anticipatory Agents Model · Auction · Evolution of Rules · Conclusions and Future Work · Acknowledgements · References
1. INTRODUCTION Economic systems can be considered as evolving complex systems where economic agents rationally perform actions in order to reach a better degree of satisfaction. In this way, the agents organize themselves and make decisions about producing, investing and consuming. Those decisions are taken in a context of uncertainty, and they are based on the beliefs (or expectations) that the agents have about possible future states of the environment. Economic agents live in a complex non-stationary world. Decision problems in this kind of world are hard to solve. Therefore, consistent models for the economic agents’ behavior have to consider two aspects: the uncertainty related to the environment and the limitations of the agents rationality. Thus we will assume Simon’s perspective of limited rationality. "Theories of bounded rationality, ... are theories of decision making and choice that assume that the decision maker wish to attain goals, and uses his or her mind as well as possible to that end; but theories that take into account in describing the decision process the actual capacities of the human mind" [13]. Our artificial economic agents have limited memory capacity to retain all its observations of the past and all its rules that allow to form expectations about the future; furthermore, they will evaluate all the time of their artificial life (this means, the whole time of simulation). The expected future (or possible future) influences the agent’s decision about his acts in order to achieve a goal (profits, for example). The expected future could be one of the factors that the agent takes into account to select the decision that leads him closer to his goal. The errors made by the agent (decisions that take the agent more distant from his goal) influence the agent’s behavior. The agent tries to minimize the difference between the results of his actions and the fixed goals. The objective of this paper is to discuss the application of artificial intelligence techniques in economic systems theory and modeling. For this, we apply the concept of anticipatory agent [3] to an economic system simulation model based on a multi-agent approach, presented for the first time by Holland and Miller [6]. Before stating our model, we discuss the relationship between computational simulation and economic theory, and we present the concepts of causal system and anticipatory system. Finally, we discuss the main differences between our model and the model of Arthur et. al. [1]. 2. COMPUTATIONAL SIMULATION AND ECONOMIC THEORY Historically, economic theorists have tried to find the determinants of the economic development process. Here, development can be understood as a process of structure changing of an economic system. Structure, in a restrictive sense, is the set of relations built by the agents for economic production. Therefore, economic agents are the building-blocks of an economic system. This is a characterization of an economic system that imposes a model with many components (agents) interacting dynamically. This means that economic systems are complex and dynamic systems. Furthermore, it is even more complex because the behavior of each agent is a complex behavior, and the community of agents works in a non-trivial way. We can also say that, not only the structure of the economic system emerges from the individuals’ behavior, but that the agents’ behavior is influenced by the structure. For example, the introduction of a new competitor in an oligopolyc market would change the strategy of concurrent companies. Another example of interaction between strategy and structure is the market shift produced by technical innovations. A typical case is the transformation that occurred in the hardware and software industry (and market) imposed by the advent of microcomputer. This complexity raises a question: how to insert a bit more of reality in our economic models without losing relevance? A possible answer to that question is computational simulation. Computational simulation based on approaches may give more flexibility and explanation power to the hypothesis about agents dynamical behavior. A question derived from that statement is: what kind of simulation is possible for the economic system characterized above? A method for solving this problem is the formulation (via econometrics) and solution (via numeric simulation) of differential non-linear equation models. This is the most used contemporary approach for computational simulation of complex economic systems. But these models have a few problems: they are hard to solve; technological progress and shifting of agents strategies generally imply in changes in the equations coefficients; the behavior of the variables cannot ever be adjusted by a probability distribution, because crucial events (investment decisions, for example) are not suitable for repetition [12] and finally the agent's strategy does not depend only on past and present states of the system, but on the agent's expectations about the future behavior of the system. Alternative computational systems may be a good solution to define analytical/computational models with less complexity. One promising approach is the use of the concept of Artificial Adaptive Agents (AAA) stated by Holland and Miller [6]. That is, the advance of the scientific analytical methods was never a linear process. Simulation techniques, like AAA, can contribute as new tools for analyzing the complex dynamics of an economic system. These techniques may produce relevant results for improving the knowledge about the system and the analytical methods themselves. Otherwise, the existing theoretical results are the necessary starting point for the elaboration of simulations. The notion of economic agent has an obvious parallel with the concept of agent taken from artificial intelligence. Three aspects of the artificial intelligence may contribute to the elaboration and definition of these simulation agent models: a) The economy can be seen as a network where each node (agent) takes decisions in order to allocate its resources without complete knowledge of the system. Those decisions are taken in a decentralized and concurrent way [9]. The strategies of each agent depends on his goals, expectations and the interaction with other agents. b) An adaptive behavior is a desirable characteristic of the agents that can be modeled. Many techniques may be used for the implementation of this adaptive process e.g. Classifier Systems, Genetic Algorithms, Neural Networks and other mechanisms of reinforcement learning. c) The use of fuzzy inference rules may be crucial to approximate the computational model to the real behavior of the agents. This kind of inference is more necessary when the agent works under uncertainty conditions. In those conditions the agents cannot forecast the future behavior of the system, but they can formulate a set of possibilities about the future behavior of the significant variables. Each of these possibilities has associated a confidence degree. The use of classifier systems to simulate the agents' behavior in an artificial stock market was presented in a working paper published by the Santa Fe Institute [1]. The use of confidence degrees to the forecasting of future states of the world is a theme stated by Keynes in his General Theory [8]: "The state of long-term expectation, upon which our decisions are based, does not solely depend, therefore, on the most probable forecast we can make. It also depends on the confidence with which we make this forecast – on how highly we rate the likelihood of our best forecast turning out quite wrong. If we expect large changes but are very uncertain as to what precise form these change will take, then our confidence will be weak". 3. CAUSAL SYSTEMS AND ANTICIPATORY SYSTEMS There is an idea inherited from the behavior of the mechanical systems of physics that imposes a high restriction degree to the analysis of the dynamical behavior of a system. This idea is that the dynamical behavior of a system is completely determined by its past. This kind of system is called a causal system. Formally this means that,
where, ws, 0 £ s £ t+1, is the state of the system at time s; and f represents the functional dependence between the present state and the past. The structure of these systems varies essentially with time, but, does not formalize the influence of the possible future on the present behavior of the system. Even if the system has stochastic components, the knowledge about the probability distributions of these components provides knowledge about the dynamical behavior of the system [11]. When economic agents are considered, it can be perceived that the expectation about the future is relevant for the determination of the agents’ actions. This is an idea clearly stated in economic literature since Keynes [8], [1], [9], [2]. That is, we can consider economic agents as anticipatory agents. The notion of anticipatory agents was defined by Rosen in [1] as "... a system containing a predictive model of itself and/or of its environment, which allows it to change state at an instant in accord with the model's predictions pertaining to a latter instant". That is, anticipatory agents will act according to the past states of the system and according to the desirable and possible future states. Formally,
where This formulation allows the agents dynamic behavior to be modeled in a more realistic form, gaining more explanatory power than a mechanistic conception. However, there is the problem of analytic solution determinations. Here, the approach of artificial intelligence can provide a tool for the formulation and test of these kind of models without exhaustive study of its mathematical properties in a first moment. We may advance making experiments which consider the agent's behavior in a more realistic form, without needing to solve the complex equations derived from that formulation. These AAA models are a complement to the analytical models, and do not intend to replace them. The study of the characteristics presented by anticipatory and adaptive, computationally formulated systems, may enhance the knowledge about its properties in a way that leads to more appropriated analytic models [6]. 4. THE ANTICIPATORY AGENTS MODEL The economic system model based on anticipatory agents has elements of the works of Holland and Miller [6] and Davidsson [3]. This model has the following characteristics: a) It has a large set of different kinds of agents (consumers, companies, government, central bank), consumers and companies with many instances. Instances of the same kind of agent may exhibit slightly different behaviors. b) The agent's strategies are continuously tested and changed depending on the results of the interaction among the agents and the market. In other words, the agents adapt their behavior to the permanently changing conditions of the economic system. c) The global dynamical behavior of an economic system results from the interaction of all instances of the various kinds of agents in the market, i. e. this aggregate behavior emerges from individual behaviors. d) The agent's behavior is influenced by the beliefs it has about the future behavior of the system. This is based on some kind of anticipation about the system’s way of working. That means, our model is formulated as a multi-agent system with
heterogeneous (a), adaptive (b), autonomous (c) and anticipatory (d) agents.
We can say that a parallel, dynamic, evolving, multi-agent simulation is
closely related to the structure exhibited by a market economy. The decision
about resource allocation in a market economy is taken by each agent. It does
not exist a walrasian auctioneer or a central planner [9] to decide costs and
solve the optimal allocation of productive resources. More than that, the
decisions about production and investments are taken by the agents according
to their expectations about the future demand. Those expectations are based
on hypothesis about the future behavior of the system.
The figure 1 shows a superficial structure of our agent model. Where:
4.1 Auction We denote by wt. the state of the world at time t. In this model ws is a vector of p environment variables - ws = (evs,1, evs,2,..., evs,p), where each evs,i, 1 £i £ p has values in a subset of real numbers. Situations are series of states of the world, that is, the situation (S) of size l at time s is S = (S1, S2, ... , Sl) = (ws-l+1, ws-l+2, ... , ws-1, ws). Actions taken by agents may change the state of the world. Changes occurring between time s and s+1 may be measured by rs = ws+1 - ws. We suppose that there is a set of simple primitive actions (sa1, sa2, ... , saq). Each action is a simple operation that an effector may execute. A composed action is defined as a series A of simple actions. At this stage we assume that the effects of a composed action is independent of the order of the execution of the composing simple actions. Then, a composed action may be represented by a vector vaÎNq(where q is the size of the universe of simple actions), indicating how many times each simple action is executed to form the composed action. In the following ||x||n means the Euclidean norms in Rn, that is:
Similarities of results may be defined as:
where r and r’ are changes of states. Similarities of actions may be defined as:
where a and a’ are actions. Similarities of situations of size l may be defined as:
where S and S’ are situations. The expectation is that similar actions in
similar situations will lead to similar results.
where C is a constant that may be adjusted. The confidence in those expectations is determined by the similarities and the size of situations in consideration may be defined as:
where 1.0 indicate the highest confidence. Let
and u be a utility function for a specific agent, then we can define expected utilities (û) of the action a in the situation S at time t for a triple (S’,a’,r’) of situation, action and changes in history, by: a) the median case:
b) the optimistic case:
c) the pessimistic case:
d) considering an factor of happiness:
where hap represents a value of happiness between 0 and 1. The auction is done by the following steps: a) Determining possible future states of the world for each rule that matches the environment state. This will be done by comparing parts of the trajectory of the world, actions taken and changes of the world state by agent’s actions. That is determination of
for actions a that are parts of the set of matching rules and considering only action for that exist a pair (S’,a’) and a present situation S with confidence near to 1. b) Determining the expected utilities for each action considering the possible future world states and the recent trajectory of the agent’s happiness, that is, the determination of the expected utilities for all expected results in (a) utilizing one of the expected utility functions defined above. c) Comparing expected utilities. The winner is the rule (action) with the highest expectation. 4.2 Evolution of Rules The mechanism used to make the evolution of the rules is a classifier system [5] [7]. A classifier system is a "machine learning system that learns syntactically simple string rules (called classifiers) to guide its performance in a arbitrary environment." [5]. A classifier system have three main components. a)
Rule and message system; The choice of this kind of system is based on its suitability to the definition of the anticipatory model. Rule and message system The structure of a classifier is a triple, (c,a,f), that joins a condition c (a state of the world or a situation), an action a (that is a simple or composed action), and a strength f. We use a ternary alphabet (0,1,#) to represent the conditions and actions of the classifier. The symbol '#' has the meaning of 'don't care'. For the similarity determination of conditions (situations) by (6) and actions by (5) we consider that all elements of the alphabet have distance 0 to #. For example, ||(1,#,0) - (0,1,#)||3 = (1 + 0 +0)1/2 = 1. For more details see [5] and [10]. Apportionment of credit system The classifiers strength at time t, (ft), is adjusted in the following way: a) All rules have to pay a Life Tax
(taxlife) at each iteration. That means ft+1 =
(1- taxlife)ft, where taxlife,
0< taxlife < 1, is a value near 0. Genetic Algorithm The evolution of the rules is actually done by the genetic algorithm (GA). The execution of the GA occurs at certain number of iterations stochastically determined, called an epoch. In each epoch only a small part of the rule set is selected for evolution, where the rules with the lower strength are the rules with greater selection probability. The rules of these small subset are substituted by new rules determined through crossover and mutation. The rules for the crossover are also determined by a probabilistic selection, but here, rules with the higher strength are the rules with greater selection probability. Two rules, (c1, a1, f1) and (c2, a2, f2), the parents, selected for the crossover may create two new rules, the child's, in the following two different ways
The mutation process in the GA corresponds to random modification of some rules, randomly selected. In this way, we are following Richards [10] approach, that uses the Michigan approach with steady state genetic algorithm (SSGA). When none of the classifier conditions match the environment state, a special type of mutation operator is used. This operator (the triggered cover detector operator – TCDO) generates a random action for the present environment state. 5. CONCLUSIONS AND FUTURE WORK The characterization of an economic system as a complex, dynamic, evolutionary system, will lead us to some suppositions that make this system a little bit more close to the real economic system. This system will not tend to some equilibrium, but, in fact, it stays "far from a global optimum or attractor" [6]. The analytic solution of a system that exhibits these features is extremely difficult to obtain. Finally, complexity and uncertainty are closely related to great part of economic problems. An analytical solution of these problems may be hard or even impossible to obtain. We think that an artificial adaptive agent approach can provide more freedom for testing hypothesis and give a powerful tool to construct scenarios closely related to reality. The main difference between the proposed agent architecture and the model of Arthur et. al. [1] are: (a) the calculation of expected utility determination is taken by considering world history and the evolution of the utility; (b) the expected utility is used for the determination of the winner and for the evolution of the rule set by the genetic algorithm. The immediate task is to build a computational application to validate the model with empirical data. 6. ACKNOWLEDGEMENTS This work was partially funded by the Brazilian Agency CNPq – project PROTEM-NALAMAS and by CAPES – PICD program. We thank also to the LSC agents group where these ideas were originally discussed and the two referees for their suggestions. 7. REFERENCES [1] Arthur, W. B, et. al. Asset Pricing Under Endogenous Expectations in an Artificial Stock Market, SFI Working Paper 96-12-093. [2] Arthur, W. B., Complexity in Economic and Financial Markets, Complexity, v.1, n.1, April, 1995. [3] Davidsson, P. Autonomous Agents and the Concept of Concepts, Lund, Sw, Department of Computer Science, Lund University, 1996 (Ph.D. Thesis) [4] Ekdahl, B. Astor, E. Davidsson, P. A Framework for Autonomous Agents Based on the concept of Anticipatory Systems. In R. Trappl, ed. Cybernetics and Systems '94, pp. 1427-1434, World Scientific, 1994. [5] Goldberg, D. E. Genetic Algorithms: in Search Optimization and Machine Learning, Reading, MA, Addison-Wesley, 1989. [6] Holland, J. H. & Miller J. H. Artificial Adaptive Agents in Economic Theory, American Economic Review, May 1991, pp. 365-370 [7] Holland, J. H. et. al. Induction: Process of Inference, Learning, and Discovery. Cambridge, MA, MIT, 5th ed., 1996. [8]Keynes, J. M. The General Theory of Employment Interest and Money. Cambridge, Macmillan, 1936. 14th edition, 1973 (The Collected Writings of JMK, v. VII) [9] Leijonhufvud, A. Towards a Not-Too-Rational Macroeconomics, Southern Economic Journal, june,1993. [10] Richards, Robert A.; Zeroth-order Shape Optimization Utilizing a Learning Classifier System Ph.D. Dissertation, Mechanical Engineering Department, Stanford University, 1995. [11] Samuelson, P.A. Foundations of Economic Analysis, Cambridge, MA. Harvard University Press, 1975 – 5. ed. [12] Shackle, G. L. S. Origens da economia contemporânea: Invenção e tradição no pensamento econômico (1926-1939). São Paulo, HUCITEC, 1991. [13] Simon, H. Models of Bounded Rationality: Empirically Grounded Reason (vol. 3), Cambridge – MA, MIT Press, 1997. |
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)