Some Fundamental Topics of Inductive Modeling

Size: px

Start display at page:

Download "Some Fundamental Topics of Inductive Modeling"

Dana Reed
5 years ago
Views:

1 2-nd International Conference on Inductive Modelling { ICIM'2008 Some Fundamental Topics of Inductive Modeling Yuriy V. Dzyadyk International Center of Information Technologies and Systems, Academician Glushkov avenue, 40, Kyiv, Ukraine iurius@inbox.ru iurius@i.com.ua Kyiv, September 15{19, 2008

2 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 2 { Agenda 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria

3 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 3 { The Most Interesting and Essential 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method Comparisons with Other Methods 7. Economic Criteria \Gentlemen! I must herald you the most disagreeable tidings... " (N. V. Gogol') It seems, the (; ){method now is the best method of forecasting. I hope for discussion.

4 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 4 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria 1. Introduction

5 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 5 { (1) We have a series of observations y = (y 1 ; y 2 ; : : : ; y n ) T : We indend to nd such set of independent variables ft 1 ; t 2 ; : : : ; t l g and their observations ft 1 ; t 2 ; : : : ; t l g for building an inductive model (2) ^y = f(t 1 ; t 2 ; : : : ; t l ): We shall assume that model (2) is linear relative to some set (x 1 ; x 2 ; : : : ; x k ) of basic functions, where each (3) x i = ' i (t 1 ; t 2 ; : : : ; t l ): Usually we pick up these basic functions among m+l m monomials of all degrees s m (as a rule, m = 1 or m = 2) (4) sy j=1 t ij ; s = 0::m; 8j : i j 2 f1::lg: Some authors [1] mention as basic for support functions harmonic and logistic functions: 1 (5) sin (t i ); cos (t i ); 1 + e t : i In other works as basic functions are using roots or fractional degrees, logarithms etc. qp (6) ti ; t p=q i (p 2 Z; q 2 N); log (t i ) etc: 1. Introduction

6 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 6 { Let us remind that n denote the dimension of statistics. Thus from l vectors ft 1 ; t 2 ; : : : ; t l g we can compute k vectors (x 1 ; x 2 ; : : : ; x k ) of a real n-dimensional space R n. Now let X denote the matrix (7) (x 1 (x 1 ); x 2 (x 2 ); : : : ; x k (x k )) = X; where (8) 8w 2 R n : (w) = 1 n Note, that x i are vector columns, i are scalars. So, matrix X has n rows and k columns. As model (2) ^y = f(t 1 ; t 2 ; : : : ; t l ) is linear relative to the set (x 1 ; x 2 ; : : : ; x k ) of basic functions, there are such vector a = (a 1 ; a 2 ; : : : ; a k ) (9) hence (10) ^y = ^y = kx i=1 kx i=1 a i x i ; a i x i ; nx t=1 y = Xa: w i : 1. Introduction

7 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 7 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria 2. Problem of Unstability in Linear Modelling

8 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 8 { Let we search for a form a of the linear dependence y = Xa, where vector y and matrix X are known, vector a is sought. As well known, the heuristic (or symbolic) way (11) y = Xa () X T y = X T Xa () (X T X) 1 X T y = a leads to the same result that least-squares method (LSM): (12) a = (X T X) 1 X T y: In order to investigate a pseudo-solution (12), let denote Gramian [3] matrix X T X = W. It is a square (k; k) matrix. Let be f 1 ; 2 ; : : : ; k g the set of all eigenvalues of W which are numbered so that 1 2 ::: k. Remind that for Gramian matrix W all i are real nonnegative: i 0 8i. Then: (13) det W = 1 2 : : : k ; trace W = k ; cond W = 1 ( k ) 1 ; where cond W = jjwjj jjw 1 jj denote a condition number, jj jj { any reasonable norm. Obviously, 8c : cond (cw) = cond W, where c { an arbitrary constant. Now, let us investigate the expression (12). It has no sense if det W = 0 () k = 0 () cond W = 1: Moreover, there is well-known fact: if cond W is near to innity, i.e. k is near to zero, then the solution y = Xa is worthless for extrapolation or forecasting [2]. The obvious scholar example: the exact polynomial interpolation model, which is the solution of some matrix equation of the form y = Xa, has no sense beyond of interval of interpolation. 2. Problem of Unstability in Linear Modelling

9 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 9 { (14) Dene the measure of stability of the matrix X as stab X = k k trace(x T X) = Obviously, for arbitrary matrix X and for any constant c: k k k (15) 0 stab X 1; 8c : stab (cx) = stab (X); stab X cond (X T X) = and from unequalities k k 1 we obtain: k k (16) 1 stab X cond (X T X) k; or (17) stab X ' [cond (X T X)] 1 The main problem is how to avoid inanity, insignicancy of the model y = Xa beyond the neighbourhood of its construction domain? 2. Problem of Unstability in Linear Modelling

10 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 10 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria 3. Factor Analysis and Stabilization

11 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 11 { What is Factor Analysis? Let reduce the Gramian matrix X T X = W by orthogonal transformation S to such diagonal form S T WS = D, that d ii = i (let us remind that 1 2 k ). Then vectors (columns) of the matrix XS = Z = (z 1 ; z 2 ; : : : ; z k ) are named as factors of X. Note, that 8i : (z i ) = 0, so (18) 8(i; y; c) : hy + c; z i i = hy; z i i: By construction, rst non-zero factors (z 1 ; z 2 ; : : : ; z p ) form an orthogonal basis of the linear enveloping space (19) L = L(x 1 1 ; x 2 2 ; : : : ; x k k ); where p = dim L min(k; n). Thus, an arbitrary linear model ^y(x 1 ; x 2 ; : : : ; x k ) by transformation S may be represented in the form of (20) ^y = y 0 + y 1 z 1 + y 2 z y p z p ; where (21) 8 0 < i p; y i = hy y 0; z i i hz i ; z i i = hy; z ii jz i j 2 = hy; z ii i : 3. Factor Analysis and Stabilization

12 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 12 { Further, we dene for every non-zero factor z i two characteristics: stability stab (z i ) and essentiality essn (y; z i ): (22) of course, (23) stab (z i ) = i trace (W) = i trace (D) = i k = stab (Z) = stab (X) = min i stab (z i ) = stab (z k ): i p ; (24) where (25) essn (y; z i ) = corr 2 (y; z i ) = corr(y; z i ) = cos(y y 0 ; z i ) = hy y 0; z i i jy y 0 jjz i j iy 2 i jy y 0 j 2 ; = hy; z ii jz i j 2 jz i j jy y 0 j = y p i i jy y 0 j : 3. Factor Analysis and Stabilization

13 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 13 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria 3. Factor Analysis and Stabilization. Method of Two Thresholds (MTT), or ( ; ){Method

14 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 14 { By (; ){reduction of a model ^y by method of two thresholds (MTT), or (; ){method, we call such diagonal projection P of the model (20), diag P = (s 1 ; s 2 ; : : : ; s p ) (26) ^y s = y 0 + s 1 y 1 z 1 + s 2 y 2 z s p y p z p ; where 8 i > 0; s i = s i (; ) = 0, if stab (z i ) < or essn (y; z i ) <, and s i = 1 for all other factors. Let's call factor z i as unstable with the threshold, or {unstable, if stab (z i ) <. Obviously, for every > 0 there exists such j 2 N that all {stable factors form the set (z 1 ; z 2 ; : : : ; z j ), and all {unstable factors form the set (z j+1 ; : : : ; z k ). Let's call factor z i as inessential for variable y with the threshold, or {inessential, if essn (y; z i ) <. In these terms, (; ){reduction is the obliteration of all {unstable and {inessential factors. We call (; 0){reduction as {stabilization. 3. Factor Analysis and Stabilization. Method of Two Thresholds (MTT), or ( ; ){Method

15 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 15 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria 4. Stabilization Principle in Linear Modelling

16 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 16 { Let L be dened as (19), i.e. L = L(x 1 1 ; x 2 2 ; : : : ; x k k ) = L(z 1 ; z 2 ; : : : ; z p ); V is arbitrary subspace of L, and P is corresponding to V projection matrix, so that XP is the projection of X to V. We can substitute matrix X by matrix XP and apply all denitions: cond (P T WP), stab (XP) etc. Let XPS = U = (u 1 ; u 2 ; : : : ; u w ) be factors of XP. We dene stability, essentiality and suciency of the subspace V as (27) stab (V ) = min v2v stab (v) = min i2f1::wg stab (u i ) = stab (u w ); (28) essn (y; V ) = min v2v essn (y; v) = min i2f1::wg essn (y; u i ); (29) su(y; V ) = dim XV i=1 essn (y; u i ) = dim XV i=1 corr 2 (y; u i ) = dim 1 XV jy y 0 j 2 i=1 2 hy; ui i : Denition. We call subspace V as: {stable, if stab (V ) > ; {essential, if essn (V ) > ; {sucient, if su (y; V ) > 1. If, or are implicit as xed, we call subspace V simply as stable, essential or sucient. As a rule, implicit values are: = 10 3, = 0 or = 10 4, = 5% or = ju i j 4. Stabilization Principle in Linear Modelling

17 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 17 { Let = fi 1 ; i 2 ; : : : ; i g be any subset of K = f1; 2; : : : ; kg. Denote by V (; X), V (; Z) the linear enveloping space of vector columns x i (x i ), z i, respectively, for all i 2. Hypothesis. The essence of GMDH is the projection of L on such stable and essential subspace V (; X), which is the most sucient. Denition. By general (; ){method, or stabilization principle, we call a search of the solution of the next optimization problem: to nd in some set S(L) 2 L of subspaces of L such {stable and {essential subspace V which is the most sucient: (30) fstab (V ) > g & fessn (y; V ) > g & fsu(y; V ) = maxg & fv 2 S(L) 2 L g; where 2 L is the set of all subspaces of L. Examples: (a) S(L) = fv (; X) : 2 2 K g; (b) S(L) = fv (; Z) : 2 2 K g; (c) S(L) = 2 L. 4. Stabilization Principle in Linear Modelling

18 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 18 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria 5. Active Agent Models. Cycles of Absorption and Reduction

19 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 19 { As a rule, neither (; ){method of two thresholds (MTT) nor general (; ){method gives us the {sucient solution a. So, model y = Xa would be stable but not suciently exact. The next stage is absorption of other variables (or, in economis terms, activities) in the information space (as a rule, in data bases in Internet). This part of inductive modelling is very interesting and complicated. It deals with terms of constructing intelligent agents (CIA) [4], articial intelligence (AI), multiagent systems (MAS) et ctera. But most of these topics are not yet included in the Inductive Modelling Theory. Let give some initial denitions t for the eld of inductive modelling. Active model is the model which are searching all relevant activities (t 1 ; t 2 ; : : : ; t l ) are necessary for its functioning (and statistics of these activities) in the information space (mainly in Internet). Active agent model is the model which are searching in Internet by the instrumentality of intelligent search agents. Synonym: active intelligent model. Active forecasting is the forecasting by the instrumentality of active intelligent model. It may be very interesting to investigate the image of every cycle of absorption and reduction on the two dimensional plane \stability { suciency". At the rst look, this image is some similar to Carnot cycle, but this impression is supercial. 5. Active Agent Models. Cycles of Absorption and Reduction

20 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 20 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria

21 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 21 { Firstly, the method of two thresholds was successfully used for modelling and forecasting of ferromolybdenum and molybdenum prices. Especially interesting was forecast of extremely quick jumping monthly prices in 2004{07. Even the possibility of any forecast under these data was under question [5].

22 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 22 { For monthly moving forecasting of molybdenum prices in , in the period 2004{05 years in Internet were selected l = 9 activities ft 1 ; t 2 ; : : : ; t l g: t 1, t 2 { molybdenum export and import prices, which were calculated from data given in tables on Internet site [6]; t 3 { import prices on unwrought copper, were calculated in the same way from [6]; t 4 ; t 5 ; : : : ; t 9 { world stainless steel prices of 6 dierent grades [7] (see Tab. ). Note, that these 9 activities were selected only after 4-th cycle of absorption and reduction. All previous activities went to the wastepaper basket (or to recycle bin). From l = 9 activities ft 1 ; t 2 ; : : : ; t l g we form a forecasting variable y() and k = 12 input basic variables fx 1 ; x 2 ; : : : ; x k g: y() = t 1 ( + 1); x 1 () = t 1 ( 2); x 2 () = t 1 ( 1); x 3 () = t 1 (); x 4 () = t 2 ( 1); x 5 () = t 2 (); x i () = t i 3 (); i = 6::12:

23 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 23 { Tab. 1: MEPS { World Stainless Steel Product Prices ($ US/tonne). Date Hot Rolled Plate Cold Rolled Coil Drawn Bar Grade 304 Grade 316 Grade 304 Grade 316 Grade 304 Grade 316 Jan Feb Mar Apr : : : : : : : : : : : : : : : : : : : : : Sep Oct Nov Dec

24 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 24 { Figure 1: Graphics of the actual values of molybdenum prices (red line), simulated (yellow line) and forecasted values (blue line)

25 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 25 { Tab. 2: Results of forecasting of Molybdenum price ($ US/kg) Month Price Die- Change Fact Forecast rence Fact Forecast Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb

26 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 26 { Other example is based on medical data from Motol hospital in Prague. It was proposed me by P. Kordik to compare (; ){method with results of [8] which were obtained by means of Group of Adaptive Models Evolution (GAME). Data of the Tab. 3 and Fig. 2 demonstrate the advantage of MTT over GMDH and GAME. It is worthy of note that GAME needs huge training data (some hundreds of observations) [8], when MTT needs about observations. Tab. 3: Comparison of the (; ){method (MTT) with GMDH and GAME [8] on examples of molybdenum price forecasting (row Mo) in 2005{07 and forecast of CO2 in brain [8] (row CO2): RMS error of prediction. Comparisons with Other Methods

27 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 27 { Fig. 2: Graphics of the actual values of molybdenum prices (line fact), and forecast values, computered by (; ){method (line Praha) and GMDH (lines FPE and AR). Comparisons with Other Methods

28 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 28 { 1. Introduction 2. Problem of Unstability in Linear Modelling 3. Factor Analysis and Stabilization Method of Two Thresholds (MTT), or (; ){Method 4. Stabilization Principle in Linear Modelling 5. Active Agent Models Cycles of Absorption and Reduction Comparisons with Other Methods 7. Economic Criteria 7. Economic Criteria

29 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 29 { (31) Let y be a price for some good. We can buy this good month by month, then expenses equal to Z 0 = nx i=1 On the end of the period we can see the least price ymin. If we buy all good by the least price, expences would equal to (32) y i : Z min = ny min : Of course, we are not omniscient. But if we have a forecast by method M, we can use it in some way W for optimal planning of purchase. Then we can compute corresponding expenses Z M; W : (33) Z M; W = nx i=1 p i (M; W )y i ; where p i (M; W) is the advice for buying, formed using forecast method M and way of using W. It gives some types of economic criteria. E.g., relative criterion Z 0 Z M; W (34) : Z 0 Z min for molybdenum price forecast by MTT equals about 68%. Let some plant consumes 20 tons of molybdenum per month and use forecast (Table 2) plus some simple heuristic rule for decision making of its purchase. Then absolute economy of this simulated plant would equal about $ 1,9 mln over period Nov 2005 { Feb 2007 (16 monthes). 7. Economic Criteria

30 ICIM'2008 Yuriy Dzyadyk. Some Fundamental Topics Of Linear Modelling { 30 { References [1] Ivakhnenko A. G., Ivakhnenko G. A.: The Review of Problems Solvable by Algorithms of the Group Method of Data Handling (pdf) [2] Koppa Yu. V., Stepashko V. S.: A Comparison of the Forecasting Properties of Regression Types Models versus GMDH (pdf) (in Russian) [3] Gramian matrix. Jrgen Pedersen Gram. [4] Joseph P. Bigus, Jennifer Bigus. Constructing Intelligent Agents Using Java: Professional Developers Guide. John Wiley & Sons, 2001, second edition. ISBN [5] Private letter of Pavel Kordik. [6] Metals Statistics, U.S. Metals Trade [7] MEPS (International) Ltd. { Independent Steel Industry Analysts, Consultants, Steel Prices, Reports and Publications. World Stainless Steel Product Prices [8] Josef Bouska, Pavel Kordik. Time Series Prediction by means of GMDH Analogues Complexing and GAME Bibliography

GAMINGRE 8/1/ of 7

GAMINGRE 8/1/ of 7 FYE 09/30/92 JULY 92 0.00 254,550.00 0.00 0 0 0 0 0 0 0 0 0 254,550.00 0.00 0.00 0.00 0.00 254,550.00 AUG 10,616,710.31 5,299.95 845,656.83 84,565.68 61,084.86 23,480.82 339,734.73 135,893.89 67,946.95