PREDICTION OF DEFORMED AND ANNEALED MICROSTRUCTURES USING BAYESIAN NEURAL NETWORKS AND GAUSSIAN PROCESSES

Transcription

1 PREDICTION OF DEFORMED AND ANNEALED MICROSTRUCTURES USING BAYESIAN NEURAL NETWORKS AND GAUSSIAN PROCESSES C.A.L. Baier-Jones, T.J. Sabin, D.J.C. MacKay, P.J. Withers Department of Materias Science and Metaurgy, University of Cambridge, Pembroke Street, Cambridge, CB2 3QZ, U.K. Cavendish Laboratory, University of Cambridge, Madingey Road, Cambridge, CB3 OHE, U.K. To be pubished in the proceedings of the Austraasia-Pacific Forum on Inteigent Processing and Manufacturing of Materias, ABSTRACT The forming of metas is important in many manufacturing industries. It has ong been known that microstructure and texture affect the properties of a materia, but to date imited progress has been made in predicting microstructura deveopment during thermomechanica forming due to the compexity of the reationship between microstructure and oca deformation conditions. In this paper we investigate the utiity of non-inear interpoation modes, in particuar Gaussian processes, to mode the deveopment of microstructure during thermomechanica processing of metas. We adopt a Bayesian approach which aows: (1) automatic contro of the compexity of the non-inear mode; (2) cacuation of error bars describing the reiabiity of the mode predictions; (3) automatic determination of the reevance of the various input variabes. Athough this method is not inteigent in that it does not attempt to provide a fundamenta understanding of the underying micromechanica deformation processes, it can ead to empirica reations that predict microstructure as a function of deformation and heat treatments. These can easiy be incorporated into existing Finite Eement forging design toos. Future work wi examine the use of these modes in reverse to guide the definition of deformation processes aimed at deivering the required microstructures. In order to thoroughy train and test a Gaussian process or neura network mode, a arge amount of representative experimenta data is required. Initia experimenta work has focused on an A-1%Mg aoy deformed in non-uniform cod compression foowed by different anneaing treatments to buid up a set of microstructura data brought about by a range of processing conditions. The DEFORM Finite Eement modeing package has been used to cacuate the oca effective strain as a function of position across the sampes. This is correated with measurements of grain areas to construct the data set with which to deveop the mode. THE METALLURGICAL PROBLEM To optimize any component it is necessary to consider not ony the aoy composition but aso its microstructure. For exampe, in the aerospace industry, high-performance nicke aoys may go through 1

2 many hot forging and anneaing stages, at great cost and over a period of severa days, before the desired microstructure is obtained [1]. A reiabe mode for predicting microstructura evoution coud greaty improve manufacturing efficiency. The principa factors infuencing microstructure during thermomechanica processing are recovery, recrystaisation and grain growth. These can be examined on the fundamenta eve of disocation densities and subgrain forms and sizes [2] from which detais of the recrystaisation are theoreticay predictabe [3]. However, such features are difficut and time-consuming to measure and ie beyond the scope of most industria companies. A more feasibe approach is to focus on more accessibe processing parameters such as oca temperature, strain, strain rate etc. and to use numerica modes to reate their infuence on microstructura features such as grain size and extent of recrystaisation [4][5]. BAYESIAN PROBABILISTIC MODELLING OF DATA In many cases, a prediction probem can be posed as an interpoation probem. The probem addressed in this paper is the prediction of the mean grain size ( ) in a region of a cod deformed and anneaed sampe as a function of oca strain ( ), anneaing temperature ( ) and anneaing time ( ). One approach to a probem of this type is to cacuate a physica reationship between the dependent variabe and the measured variabes from fundamenta scientific principes [6]. Such a reationship wi usuay be governed by a number of parameters which have to be determined empiricay [3][4]. This semi-empirica approach wi generay ony be reiabe when appied to simpe probems: Many rea prediction probems wi be too compex for such an approach to yied reaistic or usefu resuts [7]. For many probems, then, a more fexibe approach to prediction is required. The purey empirica method makes use of a set of training data to characterize the reationship between a number of inputs, (such as, and ) and the corresponding output,, which we are interested in predicting (such as ). (We sha consider predictions of ony one variabe.) The training data set,, consists of a set of inputs, (! ), and the corresponding outputs, ". We are interested in interpoating these data using a mode in order to be abe to make predictions of at vaues of which are not present in the training data. Generay, the measured vaues of wi contain noise, #, so mode s prediction, $&% (', is reated to the target output by )*$&% (',+ #. A common approach to this noisy interpoation probem is to parameterize the function, $&% -/.0', where. is a set of parameters which are determined from the training data using methods such as east-squares minimisation of some cost function, 1. This is the approach taken by feed-forward muti-ayer perceptron neura networks, which provide a suitabe framework for evauating a noninear interpoating function (interpoant) of a set of training data [8]. The parameters in these networks are represented by a set of weights: Training the neura network is the process of cacuating the optimum weights by minimizing the cost function, 12%.0'. Bayesian probabiistic data modeing is a robust and powerfu approach to prediction probems and can be readiy incorporated into the neura network approach [8]. Rather than giving a singe optimum prediction, Bayesian methods provide a probabiity distribution over the predicted vaue. This is often very important as it can be used to produce a characteristic error in the predictions which represents the uncertainty arising from interpoating noisy data. The probabiity of the data given the weights, the ikeihood, can be written 34%576.8:9('<;=>@?,ACBD. 9 is a so-caed hyperparameter which parameterizes the probabiity distribution and is reated to the noise variance, #, in the target outputs. The maximum ikeihood approach to training a neura network (minimizing 1 ) is therefore equivaent to maximizing 3E%FG6.HI9J'. However, we normay incude an expicit prior on the weights to specify our beief concerning the distribution of the weights in the absence of any data. This can be written 34%. 6 K ', where K is another hyperparameter. This prior is often used to give preference to smoother interpoating functions rather than rapidy varying ones which may over-fit the training data. This prior is particuary important when trying to mode sparse data sets as it wi generay improve the reiabiity of predictions. We then appy Bayes theorem to the prior and the ikeihood 2

3 to give the posterior probabiity distribution of the weights given the data 34%. 6L K :9(' 3E%FG6.HI9J' 3E%. 3E%F ' 6 K ' (1) It is this quantity which we shoud maximize when training the neura network [8]. (We can ignore the denominator, 34%5 ', when making predictions with a singe mode and data set,.) The Bayesian approach to prediction prescribes that we marginaize (i.e. sum) over uncertain parameters. We shoud, therefore, ideay integrate over a vaues of the weights rather than optimize them. We are interested in predicting a new vaue, -M, given its corresponding input, -M, and the set of training data,. In terms of probabiity distributions we are interested in finding 34% -M 6 -M K :9('. This is obtained by integrating over a possibe vaues of the weights: 34% -M 6 JM K I9J' ONP3E%Q -M 6 -M.8 K :9J' 34%. 6L K I9J'SRT. (2) The maximum of 34% -M 6 -M K I9J' yieds the most probabe prediction for -M. The integration can be performed by Monte Caro methods or by making simpifying assumptions about the form of 34%. 6L K :9('. This atter approach is often equivaent to making predictions at the optimum vaues of the weights found by maximizing 3E%. 6 K :9(' in equation 1. Note that we are reay interested in 34%QUJM 6 -M ' rather than 34% -M 6 -M K :9J'. This is obtained from equation 2 by aso integrating over the hyperparameters K and 9 (athough it is often adequate to optimize K and 9 ). These hyperparameters are important because they contro the compexity of the mode. They are distinct from parameters (i.e. the network weights) which parameterize the input output mapping. One of the advantages of the Bayesian approach to data modeing is that it automaticay embodies compexity contro by means of these hyperparameters [9]. GAUSSIAN PROCESSES FOR MODELLING From the Bayesian perspective, we are interested ony in 34% -M 6 -M ' : we are not interested in the network weights themseves. Given that we shoud integrate over a weights, a preferabe mode is one which does not have such weights at a. The Gaussian process can be considered as a neura network in which we have integrated over a possibe vaues of the weights. The Gaussian process approach to the prediction probem assumes that the joint probabiity distribution of any output vaues, ", is an -dimensiona Gaussian [10][11][12] 34% "<6 V WH' For exampe, this appies to the Y X =>@?Z@[ \ X % " [^]_'F`,a A % " [^]b'5c (3) vaues in the training data set,, defined on the previous page. This distribution is competey determined by the mean, ], and the covariance matrix, a. The eements of a are given by the covariance function, d-egf!hdi% e f W8', where e and f are any two inputs and W is a set of hyperparameters. The form of d is important and wi be discussed shorty. Let " -M be the vector composed of the output vaues in the training data, ", and the point we wish to predict, -M. As equation 3 can appied to any vaue of, it aso describes the probabiity distribution of "-M, which is an + 1-dimensiona Gaussian 34% "-M 6 -M WH'. The predictive probabiity distribution for -M is just 3E%Q"-M 6 -M WH' j 34% "6 V W8', which is the one-dimensiona Gaussian, 34%QUJM 6 -M WH'. The mean and standard deviation of this Gaussian distribution can be evauated anayticay in terms of the new input vaue, -M, the training data,, and the hyperparameters, W. Note that this Gaussian refers to the probabiity distribution over the 3

4 s s s u y k n r n r ˆ z k k u v p y q q z x k ƒ p n ƒ y n x o { z y n n o x y o n m k n q r m x q k r x z k k p q n r n o k k o o n k r } ~ t u v k w x k p y x z n { n q r k m n o p q n r u o k r z k y z z x Š n k n q r q } ~ ˆ n Š x o x y y q y k y o q r p y x z n { n q r u v x k r q } ~ n o p y x z n { x z Š k ƒ x q m x n r x y p q ƒ k x z r { n q r } n r x y p q ƒ k r ~ Figure 1: Schematic description of how a Gaussian process interpoates a function to make predictions. We consider here the simpe case of the parameter of interest ( ) having ony one dependent training data points, the mode provides a predictive Gaussian proba- variabe (Œ ). Given a set of biity distribution over at a new point -M. The mean and standard deviation of this Gaussian are evauated from -M, the training data points and a set of hyperparameters. These hyperparameters (which are evauated using the training data) contro the smoothness of the interpoant. Note that we do not assume that the function %FŒ ' is a sequence of Gaussians. predicted output: we do not assume that the interpoating function is a Gaussian. Figure 1 summarizes schematicay the prediction process with a Gaussian process with Ž. The form of the covariance function, d, is specified by our assumptions regarding the form of the interpoant. We wi generay be interested in producing a smooth interpoant. This can be achieved by incuding a term in d which gives a arger positive contribution to d-egf the coser together two inputs are: A arger vaue of d-egf means that e and Sf are more cosey correated, i.e. they are more ikey to have simiar vaues. The degree of correation achieved by a given proximity of the inputs (i.e. the smoothness of the interpoant) is dictated by the ength scae hyperparameters which parameterize this term in d. There is one of these hyperparameters for each input dimension. In addition to controing the smoothness of the interpoant, the reative sizes of the ength scaes are a measure of the reevance of each input dimension in determining the output. Thus we coud assess (for exampe) whether oca strain is more reevant than anneaing temperature in determining recrystaized grain size. The noise in the data is represented in the covariance function by another term with another hyperparameter. These hyperparmeters are evauated from the training data by maximizing 3E% W 6 ), the probabiity of the hyperparameters given the training data. We woud typicay incude expicit priors on the hyperparameters to express our prior knowedge of the noise in the data and the smoothness of the underying function. Once determined, we can evauate 34% -M 6 -M WH' to give predictions (with errors) at any new input vaue, -M. Figure 2 shows the appication of a Gaussian process to the interpoation of data drawn from a noisy sine function with increasing amounts of data. PREDICTING MICROSTRUCTURE We have appied a Gaussian process to the probem of predicting the area of recrystaized grains in 4

5 Figure 2: A Gaussian process interpoation of noisy data drawn from a sine function. The noise is Gaussian with standard deviation = 0.1. The four pots show the resuts of using a Gaussian process to interpoate data sets with Ž data points. The soid ine is the interpoated function; the dashed ines are the corresponding XTX error bars. The accuracy with which the interpoant approximates a sine function improves as more X data are added to the training data set. In regions more distant from the data the interpoant is ess-we confined, and this is refected by the arger error bars predicted by the mode. This is particuary true outside the imits of the data sets where the mode is attempting to extrapoate. Note that the particuar covariance function used here makes the interpoant tend towards zero in regions we away from the data. a deformed and subsequenty anneaed A-1%Mg aoy as a function of oca strain ( ), anneaing temperature ( ) and anneaing time ( ). Our training data were obtained from the pane strain compression of two workpieces (20% and 40% size reduction) which were sectioned to produce a tota of 20 sampes [13]. These were then anneaed at 325 C, 350 C or 375 C for 2, 5, 10, 30 or 60 mins. The DEFORM Finite Eement package was used to cacuate the oca strain at different regions in these sampes. This yieded a set of 57 measurements of, and to act as the inputs in the training data set. The corresponding outputs the mean grain areas, were evauated by sectioning the sampes and measuring areas using the Kontron Eektronik KS400 package. This data set was then used to train (infer the most probabe hyperparameters of) a Gaussian process mode. The ength scae hyperparameters indicated that the three inputs were of roughy equa reevance, as expected. and whist hoding the The mode was then used to predict the dependence of on each of, other two constant. These predictions are shown in Figure 3b d. As a test of the quaity of the mode we trained it on 3/4 of the data and then used it to predict the Q vaues of the remaining 1/4 of the data on which it was not trained. Figure 3a shows predicted area vs. measured area for this test data. A second Gaussian process mode was used to predict the extent of recrystaisation as a function of the same input parameters (,, ), and the resuts combined with predicted grain areas in Figure 4. DISCUSSION The resuts presented in this paper demonstrate the feasibiity of using a Gaussian process mode to predict microstructura deveopment. Figure 3a demonstrates that the mode has generaized we, i.e. 5

6 Figure 3: Gaussian process prediction of recrystaized grain area in an auminium aoy. A errors are X errors. (a) The accuracy of the Gaussian process mode was assessed by training it on 3/4 of the data and using the mode to predict the remaining 1/4. The over-potted ine is the $ = Œ ine, athough no mode can make exact predictions due to the noise in the measured Q vaues of. Because covers a arge dynamic range, the mode was deveoped using T. In (b) (d) the mode was trained on the fu data set and used to predict Q as a function of each of the three input parameters with the other two hed constant. When not being varied, the inputs were hed constant at: P T št C; R œ Tš mins; Hœš. The crosses in (d) are points from the training set: It is important to reaise that the entire training set is used to make the predictions and not just the data points shown. has identified underying trends in the training data. These trends are shown expicity in Figures 3b d, and are in broad agreement with those identified by others [1]. An exception to this is the sight fa-off in at extended anneaing times. However, the uncertainty predicted by the mode is reativey arge in this region due to the sparseness of the training data at high and the mode is not inconsistent with a eveing-off at high temperatures. A more rigorous test of the mode s capabiities is to use it to predict microstructure at different processing conditions, e.g. with different deformation geometries. Q The average size of the error bars on the predictions in Figure 3a is ž%q Ÿ T ' š, which is equivaent to an error in of 40%. These error bars are cacuated by the mode and X represent the uncertainty in its predictions. The sources of this uncertainty which wi be discussed shorty. In contrast to these error bars, we can measure the actua scatter (RMS error) of the predicted vaues of the grain areas about their measured vaues: this is That these two vaues are simiar can be seen graphicay from the fact that most of the error bars on the predictions overap with the $ = Œ ine in Figure 3a. This demonstrates that the errors predicted by the mode are commensurate with the true scatter of its predictions about their measured vaues. A mode which predicted inappropriatey sized error bars woud be of imited practica vaue, as woud one which predicted no errors at a. 6

7 X Figure 4: A 40% reduced sampe anneaed for 2 mins at 350 C. (a) The variation in microstructure across haf a section; (b) schematic iustration of the same variation in microstructure deduced by combining the resuts from a Gaussian process mode used to predict grain size variation and a second Gaussian process mode used to predict extent of recrystaisation. The contours in (b) show 0%, 50% and 90% recrystaisation as predicted by the second Gaussian process mode. The uncertainty in the mode s predictions is due argey to noise in the training data: the microstructura input data were noisy and sparse; the initia grain sizes were arge (diameter mm), making for inhomogeneous nuceation; in some cases arge recrystaised grain sizes meant that the number of grains used to evauate Q was fewer than ten; difficuties in accuratey identifying recrystaized grains eads to biased estimations of Q. Despite these deficiencies, the Gaussian process mode appears to provide good simuations (Figure 3b d) which do not overfit this noisy data. This is one of the advantages of the Bayesian approach: It trades off the compexity of the mode with obtaining a good fit of the training data. Smaer uncertainties in predictions can be obtained through use of a arger and more accurate data set. ACKNOWLEDGEMENTS The authors are gratefu to the EPSRC, DERA and INCO Aoys Limited for financia support and to Mark Gibbs for use of his Gaussian process software (obtainabe from REFERENCES 1. D. Lambert, INCO Aoys Ltd, Hereford, U.K. Private communications with T.J. Sabin, G.I. Rosen, D. Juu Jensen, D.A. Hughes, N. Hansen, Acta Meta., 43(7), 1996, p T. Furu, H.R. Sherciff, C.M. Sears, M.F. Ashby, Mat. Sci. Forum, , 1996, p J. Kusiak, M. Pietrzyk, J.-L. Chenot, ISIJ Int., 34(9), 1994, p P.L. Orsetti Rossi, C.M. Sears, Mat. Sci. Forum, , 1996, p M. Avrami, J. Chem. Phys., 7, 1939, p F.J. Humphreys, Mater. Sci. Techno., 8, 1992, p D.J.C. MacKay, Network: Computation in Neura Systems, 6, 1995, p D.J.C. MacKay, Neura Computation, 4, 1992, p N. Cressie, Statistics for Spatia Data, Wiey, Chichester, C.K.I. Wiiams, C.E. Rasmussen, in Advances in Neura Information Processing Systems 8 (D.S. Touretzky, M.C. Mozer, M.E. Hassemo, eds.), MIT Press, Boston, M.N. Gibbs, D.J.C. MacKay, in preparation (see T.J. Sabin, C.A.L. Baier-Jones, S.M. Roberts, D.J.C. MacKay, P.J. Withers, to be presented at THERMEC 97 (Internationa Conference on Thermomechanica Processing), Juy