CONCRETE STRENGTH ESTIMATION USING RELEVANCE VECTOR MACHINES

Transcription

1 Journal of Materals Scence and Engneerng wth Advanced Technology Volume, Number 1, 010, Pages CONCRETE STRENGTH ESTIMATION USING RELEVANCE VECTOR MACHINES JALE TEZCAN, QIANG CHENG and ARIF CEKIC Department of Cvl and Envronmental Engneerng Southern Illnos Unversty 130 Lncoln Dr., 6901, Carbondale, IL Unted States e-mal: Department of Computer Scence Southern Illnos Unversty Carbondale 1000 Faner Dr., 6901, Carbondale, IL Unted States Professonal Servce Industres, Inc Helm Street Plymouth, Mchgan Unted States Abstract Concrete s a hghly versatle structural materal that has been used for many centures. Recently, there has been a marked ncrease n the use of supplementary cementtous materals and chemcal admxtures. Ths ncrease, whle facltatng the adjustment of concrete propertes for specfc uses, has also complcated the task of estmatng the relevant performance measures from the mx ngredents. For structural engneerng purposes, the most relevant parameter s the 8-day compressve strength. An deal strength estmaton method should not only allow the use of dfferent sets of predctve varables, but Keywords and phrases: concrete compressve strength, mx desgn, relevance vector machnes. Receved May 6, Scentfc Advances Publshers

2 6 JALE TEZCAN et al. also account for predcton uncertanty. Ths paper proposes a probablstc approach for the estmaton of the 8-day compressve strength of concrete, based on the state-of the-art relevance vector machne (RVM). An RVM belongs to the class of sparse kernel classfers, whch are powerful tools n classfcaton and regresson. Recently, the support vector machne (SVM), a sparse kernel model wth a functonal form dentcal to that of the RVM, has already proved successful n modellng concrete behavor. The RVM-based approach proposed n ths paper offers several advantages over the ones based on SVM. Frst, by usng a probablstc kernel, an RVM provdes nformaton about predcton uncertanty. Second, compared to an SVM, an RVM uses fewer kernel functons for comparable generalzaton performance, provdng a sparser representaton. Thrd, the RVM model parameters are automatcally determned from the tranng set, unlke the SVM algorthm, where the selecton of model parameters typcally nvolves cross-valdaton. A demonstratve applcaton comparng the two approaches s presented. The results from ths study confrm the advantages of the proposed method and demonstrate ts effectveness. 1. Introducton Concrete s one of the oldest constructon materals. In ts smplest form, concrete s a mxture of cement, water, and aggregates. Wth the ntroducton of supplementary cementtous materals and chemcal admxtures, concrete has undergone a sgnfcant transformaton over tme. Due to ts hgh versatlty and adaptablty, concrete s one of the most commonly used structural materals n today s constructon ndustry. Compressve strength of concrete s the most relevant performance measure for structural desgn purposes. Mechancal propertes such as the flexural strength, tensle strength, and modulus of elastcty can be estmated from the compressve strength. In general practce, characterzaton and acceptance of concrete s based on the results of the compressve strength tests performed on standard cylndrcal specmens. The compressve strength s determned by dvdng the falure load by the cross sectonal area resstng the load, assumng unform stress dstrbuton. The average of at least two measurements from cylnders wth a dameter of 6 nches and a heght of 1 nches, made from the same mx s reported as a sngle test result (ASTM_C_39 []). To satsfy

3 CONCRETE STRENGTH ESTIMATION USING 63 the strength requrements of a partcular desgn project, the average of three consecutve tests should equal or exceed the specfed strength wth an acceptable confdence level (ACI_318 [1]). Assumng that the samples are prepared and tested properly, the compressve strength s manly determned by the mx desgn, and n partcular, the water-to-cementtous materals rato. Mx proportons to acheve certan strength wth reasonable confdence can be determned by repeated trals. However, ths approach s nconvenent, tme consumng, and costly. The number of tral mxtures can be mnmzed wth the help of predctve models. Recently, the use of sparse kernel classfers n regresson problems has proved very successful. One such classfer, support vector machne (SVM) has already been used n modellng the compressve strength (Gupta [4]), elastc modulus (Yan and Sh [1]), and fre damage n concrete specmens (Chen et al. [3]). In ths paper, we propose the use of the relevance vector machne (RVM) for estmaton of the compressve strength of concrete from the mx desgn, and demonstrate an example applcaton comparng the two methods. The proposed approach, n addton, to provdng nformaton about predcton uncertanty, has computatonal advantages over the SVM-based methods. The remander of ths paper s organzed as follows. Secton provdes a bref overvew of the predctve models proposed by varous researchers for concrete strength estmaton from the mx ngredents. Secton 3 descrbes the SVM and RVM regresson algorthms and provdes a flowchart of the proposed procedure. Secton 4 demonstrates an example applcaton comparng the SVM-to RVM-based approaches. Secton 5 summarzes the fndngs of ths study.. An Overvew of the Predctve Models for Concrete Strength The nverse relatonshp between the compressve strength and the water-to-cement rato has been recognzed snce 1890s (Nevlle [13], Sear

4 64 JALE TEZCAN et al. et al. [14]). Tradtonal emprcal equatons represent a specfc set of mx ngredents and condtons. Modern concrete typcally ncludes supplementary materals and addtves such as fly ash, slca fume, furnace slag, ar-entranng agents, and water reducers. The fact that concrete strength s not only a functon of mx ngredents, but also the envronmental factors complcates the development of a model for concrete strength. The deal method for strength estmaton should not only allow the use of dfferent sets of predctve varables, but also account for predcton uncertanty. In ths regard, nonparametrc regresson algorthms show tremendous potental because of ther ablty to model complex relatonshps wthout an assumed functonal form. Many researchers developed nonparametrc regresson models for concrete strength by usng dfferent supervsed machne learnng archtectures. The potental of artfcal neural networks (ANNs) as concrete strength estmaton tools has been nvestgated n many studes (La and Serra [10], Yeh [], Hong-Guang and J-Zong [5], Km and Km [8], Km et al. [7], Fazel Zarand et al. [4]). Genetc algorthms were also used (Lm et al. [11], Yeh [3]). Whle ANNs and genetc algorthms are powerful regresson and predcton tools, they are prone to performance degradaton wth nose and data outlers. Whle, ths problem can be somewhat allevated by adjustng the model archtecture and relevant parameters, the exstence of local mnma or nsuffcent capablty for adaptng to nosy data lmts the level of accuracy that the tradtonal machne learnng algorthms can offer. The support vector machne (SVM), a relatvely new development, offers ncreased accuracy by always locatng the global optmum (Vapnk [0]). Usng kernel functons, an SVM transforms the tranng data nto a hgh dmensonal feature space, where complex nonlnear relatonshps can be modelled usng lnear algorthms. A subgroup of the tranng set s selected as support vectors, and a predctve model s formed by usng the selected set. The use of support vectors not only removes the requrement of selectng the model sze (the number of the hdden neurons n the ANNs), but also yelds a sparse representaton.

5 CONCRETE STRENGTH ESTIMATION USING 65 Although the SVM s a state-of-the-art technque, t has a number of lmtatons (Tppng [19]). Regardng the concrete strength estmaton problem, the bggest shortcomng s that an SVM only predcts the expected (mean or medan) values of the functon and does not provde any nformaton regardng the confdence levels. It s mportant to recognze that, whle mx desgn s the man factor controllng the compressve strength, the fabrcaton, handlng, curng, and testng procedures all contrbute to the test result and devaton among the test results s expected. The knowledge of confdence levels, or equvalently an estmate of the populaton varance s very useful. In addton, the accuracy of the SVM depends on optmal selecton of the model parameters, typcally nvolvng a cross valdaton procedure over a wde range of possble values. 3. Algorthm 3.1. Problem defnton Gven a set of data ponts {( x 1, y1 ), ( x, y ), K, ( x N, y N )} such that d R d x and y R, estmate the unknown functon f : R R descrbng the nput-output mappng. 3.. Support vector machne (SVM) regresson An SVM transforms the tranng data nto a hgh dmensonal feature space and locates a hyperplane, where the output can be wrtten as a lnear combnaton of the weghted nput. Usng the kernel representaton (Smola and Schölkopf [17]), the unknown functon can be wrtten as: N f ( x) = w K( x, x ) + w0, = 1 (1) where w and w 0 represent the model weghts and the bas term, respectvely, and K ( x, x ) s a sutable kernel functon satsfyng a set of

6 66 JALE TEZCAN et al. mathematcal requrements known as Mercer s condtons (Tppng [18]). Radal bass functon (rbf) s a commonly used kernel functon: K ( x, x ) exp( γ x x ), γ > 0. () j = j The classcal SVM algorthm gnores errors below a user-defned threshold () ε through the use of the ε- nsenstve loss functon (Huber [6]) y f ( x) ε, f y f ( x) > ε, y f ( x) ε = (3) 0, otherwse, and solves the mnmzaton problem: mn w, δ, δ 1 w T w N + C = 1 ( δ + δ ), subject to y f ( ) δ + ε, (4) x f ( x ) y δ + ε, δ, δ 0, = 1, K, N. The parameter C n Equaton (4) s a user-defned parameter controllng the trade-off between the model complexty and tranng errors. The slack varables δ, δ, shown n Fgure 1, are used to adjust the wdth of the tolerance zone as needed.

7 CONCRETE STRENGTH ESTIMATION USING 67 Fgure 1. Slack varables and the ε- nsenstve zone. Durng regresson, some of the weght coeffcents are set to zero, effectvely removng the correspondng tranng samples from the model. The predctve model s then expressed n terms of the remanng samples (the so-called support vectors). The number of support vectors depends on the specfcs of the tranng set and the model parameters. If the number of support vectors s only a small fracton of the number of tranng examples, a hghly sparse kernel representaton s acheved. One mportant lmtaton of the SVM s the lack of probablstc outputs, only the expected (mean or medan) values of the output are provded. In addton, the accuracy of the SVM algorthm depends on proper selecton of the model parameters (C and ε ) n addton to a kernel type and ts related parameters. For example, assumng an rbf kernel, there are three parameters ( C, ε, and γ ) to be specfed by the user. The trade-off parameter C, has no ntutve meanng (Shawe-Taylor and Crstann [15]). The optmal of value for the parameter C can be determned over a wde range of values. A process called cross-valdaton s often used to assess the model performance. Optmal selecton of the nsenstvty parameter ε requres the knowledge of the nose varance, whch s generally unknown. Asymptotcally, unbased estmators for ε have been proposed (Smola et al. [16], Kwok [9]), but they do not

8 68 JALE TEZCAN et al. consder sample sze. The scale parameter ( γ ) can be set ether by usng several trals wth dfferent values, or through a cross-valdaton procedure along wth C and ε. Consderng that a three-parameter grd search wth only 10 ponts per grd requres 10 3 evaluatons, the computatonal cost of ensurng global optmalty may become hgh Relevance vector machne (RVM) regresson A relevance vector machne (Tppng [18]) s a probablstc sparse kernel model that uses the same functonal form as the SVM. Assume that the ( x, y) pars n the tranng data are related through the same f ( x) gven n Equaton (1) wth some added nose: y = f ( x ) + n, = 1, K, N, (5) where n are ndependent samples from zero-mean Gaussan nose wth varance σ. Therefore, the target values are assumed to have a Gaussan dstrbuton wth mean f ( x ) and varance σ : p( y x ) N ( y f ( x ), σ ). (6) = Assumng each observaton s ndependent, the lkelhood of the entre dataset can be wrtten as N 1 y Φw p( y w, σ ) = ( πσ ) e σ, (7) T where y = ( y1, K, yn ), w = ( w0, K, wn ), and Φ N N + 1 s a matrx Φnm = K ( xn, xm 1 ) and Φ n 1 = 1. To prevent overfttng, the weghts are descrbed by an automatc relevance determnaton Gaussan pror (Neal [1]): N 1 p( w α) = p( w α ) = N w 0,. (8) α = 0 N = 0 T

9 CONCRETE STRENGTH ESTIMATION USING 69 The use of ndvdual α for each w s what makes the RVM computatonally effcent. To complete the probablstc descrpton of the weghts, the nverse varances α 1, K, αn and the nose varance σ need to be defned. For consstency, let β = 1 σ. Snce both α and β are scale parameters, they can be represented by a gamma dstrbuton (Tppng [19]): p( α a, b) = Γ( α a, b) = a a 1 ba b α e Γ( a), (9) a 1 t 0 wth Γ ( a) = t e dt. The pror dstrbuton over α s wrtten as N p( α) = p( α a, b). = 0 (10) Smlarly, β s descrbed by: p( β c, d) = Γ( β c, d) = c c 1 d β e Γ() c dc. (11) The overall pror on w can be evaluated by usng margnalzaton over α : N 0 = 0 p( w a, b) = p( w α ) p( α a, b) dα, (1) or, equvalently, N 0 = 0 p( w a, b) = N ( w 0, 1 α ) Γ( α a, b) dα. (13) Because gamma dstrbuton s the conjugate pror to normal dstrbuton, the above ntegral can be evaluated explctly. The resultng dstrbuton s a student s t-dstrbuton (Tppng [19]), whch s hghly peaked at the orgn and yelds a sparse representaton for β. The hyperparameters α 0, K, α N and β are computed teratvely. In ths Bayesan estmaton framework, after computng the values of α 0, K, αn and β, we obtan

10 70 JALE TEZCAN et al. the nose varance σ. The uncertanty from the nput s automatcally propagated through the herarches of the Bayesan model, defnng the uncertanty of the output. In other words, we can automatcally derve the confdence ntervals regardng the estmated output values. Fgure shows an outlne of the RVM procedure for estmatng concrete strength. The parameters of the tranng data set can be modfed to ft other sets of ngredents. Fgure. Algorthm for concrete strength estmaton.

11 CONCRETE STRENGTH ESTIMATION USING Numercal Results The RVM algorthm descrbed n the prevous secton was traned and tested by usng publshed data (Lm et al. [11]). Ther data contans the slump and compressve strength measurements from a total of 189 sets of mxtures, 108 of whch were proportoned to have a compressve strength between 40 and 80 MPa. The set of 108 samples s used n ths study. Table 1 lsts the mx parameters and ther ranges. The lst of ngredents for ndvdual mxes can be obtaned from the paper by Lm et al. [11]. Table 1. Mx proporton ranges Parameter Defnton Range p1 Water-to-bnder rato p Water (kg/m 3 ) p3 Fne -to-all aggregate weght rato p4 Super plastczer (kg/m 3 ) p5 Fly ash-to-bnder rato p6 Ar entranng agent (kg/m 3 ) Ten of the 108 samples were set asde for testng, and the remanng 98 were used n tranng the regresson models. Therefore, the tranng set s composed of 98 nput-output pars {( x y ), ( x, y ), K, ( x, )}, where [ p, p, p, p, p p ] x , 6 1, 1 98 y98 = s a vector consstng of the sx parameters defned n Table 1, and y s the correspondng 8-day compressve strength Tranng of the SVM model The SVM model was traned by usng the radal bass kernel functon. The learnng parameters ( ε, C ) and the kernel parameter ( γ ) were determned through a fve-fold cross valdaton procedure by usng a grd search. Optmal parameters for the SVM were found to be ε = 0.018, C = 6.4, and γ = 0.18.

12 7 JALE TEZCAN et al. At the end of tranng, the SVM selected 75 of the 98 tranng samples as support vectors. Fgure 3 shows the tranng resduals n terms of the number of standard errors. The dots n the fgure show the tranng samples, and the crcles represent the support vectors. The root-meansquare-error and the correlaton coeffcent for tranng were RMSE = and r = 0.999, respectvely. Fgure 3. Tranng resduals and support vectors n SVM algorthm. 4.. Tranng of the RVM model The hyperparameters of the RVM algorthm are determned automatcally, but the kernel functon and ts parameters need to be specfed. Radal bass kernel was selected as n the SVM case. The recommended startng value for γ s (1/N), where N s the number of tranng samples. Consderng that there are 98 samples n the tranng set, the kernel parameter was set to γ = Ths choce of γ proved to be a sutable choce n terms of small predcton errors, and t was observed that the performance of the RVM was pretty stable around ths value.

13 CONCRETE STRENGTH ESTIMATION USING 73 Fgure 4 shows the tranng resduals correspondng to the tranng samples n addton to the relevance vectors. Only 7 of the 98 samples were selected as relevance vectors. The root-mean-square-error and the correlaton coeffcent for tranng were RMSE = and r = , respectvely. Note that the tranng resduals were reported n terms of the number of standard devatons, not standard errors as n the SVM case. Ths s because RVM s a probablstc kernel model, and t s able to predct the standard devaton from the tranng set. Fgure 4. Tranng resduals and relevance vectors n RVM algorthm Comparson of the test results The SVM and RVM models were tested usng the 10 samples that were excluded from the tranng set. Fgure 5 shows the predctons from the two models n addton to the measured compressve strengths of the test samples. As the fgure shows, the predctons from both models are very close to the measurements. The root-mean-square-error (RMSE) and the correlaton coeffcent (r) from the SVM algorthm for testng are RMSE = and r = , respectvely. For the RVM model, RMSE = and r =

14 74 JALE TEZCAN et al. Fgure 5. Comparson of SVM and RVM test results. 5. Concluson Ths paper proposes an RVM-based regresson algorthm for predctng the 8-day compressve strength of concrete n a probablstc framework. An RVM s a probablstc kernel model that uses the same functonal form as the SVM. The key dfference between the two s the ablty of the RVM to predct the populaton varance n addton to the expected compressve strength. The RVM was traned and tested usng publshed data, and ts performance was compared to the SVM. A radal bass functon was selected as the kernel functon n both models. The parameters of the SVM were determned by usng fve-fold cross valdaton wth grd search. The tranng resduals from the SVM were calculated n terms of the number of standard errors and plotted along wth the support vectors. The tranng resduals from the RVM were calculated n terms of the number of standard devatons and plotted along wth the relevance vectors. The RVM model employed dramatcally fewer kernel functons, resultng n a much sparser model. Only 7 out of

15 CONCRETE STRENGTH ESTIMATION USING samples were kept by the RVM algorthm, as opposed to 75 kept by the SVM. The two models were tested usng the 10 samples that were set asde before tranng. The performance of the two models was evaluated usng the total root-mean-square-errors, and the correlaton coeffcents from tranng and testng. It was observed that the generalzaton performance of the two models s comparable. The followng conclusons can be drawn from the results of ths study: (1) The proposed algorthm successfully predcts the 8-day compressve strength of concrete from the mx ngredents. () The proposed RVM-based model provdes the populaton varance n addton to the compressve strength, whereas the SVM only predcts the compressve strength. (3) The RVM uses far fewer kernel functons compared to the SVM model for comparable generalzaton performance. (4) The RVM model parameters are automatcally determned from the tranng set, unlke the SVM algorthm, where the selecton of model parameters typcally nvolves cross-valdaton. References [1] ACI_318, Buldng Code Requrements for Structural Concrete and Commentary Detrot, Amercan Concrete Insttute 008. [] ASTM_C_39, Standard Test Method for Compressve Strength of Cylndrcal Concrete Specmens, ASTM Internatonal 001. [3] B. T. Chen and T. P. Chang et al., Estmaton of exposed temperature for fredamaged concrete usng support vector machne, Computatonal Materals Scence 44(3) (009), [4] S. M. Gupta, Support vector machnes based modellng of concrete strength, Engneerng and Technology 36 (007), [5] N. Hong-Guang and W. J-Zong, Predcton of compressve strength of concrete by neural networks, Cement and Concrete Research 30(8) (000), [6] P. J. Huber, Robust estmaton of a locaton parameter, The Annals of Mathematcal Statstcs (1964), [7] D. K. Km and J. J. Lee et al., Applcaton of probablstc neural networks for predcton of concrete strength, Journal of Materals n Cvl Engneerng 17 (005), 353.

16 76 JALE TEZCAN et al. [8] J. I. Km and D. K. Km, Applcaton of neural networks for estmaton of concrete strength, KSCE Journal of Cvl Engneerng 6(4) (00), [9] J. Kwok, Lnear dependency between and the nput nose n-support vector regresson, Artfcal Neural Networks-ICANN 001 (001), [10] S. La and M. Serra, Concrete strength predcton by means of neural network, Constructon and Buldng Materals 11() (1997), [11] C. H. Lm and Y. S. Yoon et al., Genetc algorthm n mx proportonng of hghperformance concrete, Cement and Concrete Research 34(3) (004), [1] R. M. Neal, Bayesan Learnng for Neural Networks, Sprnger Verlag, New York, [13] A. M. Nevlle, Propertes of Concrete, Longman Scentfc & Techncal, England, [14] L. K. A. Sear and J. Dews et al., Abrams law, ar and hgh water-to-cement ratos, Constructon and Buldng Materals 10(3) (1996), 1-6. [15] J. Shawe-Taylor and N. Crstann, Kernel Methods for Pattern Analyss, Cambrdge Unv. Press, 004. [16] A. Smola and N. Murata et al., Asymptotcally optmal choce of -loss for support vector machnes [17] A. J. Smola and B. Schölkopf, A tutoral on support vector regresson, Statstcs and Computng 14(3) (004), [18] M. Tppng, The Relevance Vector Machne, Advances n Neural Informaton Processng Systems MIT Press, 000. [19] M. E. Tppng, Sparse Bayesan learnng and the relevance vector machne, Journal of Machne Learnng Research 1 (001), [0] V. N. Vapnk, The Nature of Statstcal Learnng Theory, Sprnger Verlag, New York, 000. [1] K. Yan and C. Sh, Predcton of elastc modulus of normal and hgh strength concrete by support vector machne, Constructon and Buldng Materals 4(8) (010), [] I. C. Yeh, Modellng of strength of hgh-performance concrete usng artfcal neural networks, Cement and Concrete Research 8(1) (1998), [3] I. Yeh, Knowledge dscovery of concrete materal usng genetc operaton trees, Expert Systems wth Applcatons 36(3) (009), [4] M. H. Fazel Zarand and I. B. Türksen et al., Fuzzy polynomal neural networks for approxmaton of the compressve strength of concrete, Appled Soft Computng 8(1) (008), g