Moreau-Yosida Regularization for Grouped Tree Structure Learning
|
|
- Mariah McCoy
- 5 years ago
- Views:
Transcription
1 Moreau-Yosida Reguarization for Grouped Tree Structure Learning Jun Liu Computer Science and Engineering Arizona State University Jieping Ye Computer Science and Engineering Arizona State University Abstract We consider the tree structured group Lasso where the structure over the features can be represented as a tree with eaf nodes as features and interna nodes as custers of the features. The structured reguarization with a pre-defined tree structure is based on a group-lasso penaty, where one group is defined for each node in the tree. Such a reguarization can hep uncover the structured sparsity, which is desirabe for appications with some meaningfu tree structures on the features. However, the tree structured group Lasso is chaenging to sove due to the compex reguarization. In this paper, we deveop an efficient agorithm for the tree structured group Lasso. One of the key steps in the proposed agorithm is to sove the Moreau-Yosida reguarization associated with the grouped tree structure. The main technica contributions of this paper incude (1) we show that the associated Moreau-Yosida reguarization admits an anaytica soution, and (2) we deveop an efficient agorithm for determining the effective interva for the reguarization parameter. Our experimenta resuts on the AR and JAFFE face data sets demonstrate the efficiency and effectiveness of the proposed agorithm. 1 Introduction Many machine earning agorithms can be formuated as a penaized optimization probem: min x (x) + λφ(x), (1) where (x) is the empirica oss function (e.g., the east squares oss and the ogistic oss), λ > 0 is the reguarization parameter, and φ(x) is the penaty term. Recenty, sparse earning via 1 reguarization [20] and its various extensions has received increasing attention in many areas incuding machine earning, signa processing, and statistics. In particuar, the group Lasso [1, 16, 22] utiizes the group information of the features, and yieds a soution with grouped sparsity. The traditiona group Lasso assumes that the groups are non-overapping. However, in many appications the features may form more compex overapping groups. Zhao et a. [23] extended the group Lasso to the case of overapping groups, imposing hierarchica reationships for the features. Jacob et a. [6] considered group Lasso with overaps, and studied theoretica properties of the estimator. Jenatton et a. [7] considered the consistency property of the structured overapping group Lasso, and designed an active set agorithm. In many appications, the features can naturay be represented using certain tree structures. For exampe, the image pixes of the face image shown in Figure 1 can be represented as a tree, where each parent node contains a series of chid nodes that enoy spatia ocaity; genes/proteins may form certain hierarchica tree structures. Kim and Xing [9] studied the tree structured group Lasso for muti-task earning, where mutipe reated tasks foow a tree structure. One chaenge in the practica appication of the tree structured group Lasso is that the resuting optimization probem is much more difficut to sove than Lasso and group Lasso, due to the compex reguarization. 1
2 (a) (b) (c) (d) Figure 1: Iustration of the tree structure of a two-dimensiona face image. The image (a) can be divided into 16 sub-images in (b) according to the spatia ocaity, where the sub-images can be viewed as the chid nodes of (a). Simiary, each sub-image in (b) can be divided into 16 sub-images in (c), and such a process is repeated for the sub-images in (c) to get (d). G 0 1 G 1 1 G 1 2 G 1 3 G 2 1 G 2 2 G 2 3 G 2 4 Figure 2: A sampe index tree for iustration. Root: G 0 1 = {1, 2, 3, 4, 5, 6, 7, 8}. Depth 1: G 1 1 = {1, 2}, G 1 2 = {3, 4, 5, 6}, G 1 3 = {7, 8}. Depth 2: G 2 1 = {1}, G 2 2 = {2}, G 2 3 = {3, 4}, G 2 4 = {5, 6}. In this paper, we deveop an efficient agorithm for the tree structured group Lasso, i.e., the optimization probem (1) with φ( ) being the grouped tree structure reguarization (see Equation 2). One of the key steps in the proposed agorithm is to sove the Moreau-Yosida reguarization [17, 21] associated with the grouped tree structure. The main technica contributions of this paper incude: (1) we show that the associated Moreau-Yosida reguarization admits an anaytica soution, and the resuting agorithm for the tree structured group Lasso has a time compexity comparabe to Lasso and group Lasso, and (2) we deveop an efficient agorithm for determining the effective interva for the parameter λ, which is important in the practica appication of the agorithm. We have performed experimenta studies using the AR and JAFFE face data sets, where the features form a hierarchica tree structure based on the spatia ocaity as shown in Figure 1. Our experimenta resuts demonstrate the efficiency and effectiveness of the proposed agorithm. Note that whie the present paper was under review, we became aware of a recent work by Jenatton et a. [8] which appied bock coordinate ascent in the dua and showed that the agorithm converges in one pass. 2 Grouped Tree Structure Reguarization We begin with the definition of the so-caed index tree: Definition 1. For an index tree T of depth d, we et T i = {G i 1, G i 2,..., G i n i } contain a the node(s) corresponding to depth i, where n 0 = 1, G 0 1 = {1, 2,..., p} and n i 1, i = 1, 2,..., d. The nodes satisfy the foowing conditions: 1) the nodes from the same depth eve have non-overapping indices, i.e., G i Gi k =, i = 1,..., d, k, 1, k n i; and 2) et G i 1 0 of a non-root node G i, then Gi Gi 1 0. be the parent node Figure 2 shows a sampe index tree. We can observe that 1) the index sets from different nodes may overap, e.g., any parent node overaps with its chid nodes; 2) the nodes from the same depth eve do not overap; and 3) the index set of a chid node is a subset of that of its parent node. The grouped tree structure reguarization is defined as: d n i φ(x) = w x i G i, (2) i=0 =1 where x R p, w i 0 (i = 0, 1,..., d, = 1, 2,..., n i) is the pre-defined weight for the node G i, is the Eucidean norm, and x G i is a vector composed of the entries of x with the indices in G i. 2
3 In the next section, we study the Moreau-Yosida reguarization [17, 21] associated with (2), deveop an anaytica soution for such a reguarization, propose an efficient agorithm for soving (1), and specify the meaningfu interva for the reguarization parameter λ. 3 Moreau-Yosida Reguarization of φ( ) The Moreau-Yosida reguarization associated with the grouped tree structure reguarization φ( ) for a given v R p is given by: φ λ (v) = min x f(x) = 1 d 2 x n i v 2 + λ w x i G i, (3) i=0 =1 for some λ > 0. Denote the minimizer of (3) as π λ (v). The Moreau-Yosida reguarization has many usefu properties: 1) φ λ ( ) is continuousy differentiabe despite the fact that φ( ) is non-smooth; 2) π λ ( ) is a non-expansive operator. More properties on the genera Moreau-Yosida reguarization can be found in [5, 10]. Note that, f( ) in (3) is indeed a specia case of the probem (1) with (x) = 1 2 x v 2. Our recent study has shown that, the efficient optimization of the Moreau- Yosida reguarization is key to many optimization agorithms [13, Section 2]. Next, we focus on the efficient optimization of (3). For convenience of subsequent discussion, we denote λ i = λwi. 3.1 An Anaytica Soution We show that the minimization of (3) admits an anaytica soution. We first present the detaied procedure for finding the minimizer in Agorithm 1. Agorithm 1 Moreau-Yosida Reguarization of the tree structured group Lasso (MY tglasso ) Input: v R p, the index tree T with nodes G i (i = 0, 1,..., d, = 1, 2,..., n i) that satisfy Definition 1, the weights w i 0 (i = 0, 1,..., d, = 1, 2,..., n i), λ > 0, and λ i = λwi Output: u 0 R p 1: Set u d+1 = v, (4) 2: for i = d to 0 do 3: for = 1 to n i do 4: Compute 5: end for 6: end for u i G i = u i+1 G i λ i u i+1 G i 0 u i+1 λ i G i u i+1 G i u i+1 > λ i G i, (5) In the impementation of the MY tglasso agorithm, we ony need to maintain a working variabe u, which is initiaized with v. We then traverse the index tree T in the reverse breadth-first order to update u. At the traversed node G i, we update u G i according to the operation in (5), which reduces the Eucidean norm of u G i by at most λ i. The time compexity of MY tglasso is O( d ni i=0 =1 Gi ). By using Definition 1, we have n i =1 Gi p. Therefore, the time compexity of MY tglasso is O(pd). If the tree is baanced, i.e., d = O(og p), then the time compexity of MY tglasso is O(p og p). MY tglasso can hep expain why the structured group sparsity can be induced. Let us anayze the tree given in Figure 2, with the soution denoted as x. We et w i = 1, i,, λ = 2, and v = [1, 2, 1, 1, 4, 4, 1, 1] T. After traversing the nodes of depth 2, we can get that the eements of x with indices in G 2 1 and G 2 3 are zero; and when the traversa continues to the nodes of depth 1, the eements of x with indices in G 1 1 and G 1 3 are set to zero, but those with G 2 4 are sti nonzero. Finay, after traversing the root node, we obtain x = [0, 0, 0, 0, 1, 1, 0, 0] T. 3
4 Next, we show that MY tglasso finds the exact minimizer of (3). The main resut is summarized in the foowing theorem: Theorem 1. u 0 returned by Agorithm 1 is the unique soution to (3). Before giving the detaied proof for Theorem 1, we introduce some notations, and present severa technica emmas. Define the mapping φ i : Rp R as We can then express φ(x) defined in (2) as: φ(x) = φ i (x) = x G i. (6) d n i λ i φ i (x). i=0 =1 The subdifferentia of f( ) defined in (3) at the point x can be written as: d n i f(x) = x v + λ i φ i (x), (7) where φ i (x) = and G i denotes the compementary set of Gi. i=0 =1 { } y R p : y 1, y G i = 0 { y R p : y G i = x G i x G i, y G i = 0 } if x G i = 0 if x G i 0, Lemma 1. For any 1 i d, 1 n i, we can find a unique path from the node G i to the root node G 0 1. Let the nodes on this path be G r, for = 0, 1,..., i with r 0 = 1 and r i =. We have G i G r, = 0, 1,..., i 1. (9) G i G r =, r r, = 1, 2,..., i 1, r = 1, 2,..., n i. (10) Proof: According to Definition 1, we can find a unique path from the node G i to the root node G0 1. In addition, based on the structure of the index tree, we have (9) and (10). Lemma 2. For any i = 1, 2,..., d, = 1, 2,..., n i, we have u i G u i+1 λ i ( i G i φ i (u i ) ), (11) G i Proof: We can verify (11) using (5), (6) and (8). For (12), it foows from (6) and (8) that, it is sufficient to verify that u 0 G i φ i (u i ) φ i (u 0 ). (12) = α i u i G i, for some α i 0. (13) It foows from Lemma 1 that we can find a unique path from G i to G0 1. Denote the nodes on the path as: G r, where = 0, 1,..., i, r i =, and r 0 = 1. We first anayze the reationship between u i and u i 1. If G i G i ui G i 1 λi 1, we have u i 1 = 0, which eads to u i 1 = 0 by using G i 1 G i (9). Otherwise, if u i 1 G i = ui G i 1 ui G i 1 λi 1 u i G i ui G i 1 > λi 1, we have u i 1 = G i 1 by using (9). Therefore, we have u i 1 G i ui G i 1 λi 1 ui G i 1 (8) u i G i 1, which eads to = β i u i G i, for some β i 0. (14) 4
5 By a simiar argument, we have Together with (9), we have u 1 G r = β u G r, β 0, = 1, 2,..., i 1. (15) u 1 G i = β u G i, β 0,, = 1, 2,..., i 1. (16) From (14) and (16), we show (13) hods with α i = Πi =1 β. This competes the proof. We are now ready to prove our main resut: Proof of Theorem 1: It is easy to verify that f( ) defined in (3) is strongy convex, thus it admits a unique minimizer. Our methodoogy for the proof is to show that 0 f(u 0 ), (17) which is the sufficient and necessary condition for u 0 to be the minimizer of f( ). According to Definition 1, the eaf nodes are non-overapping. We assume that the union of the eaf nodes equas to {1, 2,..., p}; otherwise, we can add to the index tree the additiona eaf nodes with weight 0 to satisfy the aforementioned assumption. Ceary, the origina index tree and the new index tree with the additiona eaf nodes of weight 0 yied the same penaty φ( ) in (2), the same Moreau- Yosida reguarization in (3), and the same soution from Agorithm 1. Therefore, to prove (17), it suffices to show 0 f(u 0 ) G i, for a the eaf nodes G i. Next, we focus on estabishing the foowing reationship: 0 f(u 0 ) G d 1. (18) It foows from Lemma 1 that, we can find a unique path from the node G d 1 to the root G 0 1. Let the nodes on this path are G r, for = 0, 1,..., d with r 0 = 1 and r d = 1. By using (10) of Lemma 1, we can get that the nodes that contain the index set G d 1 are exacty on the aforementioned path. In other words, x, we have ( φ r (x) ) = {0}, r r G d, = 1, 2,..., d 1, r = 1, 2,..., n i (19) 1 by using (6) and (8). Appying (11) and (12) of Lemma 2 to each node on the aformetioned path, we have u +1 G r u G r λ r ( φ r (u ) ) G r λ r ( φ r (u 0 ) ) G r, = 0, 1,..., d. (20) Making using of (9), we obtain from (20) the foowing reationship: u +1 G d 1 Adding (21) for = 0, 1,..., d, we have u G λ ( d r 1 φ r (u 0 ) ), = 0, 1,..., d. (21) G d 1 u d+1 G d 1 u 0 G d 1 d λ ( r φ r (u 0 ) ) G d 1 =0 It foows from (4), (7), (19) and (22) that (18) hods. Simiary, we have 0 f(u 0 ) G i for the other eaf nodes G i. Thus, we have (17). 3.2 The Proposed Optimization Agorithm With the anaytica soution for π λ ( ), the minimizer of (3), we can appy many existing methods for soving (1). First, we show in the foowing emma that, the optima soution to (1) can be computed as a fixed point. We sha show in Section 3.3 that, the resut in this emma can aso hep determine the interva for the vaues of λ. Lemma 3. Let x be an optima soution to (1). Then, x satisfies: (22) x = π λτ (x τ (x )), τ > 0. (23) 5
6 Proof: x is an optima soution to (1), if and ony if 0 (x ) + λ φ(x ), (24) which eads to 0 x (x τ (x )) + λτ φ(x ), τ > 0. (25) Thus, we have x 1 = arg min x 2 x (x τ (x )) 2 +λτφ(x). Reca that π λ ( ) is the minimizer of (3). We have (23). It foows from Lemma 3 that we can appy the fixed point continuation method [4] for soving (1). It is interesting to note that, with an appropriatey chosen τ, the scheme in (23) indeed corresponds to the gradient method deveoped for the composite function optimization [2, 19], achieving the goba convergence rate of O(1/k) for k iterations. In addition, the scheme in (23) can be acceerated to obtain the acceerated gradient descent [2, 19], where the Moreau-Yosidea reguarization aso needs to be evauated in each of its iteration. We empoy the acceerated gradient descent deveoped in [2] for the optimization in this paper. The agorithm is caed tglasso, which stands for the tree structured group Lasso. Note that, tglasso incudes our previous agorithm [11] as a specia case, when the index tree is of depth 1 and w 0 1 = The Effective Interva for the Vaues of λ When estimating the mode parameters via (1), a key issue is to choose the appropriate vaues for the reguarization parameter λ. A commony used approach is to seect the reguarization parameter from a set of candidate vaues, whose vaues, however, need to be pre-specified in advance. Therefore, it is essentia to specify the effective interva for the vaues of λ. An anaysis of MY tglasso in Agorithm 1 shows that, with increasing λ, the entries of the soution to (3) are monotonicay decreasing. Intuitivey, the soution to (3) sha be exacty zero if λ is sufficienty arge and a the entries of x are penaized in φ(x). Next, we summarize the main resuts of this subsection. Theorem 2. The zero point is a soution to (1) if and ony if the zero point is a soution to (3) with v = (0). For the penaty φ(x), et us assume that a entries of x are penaized, i.e., {1, 2,..., p}, there exists at east one node G i that contains and meanwhie wi > 0. Then, for any 0 < (0) < +, there exists a unique λ max < + satisfying: 1) if λ λ max the zero point is a soution to (1), and 2) if 0 < λ < λ max, the zero point is not a soution to (1). Proof: If x = 0 is the soution to (1), we have (24). Setting τ = 1 in (23), we obtain that x = 0 is aso the soution to (3) with v = (0). If x = 0 is the soution to (3) with v = (0), we have 0 (0) + λ φ(0), which indicates that x = 0 is the soution to (1). The function φ(x) is cosed convex. According to [18, Chapater 3.1.5], φ(0) is a cosed convex and non-empty bounded set. From (8), it is cear that 0 φ(0). Therefore, we have x R, x φ(0), where R is a finite radius constant. Let S = {x : x = αr (0)/ (0), α [0, 1]} be the ine segment from 0 to R (0)/ (0). It is obvious that S is cosed convex and bounded. Define I = S φ(0), which is ceary cosed convex and bounded. Define λ max = (0) / max x I x. It foows from (0) > 0 and the boundedness of I that λ max > 0. We first show λ max < +. Otherwise, we have I = {0}. Thus, λ > 0, we have (0)/λ / φ(0), which indicates that 0 is neither the soution to (1) nor (3) with v = (0). Reca the assumption that, {1, 2,..., p}, there exists at east one node G i that contains and meanwhie wi > 0. It foows from Agorithm 1 that, there exists a λ < + such that when λ > λ, 0 is a soution to (3) with v = (0), eading to a contradiction. Therefore, we have 0 < λ max < +. Let λ max = λ max. The arguments hod since 1) if λ λ max, then (0)/λ I φ(0); and 2) if 0 < λ < λ max, then (0)/λ / φ(0). When (0) = 0, the probem (1) has a trivia zero soution. We next focus on the nontrivia case (0) 0. We present the agorithm for efficienty soving λ max in Agorithm 2. In Step 1, λ 0 is an initia guess of the soution. Our empirica study shows that λ 0 = (0) 2 d ni works quite i=0 =1 (wi )2 we. In Step 2-6, we specify an interva [λ 1, λ 2 ] in which λ max resides. Finay, in Step 7-14, we appy bisection for computing λ max. 6
7 Agorithm 2 Finding λ max via Bisection Input: (0), the index tree T with nodes G i (i = 0, 1,..., d, = 1, 2,..., n i), the weights w i 0 (i = 0, 1,..., d, = 1, 2,..., n i ), λ 0, and δ = Output: λ max 1: Set λ = λ 0 2: if φ λ ( (0)) = 0 then 3: Set λ 2 = λ, and find the argest λ 1 = 2 i λ, i = 1, 2,... such that π λ1 ( (0)) 0 4: ese 5: Set λ 1 = λ, and find the smaest λ 2 = 2 i λ, i = 1, 2,... such that π λ2 ( (0)) = 0 6: end if 7: whie λ 2 λ 1 δ do 8: Set λ = λ1+λ2 2 9: if π λ ( (0)) = 0 then 10: Set λ 2 = λ 11: ese 12: Set λ 1 = λ 13: end if 14: end whie 15: λ max = λ 4 Experiments We have conducted experiments to evauate the efficiency and effectiveness of the proposed tglasso agorithm on the face data sets JAFFE [14] and AR [15]. JAFFE contains 213 images of ten Japanese actresses with seven facia expressions: neutra, happy, disgust, fear, anger, sadness, and suprise. We used a subsect of AR that contains 400 images corresponding to 100 subects, with each subect containing four facia expression: neutra, smie, anger, and scream. For both data sets, we resize the image size to 64 64, and make use of the tree structure depicted in Figure 1. Our task is to discriminate each facia expression from the rest ones. Thus, we have seven and four binary cassification tasks for JAFFE and AR, respectivey. We empoy the east squares oss for ( ), and set the reguarization parameter λ = r λ max, where λ max is computed using Agorithm 2, and r = {5 10 1, , , , , , , }. The source codes, incuded in the SLEP package [12], are avaiabe onine 1. Tabe 1: Computationa time (seconds) for one binary cassification task (averaged over 7 and 4 runs for JAFFE and AR, respectivey). The tota time for a eight reguarization parameters is reported. JAFFE AR tglasso aternating agorithm [9] Efficiency of the Proposed tglasso We compare our proposed tglasso with the recenty proposed aternating agorithm [9] designed for the tree-guided group Lasso. We report the tota computationa time (seconds) for running one binary cassification task (averaged over 7 and 4 tasks for JAFFE and AR, respectivey) corresponding to the eight reguarization parameters in Tabe 1. We can obseve that tglasso is much more efficient than the aternating agorithm. We note that, the key step of tglasso in each iteration is the associated Moreau-Yosida reguarization, which can be efficienty computed due to the existence of an anaytica soution; and the key step of the aternating agorithm in each iteration is the matrix inversion, which does not scae we to high-dimensiona data. Cassification Performance We compare the cassification performance of tglasso with Lasso. On AR, we use 50 subects for training, and the remaining 50 subects for testing; and on JAFFE, we use 8 subects for training, and the remaining 2 subects for testing. This subect-independent setting is chaenging, as the subects to be tested are not incuded in the training set. The reported resuts are averaged over 10 runs for randomy chosen subects. For each binary cassification task, we compute the baanced error rate [3] to cope with the unbaanced positive and negative sampes. We 1 ye02/software/slep/ 7
8 26 25 AR tglasso Lasso JAFFE tglasso Lasso baanced error rate (%) baanced error rate (%) e 1 2e 1 1e 1 5e 2 2e 2 1e 2 5e 3 2e 3 reguarization parameter r 36 5e 1 2e 1 1e 1 5e 2 2e 2 1e 2 5e 3 2e 3 reguarization parameter r Figure 3: Cassification performance comparison between Lasso and the tree structured group Lasso. The horizonta axis corresponds to different reguarization parameters λ = r λ max. Neutra Smie Anger Sceam Figure 4: Markers obtained by Lasso, and tree structured group Lasso (white pixes correspond to the markers). First row: face images of four expression from the AR data set; Second row: the markers identified by tree structured group Lasso; Third row: the markers identified by Lasso. report the averaged resuts in Figure 3. Resuts show that tglasso outperforms Lasso in both cases. This verifies the effectiveness of tglasso in incorporating the tree structure in the formuation, i.e., the spatia ocaity information of the face images. Figure 4 shows the markers identified by tglasso and Lasso under the best reguarization parameter. We can observe from the figure that tglasso resuts in a bock sparsity soution, and most of the seected pixes are around mouths and eyes. 5 Concusion In this paper, we consider the efficient optimization for the tree structured group Lasso. Our main technica resut show the Moreau-Yosida reguarization associated with the tree structured group Lasso admits an anaytica soution. Based on the Moreau-Yosida reguarization, we an design efficient agorithm for soving the grouped tree structure reguarized optimization probem for smooth convex oss functions, and deveop an efficient agorithm for determining the effective interva for the parameter λ. Our experimenta resuts on the AR and JAFFE face data sets demonstrate the efficiency and effectiveness of the proposed agorithm. We pan to appy the proposed agorithm to other appications in computer vision and bioinformatics invoving the tree structure. Acknowedgments This work was supported by NSF IIS , IIS , CCF , IIS , NGA HM , NSFC , , and the Office of the Director of Nationa Inteigence (ODNI), Inteigence Advanced Research Proects Activity (IARPA), through the US Army. 8
9 References [1] F. Bach, G. Lanckriet, and M. Jordan. Mutipe kerne earning, conic duaity, and the SMO agorithm. In Internationa conference on Machine earning, [2] A. Beck and M. Teboue. A fast iterative shrinkage-threshoding agorithm for inear inverse probems. SIAM Journa on Imaging Sciences, 2(1): , [3] I. Guyon, A. B. Hur, S. Gunn, and G. Dror. Resut anaysis of the nips 2003 feature seection chaenge. In Neura Information Processing Systems, pages , [4] E.T. Hae, W. Yin, and Y. Zhang. Fixed-point continuation for 1-minimization: Methodoogy and convergence. SIAM Journa on Optimization, 19(3): , [5] J. Hiriart-Urruty and C. Lemarécha. Convex Anaysis and Minimization Agorithms I & II. Springer Verag, Berin, [6] L. Jacob, G. Obozinski, and J. Vert. Group asso with overap and graph asso. In Internationa Conference on Machine Learning, [7] R. Jenatton, J.-Y. Audibert, and F. Bach. Structured variabe seection with sparsity-inducing norms. Technica report, arxiv: v2, [8] R. Jenatton, J. Maira, G. Obozinski, and F. Bach. Proxima methods for sparse hierarchica dictionary earning. In Internationa Conference on Machine Learning, [9] S. Kim and E. P. Xing. Tree-guided group asso for muti-task regression with structured sparsity. In Internationa Conference on Machine Learning, [10] C. Lemarécha and C. Sagastizába. Practica aspects of the Moreau-Yosida reguarization I: Theoretica properties. SIAM Journa on Optimization, 7(2): , [11] J. Liu, S. Ji, and J. Ye. Muti-task feature earning via efficient 2,1-norm minimization. In Uncertainty in Artificia Inteigence, [12] J. Liu, S. Ji, and J. Ye. SLEP: Sparse Learning with Efficient Proections. Arizona State University, [13] J. Liu, L. Yuan, and J. Ye. An efficient agorithm for a cass of fused asso probems. In ACM SIGKDD Conference on Knowedge Discovery and Data Mining, [14] M. J. Lyons, J. Budynek, and S. Akamatsu. Automatic cassification of singe facia images. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 21(12): , [15] A.M. Martinez and R. Benavente. The AR face database. Technica report, [16] L. Meier, S. Geer, and P. Bühmann. The group asso for ogistic regression. Journa of the Roya Statistica Society: Series B, 70:53 71, [17] J.-J. Moreau. Proximité et duaité dans un espace hibertien. Buetin de a Societe mathematique de France, 93: , [18] Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Kuwer Academic Pubishers, [19] Y. Nesterov. Gradient methods for minimizing composite obective function. CORE Discussion Paper, [20] R. Tibshirani. Regression shrinkage and seection via the asso. Journa of the Roya Statistica Society Series B, 58(1): , [21] K. Yosida. Functiona Anaysis. Springer Verag, Berin, [22] M. Yuan and Y. Lin. Mode seection and estimation in regression with grouped variabes. Journa Of The Roya Statistica Society Series B, 68(1):49 67, [23] P. Zhao, G. Rocha, and B. Yu. The composite absoute penaties famiy for grouped and hierarchica variabe seection. Annas of Statistics, 37(6A): ,
Statistical Learning Theory: A Primer
Internationa Journa of Computer Vision 38(), 9 3, 2000 c 2000 uwer Academic Pubishers. Manufactured in The Netherands. Statistica Learning Theory: A Primer THEODOROS EVGENIOU, MASSIMILIANO PONTIL AND TOMASO
More informationProjection onto A Nonnegative Max-Heap
Projection onto A Nonnegatie Max-Heap Jun Liu Arizona State Uniersity Tempe, AZ 85287, USA j.iu@asu.edu Liang Sun Arizona State Uniersity Tempe, AZ 85287, USA sun.iang@asu.edu Jieping Ye Arizona State
More informationStatistical Learning Theory: a Primer
??,??, 1 6 (??) c?? Kuwer Academic Pubishers, Boston. Manufactured in The Netherands. Statistica Learning Theory: a Primer THEODOROS EVGENIOU AND MASSIMILIANO PONTIL Center for Bioogica and Computationa
More informationDistributed average consensus: Beyond the realm of linearity
Distributed average consensus: Beyond the ream of inearity Usman A. Khan, Soummya Kar, and José M. F. Moura Department of Eectrica and Computer Engineering Carnegie Meon University 5 Forbes Ave, Pittsburgh,
More informationStochastic Variational Inference with Gradient Linearization
Stochastic Variationa Inference with Gradient Linearization Suppementa Materia Tobias Pötz * Anne S Wannenwetsch Stefan Roth Department of Computer Science, TU Darmstadt Preface In this suppementa materia,
More informationExplicit overall risk minimization transductive bound
1 Expicit overa risk minimization transductive bound Sergio Decherchi, Paoo Gastado, Sandro Ridea, Rodofo Zunino Dept. of Biophysica and Eectronic Engineering (DIBE), Genoa University Via Opera Pia 11a,
More informationA Novel Learning Method for Elman Neural Network Using Local Search
Neura Information Processing Letters and Reviews Vo. 11, No. 8, August 2007 LETTER A Nove Learning Method for Eman Neura Networ Using Loca Search Facuty of Engineering, Toyama University, Gofuu 3190 Toyama
More informationAsynchronous Control for Coupled Markov Decision Systems
INFORMATION THEORY WORKSHOP (ITW) 22 Asynchronous Contro for Couped Marov Decision Systems Michae J. Neey University of Southern Caifornia Abstract This paper considers optima contro for a coection of
More informationA. Distribution of the test statistic
A. Distribution of the test statistic In the sequentia test, we first compute the test statistic from a mini-batch of size m. If a decision cannot be made with this statistic, we keep increasing the mini-batch
More informationarxiv: v2 [cs.lg] 4 Sep 2014
Cassification with Sparse Overapping Groups Nikhi S. Rao Robert D. Nowak Department of Eectrica and Computer Engineering University of Wisconsin-Madison nrao2@wisc.edu nowak@ece.wisc.edu ariv:1402.4512v2
More informationAn Algorithm for Pruning Redundant Modules in Min-Max Modular Network
An Agorithm for Pruning Redundant Modues in Min-Max Moduar Network Hui-Cheng Lian and Bao-Liang Lu Department of Computer Science and Engineering, Shanghai Jiao Tong University 1954 Hua Shan Rd., Shanghai
More informationBP neural network-based sports performance prediction model applied research
Avaiabe onine www.jocpr.com Journa of Chemica and Pharmaceutica Research, 204, 6(7:93-936 Research Artice ISSN : 0975-7384 CODEN(USA : JCPRC5 BP neura networ-based sports performance prediction mode appied
More informationDIFFERENCE-OF-CONVEX LEARNING: DIRECTIONAL STATIONARITY, OPTIMALITY, AND SPARSITY
SIAM J. OPTIM. Vo. 7, No. 3, pp. 1637 1665 c 017 Society for Industria and Appied Mathematics DIFFERENCE-OF-CONVEX LEARNING: DIRECTIONAL STATIONARITY, OPTIMALITY, AND SPARSITY MIJU AHN, JONG-SHI PANG,
More informationLecture Note 3: Stationary Iterative Methods
MATH 5330: Computationa Methods of Linear Agebra Lecture Note 3: Stationary Iterative Methods Xianyi Zeng Department of Mathematica Sciences, UTEP Stationary Iterative Methods The Gaussian eimination (or
More informationDetermining The Degree of Generalization Using An Incremental Learning Algorithm
Determining The Degree of Generaization Using An Incrementa Learning Agorithm Pabo Zegers Facutad de Ingeniería, Universidad de os Andes San Caros de Apoquindo 22, Las Condes, Santiago, Chie pzegers@uandes.c
More informationarxiv: v1 [math.co] 17 Dec 2018
On the Extrema Maximum Agreement Subtree Probem arxiv:1812.06951v1 [math.o] 17 Dec 2018 Aexey Markin Department of omputer Science, Iowa State University, USA amarkin@iastate.edu Abstract Given two phyogenetic
More informationMinimum Enclosing Circle of a Set of Fixed Points and a Mobile Point
Minimum Encosing Circe of a Set of Fixed Points and a Mobie Point Aritra Banik 1, Bhaswar B. Bhattacharya 2, and Sandip Das 1 1 Advanced Computing and Microeectronics Unit, Indian Statistica Institute,
More informationMultilayer Kerceptron
Mutiayer Kerceptron Zotán Szabó, András Lőrincz Department of Information Systems, Facuty of Informatics Eötvös Loránd University Pázmány Péter sétány 1/C H-1117, Budapest, Hungary e-mai: szzoi@csetehu,
More informationOn the Goal Value of a Boolean Function
On the Goa Vaue of a Booean Function Eric Bach Dept. of CS University of Wisconsin 1210 W. Dayton St. Madison, WI 53706 Lisa Heerstein Dept of CSE NYU Schoo of Engineering 2 Metrotech Center, 10th Foor
More informationLearning Structural Changes of Gaussian Graphical Models in Controlled Experiments
Learning Structura Changes of Gaussian Graphica Modes in Controed Experiments Bai Zhang and Yue Wang Bradey Department of Eectrica and Computer Engineering Virginia Poytechnic Institute and State University
More informationA Solution to the 4-bit Parity Problem with a Single Quaternary Neuron
Neura Information Processing - Letters and Reviews Vo. 5, No. 2, November 2004 LETTER A Soution to the 4-bit Parity Probem with a Singe Quaternary Neuron Tohru Nitta Nationa Institute of Advanced Industria
More informationA Brief Introduction to Markov Chains and Hidden Markov Models
A Brief Introduction to Markov Chains and Hidden Markov Modes Aen B MacKenzie Notes for December 1, 3, &8, 2015 Discrete-Time Markov Chains You may reca that when we first introduced random processes,
More informationFrom Margins to Probabilities in Multiclass Learning Problems
From Margins to Probabiities in Muticass Learning Probems Andrea Passerini and Massimiiano Ponti 2 and Paoo Frasconi 3 Abstract. We study the probem of muticass cassification within the framework of error
More informationarxiv: v1 [cs.lg] 31 Oct 2017
ACCELERATED SPARSE SUBSPACE CLUSTERING Abofaz Hashemi and Haris Vikao Department of Eectrica and Computer Engineering, University of Texas at Austin, Austin, TX, USA arxiv:7.26v [cs.lg] 3 Oct 27 ABSTRACT
More informationFORECASTING TELECOMMUNICATIONS DATA WITH AUTOREGRESSIVE INTEGRATED MOVING AVERAGE MODELS
FORECASTING TEECOMMUNICATIONS DATA WITH AUTOREGRESSIVE INTEGRATED MOVING AVERAGE MODES Niesh Subhash naawade a, Mrs. Meenakshi Pawar b a SVERI's Coege of Engineering, Pandharpur. nieshsubhash15@gmai.com
More informationarxiv: v1 [cs.db] 1 Aug 2012
Functiona Mechanism: Regression Anaysis under Differentia Privacy arxiv:208.029v [cs.db] Aug 202 Jun Zhang Zhenjie Zhang 2 Xiaokui Xiao Yin Yang 2 Marianne Winsett 2,3 ABSTRACT Schoo of Computer Engineering
More informationFirst-Order Corrections to Gutzwiller s Trace Formula for Systems with Discrete Symmetries
c 26 Noninear Phenomena in Compex Systems First-Order Corrections to Gutzwier s Trace Formua for Systems with Discrete Symmetries Hoger Cartarius, Jörg Main, and Günter Wunner Institut für Theoretische
More informationProbabilistic Graphical Models
Schoo of Computer Science Probabiistic Graphica Modes Gaussian graphica modes and Ising modes: modeing networks Eric Xing Lecture 0, February 0, 07 Reading: See cass website Eric Xing @ CMU, 005-07 Network
More informationMelodic contour estimation with B-spline models using a MDL criterion
Meodic contour estimation with B-spine modes using a MDL criterion Damien Loive, Ney Barbot, Oivier Boeffard IRISA / University of Rennes 1 - ENSSAT 6 rue de Kerampont, B.P. 80518, F-305 Lannion Cedex
More informationUniprocessor Feasibility of Sporadic Tasks with Constrained Deadlines is Strongly conp-complete
Uniprocessor Feasibiity of Sporadic Tasks with Constrained Deadines is Strongy conp-compete Pontus Ekberg and Wang Yi Uppsaa University, Sweden Emai: {pontus.ekberg yi}@it.uu.se Abstract Deciding the feasibiity
More informationAdaptive Regularization for Transductive Support Vector Machine
Adaptive Reguarization for Transductive Support Vector Machine Zengin Xu Custer MMCI Saarand Univ. & MPI INF Saarbrucken, Germany zxu@mpi-inf.mpg.de Rong Jin Computer Sci. & Eng. Michigan State Univ. East
More informationSVM: Terminology 1(6) SVM: Terminology 2(6)
Andrew Kusiak Inteigent Systems Laboratory 39 Seamans Center he University of Iowa Iowa City, IA 54-57 SVM he maxima margin cassifier is simiar to the perceptron: It aso assumes that the data points are
More informationOptimality of Inference in Hierarchical Coding for Distributed Object-Based Representations
Optimaity of Inference in Hierarchica Coding for Distributed Object-Based Representations Simon Brodeur, Jean Rouat NECOTIS, Département génie éectrique et génie informatique, Université de Sherbrooke,
More informationConvergence Property of the Iri-Imai Algorithm for Some Smooth Convex Programming Problems
Convergence Property of the Iri-Imai Agorithm for Some Smooth Convex Programming Probems S. Zhang Communicated by Z.Q. Luo Assistant Professor, Department of Econometrics, University of Groningen, Groningen,
More informationCONJUGATE GRADIENT WITH SUBSPACE OPTIMIZATION
CONJUGATE GRADIENT WITH SUBSPACE OPTIMIZATION SAHAR KARIMI AND STEPHEN VAVASIS Abstract. In this paper we present a variant of the conjugate gradient (CG) agorithm in which we invoke a subspace minimization
More informationSUPPLEMENTARY MATERIAL TO INNOVATED SCALABLE EFFICIENT ESTIMATION IN ULTRA-LARGE GAUSSIAN GRAPHICAL MODELS
ISEE 1 SUPPLEMENTARY MATERIAL TO INNOVATED SCALABLE EFFICIENT ESTIMATION IN ULTRA-LARGE GAUSSIAN GRAPHICAL MODELS By Yingying Fan and Jinchi Lv University of Southern Caifornia This Suppementary Materia
More information(This is a sample cover image for this issue. The actual cover is not yet available at this time.)
(This is a sampe cover image for this issue The actua cover is not yet avaiabe at this time) This artice appeared in a journa pubished by Esevier The attached copy is furnished to the author for interna
More informationIntegrating Factor Methods as Exponential Integrators
Integrating Factor Methods as Exponentia Integrators Borisav V. Minchev Department of Mathematica Science, NTNU, 7491 Trondheim, Norway Borko.Minchev@ii.uib.no Abstract. Recenty a ot of effort has been
More informationA Simple and Efficient Algorithm of 3-D Single-Source Localization with Uniform Cross Array Bing Xue 1 2 a) * Guangyou Fang 1 2 b and Yicai Ji 1 2 c)
A Simpe Efficient Agorithm of 3-D Singe-Source Locaization with Uniform Cross Array Bing Xue a * Guangyou Fang b Yicai Ji c Key Laboratory of Eectromagnetic Radiation Sensing Technoogy, Institute of Eectronics,
More informationNon-parametric Group Orthogonal Matching Pursuit for Sparse Learning with Multiple Kernels
Non-parametric Group Orthogona Matching Pursuit for Sparse Learning with Mutipe Kernes Vikas Sindhwani and Auréie C. Lozano IBM T.J. Watson Research Center Yorktown Heights, NY 0598 {vsindhw,acozano}@us.ibm.com
More informationDiscriminant Analysis: A Unified Approach
Discriminant Anaysis: A Unified Approach Peng Zhang & Jing Peng Tuane University Eectrica Engineering & Computer Science Department New Oreans, LA 708 {zhangp,jp}@eecs.tuane.edu Norbert Riede Tuane University
More informationC. Fourier Sine Series Overview
12 PHILIP D. LOEWEN C. Fourier Sine Series Overview Let some constant > be given. The symboic form of the FSS Eigenvaue probem combines an ordinary differentia equation (ODE) on the interva (, ) with a
More informationAppendix of the Paper The Role of No-Arbitrage on Forecasting: Lessons from a Parametric Term Structure Model
Appendix of the Paper The Roe of No-Arbitrage on Forecasting: Lessons from a Parametric Term Structure Mode Caio Ameida cameida@fgv.br José Vicente jose.vaentim@bcb.gov.br June 008 1 Introduction In this
More informationSupplement of Limited-memory Common-directions Method for Distributed Optimization and its Application on Empirical Risk Minimization
Suppement of Limited-memory Common-directions Method for Distributed Optimization and its Appication on Empirica Risk Minimization Ching-pei Lee Po-Wei Wang Weizhu Chen Chih-Jen Lin I Introduction In this
More informationMath 124B January 17, 2012
Math 124B January 17, 212 Viktor Grigoryan 3 Fu Fourier series We saw in previous ectures how the Dirichet and Neumann boundary conditions ead to respectivey sine and cosine Fourier series of the initia
More informationFUSED MULTIPLE GRAPHICAL LASSO
FUSED MULTIPLE GRAPHICAL LASSO SEN YANG, ZHAOSONG LU, XIAOTONG SHEN, PETER WONKA, JIEPING YE Abstract. In this paper, we consider the probem of estimating mutipe graphica modes simutaneousy using the fused
More informationPARSIMONIOUS VARIATIONAL-BAYES MIXTURE AGGREGATION WITH A POISSON PRIOR. Pierrick Bruneau, Marc Gelgon and Fabien Picarougne
17th European Signa Processing Conference (EUSIPCO 2009) Gasgow, Scotand, August 24-28, 2009 PARSIMONIOUS VARIATIONAL-BAYES MIXTURE AGGREGATION WITH A POISSON PRIOR Pierric Bruneau, Marc Gegon and Fabien
More informationSome Measures for Asymmetry of Distributions
Some Measures for Asymmetry of Distributions Georgi N. Boshnakov First version: 31 January 2006 Research Report No. 5, 2006, Probabiity and Statistics Group Schoo of Mathematics, The University of Manchester
More informationData Mining Technology for Failure Prognostic of Avionics
IEEE Transactions on Aerospace and Eectronic Systems. Voume 38, #, pp.388-403, 00. Data Mining Technoogy for Faiure Prognostic of Avionics V.A. Skormin, Binghamton University, Binghamton, NY, 1390, USA
More informationSoft Clustering on Graphs
Soft Custering on Graphs Kai Yu 1, Shipeng Yu 2, Voker Tresp 1 1 Siemens AG, Corporate Technoogy 2 Institute for Computer Science, University of Munich kai.yu@siemens.com, voker.tresp@siemens.com spyu@dbs.informatik.uni-muenchen.de
More informationBALANCING REGULAR MATRIX PENCILS
BALANCING REGULAR MATRIX PENCILS DAMIEN LEMONNIER AND PAUL VAN DOOREN Abstract. In this paper we present a new diagona baancing technique for reguar matrix pencis λb A, which aims at reducing the sensitivity
More informationAvailable online at ScienceDirect. IFAC PapersOnLine 50-1 (2017)
Avaiabe onine at www.sciencedirect.com ScienceDirect IFAC PapersOnLine 50-1 (2017 3412 3417 Stabiization of discrete-time switched inear systems: Lyapunov-Metzer inequaities versus S-procedure characterizations
More informationEvolutionary Product-Unit Neural Networks for Classification 1
Evoutionary Product-Unit Neura Networs for Cassification F.. Martínez-Estudio, C. Hervás-Martínez, P. A. Gutiérrez Peña A. C. Martínez-Estudio and S. Ventura-Soto Department of Management and Quantitative
More informationApproximated MLC shape matrix decomposition with interleaf collision constraint
Agorithmic Operations Research Vo.4 (29) 49 57 Approximated MLC shape matrix decomposition with intereaf coision constraint Antje Kiese and Thomas Kainowski Institut für Mathematik, Universität Rostock,
More informationThe EM Algorithm applied to determining new limit points of Mahler measures
Contro and Cybernetics vo. 39 (2010) No. 4 The EM Agorithm appied to determining new imit points of Maher measures by Souad E Otmani, Georges Rhin and Jean-Marc Sac-Épée Université Pau Veraine-Metz, LMAM,
More informationParagraph Topic Classification
Paragraph Topic Cassification Eugene Nho Graduate Schoo of Business Stanford University Stanford, CA 94305 enho@stanford.edu Edward Ng Department of Eectrica Engineering Stanford University Stanford, CA
More informationSVM-based Supervised and Unsupervised Classification Schemes
SVM-based Supervised and Unsupervised Cassification Schemes LUMINITA STATE University of Pitesti Facuty of Mathematics and Computer Science 1 Targu din Vae St., Pitesti 110040 ROMANIA state@cicknet.ro
More informationSymbolic models for nonlinear control systems using approximate bisimulation
Symboic modes for noninear contro systems using approximate bisimuation Giordano Poa, Antoine Girard and Pauo Tabuada Abstract Contro systems are usuay modeed by differentia equations describing how physica
More informationEffect of transport ratio on source term in determination of surface emission coefficient
Internationa Journa of heoretica & Appied Sciences, (): 74-78(9) ISSN : 975-78 Effect of transport ratio on source term in determination of surface emission coefficient Sanjeev Kumar and Apna Mishra epartment
More informationPrimal and dual active-set methods for convex quadratic programming
Math. Program., Ser. A 216) 159:469 58 DOI 1.17/s117-15-966-2 FULL LENGTH PAPER Prima and dua active-set methods for convex quadratic programming Anders Forsgren 1 Phiip E. Gi 2 Eizabeth Wong 2 Received:
More informationSTA 216 Project: Spline Approach to Discrete Survival Analysis
: Spine Approach to Discrete Surviva Anaysis November 4, 005 1 Introduction Athough continuous surviva anaysis differs much from the discrete surviva anaysis, there is certain ink between the two modeing
More informationDiscrete Techniques. Chapter Introduction
Chapter 3 Discrete Techniques 3. Introduction In the previous two chapters we introduced Fourier transforms of continuous functions of the periodic and non-periodic (finite energy) type, as we as various
More informationXSAT of linear CNF formulas
XSAT of inear CN formuas Bernd R. Schuh Dr. Bernd Schuh, D-50968 Kön, Germany; bernd.schuh@netcoogne.de eywords: compexity, XSAT, exact inear formua, -reguarity, -uniformity, NPcompeteness Abstract. Open
More informationApproximated MLC shape matrix decomposition with interleaf collision constraint
Approximated MLC shape matrix decomposition with intereaf coision constraint Thomas Kainowski Antje Kiese Abstract Shape matrix decomposition is a subprobem in radiation therapy panning. A given fuence
More informationWavelet shrinkage estimators of Hilbert transform
Journa of Approximation Theory 163 (2011) 652 662 www.esevier.com/ocate/jat Fu ength artice Waveet shrinkage estimators of Hibert transform Di-Rong Chen, Yao Zhao Department of Mathematics, LMIB, Beijing
More informationSparse canonical correlation analysis from a predictive point of view
Sparse canonica correation anaysis from a predictive point of view arxiv:1501.01231v1 [stat.me] 6 Jan 2015 Ines Wims Facuty of Economics and Business, KU Leuven and Christophe Croux Facuty of Economics
More informationInductive Bias: How to generalize on novel data. CS Inductive Bias 1
Inductive Bias: How to generaize on nove data CS 478 - Inductive Bias 1 Overfitting Noise vs. Exceptions CS 478 - Inductive Bias 2 Non-Linear Tasks Linear Regression wi not generaize we to the task beow
More informationConverting Z-number to Fuzzy Number using. Fuzzy Expected Value
ISSN 1746-7659, Engand, UK Journa of Information and Computing Science Vo. 1, No. 4, 017, pp.91-303 Converting Z-number to Fuzzy Number using Fuzzy Expected Vaue Mahdieh Akhbari * Department of Industria
More informationDiscrete Techniques. Chapter Introduction
Chapter 3 Discrete Techniques 3. Introduction In the previous two chapters we introduced Fourier transforms of continuous functions of the periodic and non-periodic (finite energy) type, we as various
More informationStructured sparsity for automatic music transcription
Structured sparsity for automatic music transcription O'Hanon, K; Nagano, H; Pumbey, MD; Internationa Conference on Acoustics, Speech and Signa Processing (ICASSP 2012) For additiona information about
More informationSequential Decoding of Polar Codes with Arbitrary Binary Kernel
Sequentia Decoding of Poar Codes with Arbitrary Binary Kerne Vera Miosavskaya, Peter Trifonov Saint-Petersburg State Poytechnic University Emai: veram,petert}@dcn.icc.spbstu.ru Abstract The probem of efficient
More informationMARKOV CHAINS AND MARKOV DECISION THEORY. Contents
MARKOV CHAINS AND MARKOV DECISION THEORY ARINDRIMA DATTA Abstract. In this paper, we begin with a forma introduction to probabiity and expain the concept of random variabes and stochastic processes. After
More informationA unified framework for Regularization Networks and Support Vector Machines. Theodoros Evgeniou, Massimiliano Pontil, Tomaso Poggio
MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES A.I. Memo No. 1654 March23, 1999
More informationA Ridgelet Kernel Regression Model using Genetic Algorithm
A Ridgeet Kerne Regression Mode using Genetic Agorithm Shuyuan Yang, Min Wang, Licheng Jiao * Institute of Inteigence Information Processing, Department of Eectrica Engineering Xidian University Xi an,
More informationAlgorithms to solve massively under-defined systems of multivariate quadratic equations
Agorithms to sove massivey under-defined systems of mutivariate quadratic equations Yasufumi Hashimoto Abstract It is we known that the probem to sove a set of randomy chosen mutivariate quadratic equations
More informationA proposed nonparametric mixture density estimation using B-spline functions
A proposed nonparametric mixture density estimation using B-spine functions Atizez Hadrich a,b, Mourad Zribi a, Afif Masmoudi b a Laboratoire d Informatique Signa et Image de a Côte d Opae (LISIC-EA 4491),
More informationSource and Relay Matrices Optimization for Multiuser Multi-Hop MIMO Relay Systems
Source and Reay Matrices Optimization for Mutiuser Muti-Hop MIMO Reay Systems Yue Rong Department of Eectrica and Computer Engineering, Curtin University, Bentey, WA 6102, Austraia Abstract In this paper,
More informationAvailable online at ScienceDirect. Procedia Computer Science 96 (2016 )
Avaiabe onine at www.sciencedirect.com ScienceDirect Procedia Computer Science 96 (206 92 99 20th Internationa Conference on Knowedge Based and Inteigent Information and Engineering Systems Connected categorica
More informationAppendix A: MATLAB commands for neural networks
Appendix A: MATLAB commands for neura networks 132 Appendix A: MATLAB commands for neura networks p=importdata('pn.xs'); t=importdata('tn.xs'); [pn,meanp,stdp,tn,meant,stdt]=prestd(p,t); for m=1:10 net=newff(minmax(pn),[m,1],{'tansig','purein'},'trainm');
More informationTarget Location Estimation in Wireless Sensor Networks Using Binary Data
Target Location stimation in Wireess Sensor Networks Using Binary Data Ruixin Niu and Pramod K. Varshney Department of ectrica ngineering and Computer Science Link Ha Syracuse University Syracuse, NY 344
More informationSpring Gravity Compensation Using the Noncircular Pulley and Cable For the Less-Spring Design
The 14th IFToMM Word Congress, Taipei, Taiwan, October 25-30, 2015 DOI Number: 10.6567/IFToMM.14TH.WC.PS3.010 Spring Gravity Compensation Using the Noncircuar Puey and Cabe For the Less-Spring Design M.C.
More informationHaar Decomposition and Reconstruction Algorithms
Jim Lambers MAT 773 Fa Semester 018-19 Lecture 15 and 16 Notes These notes correspond to Sections 4.3 and 4.4 in the text. Haar Decomposition and Reconstruction Agorithms Decomposition Suppose we approximate
More informationTwo view learning: SVM-2K, Theory and Practice
Two view earning: SVM-2K, Theory and Practice Jason D.R. Farquhar jdrf99r@ecs.soton.ac.uk Hongying Meng hongying@cs.york.ac.uk David R. Hardoon drh@ecs.soton.ac.uk John Shawe-Tayor jst@ecs.soton.ac.uk
More informationConstruction of Supersaturated Design with Large Number of Factors by the Complementary Design Method
Acta Mathematicae Appicatae Sinica, Engish Series Vo. 29, No. 2 (2013) 253 262 DOI: 10.1007/s10255-013-0214-6 http://www.appmath.com.cn & www.springerlink.com Acta Mathema cae Appicatae Sinica, Engish
More informationFactorized Sparse Learning Models with Interpretable High Order Feature Interactions
Factorized Sparse Learning Modes with Interpretabe High Order Feature Interactions Sanjay Purushotham University of Southern Caifornia Los Angees, CA, USA spurusho@usc.edu Martin Renqiang Min NEC Labs
More informationStochastic Automata Networks (SAN) - Modelling. and Evaluation. Paulo Fernandes 1. Brigitte Plateau 2. May 29, 1997
Stochastic utomata etworks (S) - Modeing and Evauation Pauo Fernandes rigitte Pateau 2 May 29, 997 Institut ationa Poytechnique de Grenobe { IPG Ecoe ationae Superieure d'informatique et de Mathematiques
More informationCryptanalysis of PKP: A New Approach
Cryptanaysis of PKP: A New Approach Éiane Jaumes and Antoine Joux DCSSI 18, rue du Dr. Zamenhoff F-92131 Issy-es-Mx Cedex France eiane.jaumes@wanadoo.fr Antoine.Joux@ens.fr Abstract. Quite recenty, in
More informationIncremental Reformulated Automatic Relevance Determination
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 60, NO. 9, SEPTEMBER 22 4977 Incrementa Reformuated Automatic Reevance Determination Dmitriy Shutin, Sanjeev R. Kukarni, and H. Vincent Poor Abstract In this
More informationarxiv: v2 [stat.ml] 19 Oct 2016
Sparse Quadratic Discriminant Anaysis and Community Bayes arxiv:1407.4543v2 [stat.ml] 19 Oct 2016 Ya Le Department of Statistics Stanford University ye@stanford.edu Abstract Trevor Hastie Department of
More informationConsistent linguistic fuzzy preference relation with multi-granular uncertain linguistic information for solving decision making problems
Consistent inguistic fuzzy preference reation with muti-granuar uncertain inguistic information for soving decision making probems Siti mnah Binti Mohd Ridzuan, and Daud Mohamad Citation: IP Conference
More informationNEW DEVELOPMENT OF OPTIMAL COMPUTING BUDGET ALLOCATION FOR DISCRETE EVENT SIMULATION
NEW DEVELOPMENT OF OPTIMAL COMPUTING BUDGET ALLOCATION FOR DISCRETE EVENT SIMULATION Hsiao-Chang Chen Dept. of Systems Engineering University of Pennsyvania Phiadephia, PA 904-635, U.S.A. Chun-Hung Chen
More informationpp in Backpropagation Convergence Via Deterministic Nonmonotone Perturbed O. L. Mangasarian & M. V. Solodov Madison, WI Abstract
pp 383-390 in Advances in Neura Information Processing Systems 6 J.D. Cowan, G. Tesauro and J. Aspector (eds), Morgan Kaufmann Pubishers, San Francisco, CA, 1994 Backpropagation Convergence Via Deterministic
More informationLearning Fully Observed Undirected Graphical Models
Learning Fuy Observed Undirected Graphica Modes Sides Credit: Matt Gormey (2016) Kayhan Batmangheich 1 Machine Learning The data inspires the structures we want to predict Inference finds {best structure,
More informationEquilibrium of Heterogeneous Congestion Control Protocols
Equiibrium of Heterogeneous Congestion Contro Protocos Ao Tang Jiantao Wang Steven H. Low EAS Division, Caifornia Institute of Technoogy Mung Chiang EE Department, Princeton University Abstract When heterogeneous
More informationExpectation-Maximization for Estimating Parameters for a Mixture of Poissons
Expectation-Maximization for Estimating Parameters for a Mixture of Poissons Brandon Maone Department of Computer Science University of Hesini February 18, 2014 Abstract This document derives, in excrutiating
More informationIterative Decoding Performance Bounds for LDPC Codes on Noisy Channels
Iterative Decoding Performance Bounds for LDPC Codes on Noisy Channes arxiv:cs/060700v1 [cs.it] 6 Ju 006 Chun-Hao Hsu and Achieas Anastasopouos Eectrica Engineering and Computer Science Department University
More informationAn Information Geometrical View of Stationary Subspace Analysis
An Information Geometrica View of Stationary Subspace Anaysis Motoaki Kawanabe, Wojciech Samek, Pau von Bünau, and Frank C. Meinecke Fraunhofer Institute FIRST, Kekuéstr. 7, 12489 Berin, Germany Berin
More informationCombining reaction kinetics to the multi-phase Gibbs energy calculation
7 th European Symposium on Computer Aided Process Engineering ESCAPE7 V. Pesu and P.S. Agachi (Editors) 2007 Esevier B.V. A rights reserved. Combining reaction inetics to the muti-phase Gibbs energy cacuation
More informationSupport Vector Machine and Its Application to Regression and Classification
BearWorks Institutiona Repository MSU Graduate Theses Spring 2017 Support Vector Machine and Its Appication to Regression and Cassification Xiaotong Hu As with any inteectua project, the content and views
More informationKernel Matching Pursuit
Kerne Matching Pursuit Pasca Vincent and Yoshua Bengio Dept. IRO, Université demontréa C.P. 6128, Montrea, Qc, H3C 3J7, Canada {vincentp,bengioy}@iro.umontrea.ca Technica Report #1179 Département d Informatique
More information