A Fundamental Storage-Communication Tradeoff in Distributed Computing with Straggling Nodes
|
|
- Paulina Bridges
- 5 years ago
- Views:
Transcription
1 A Fundamenta Storage-Communication Tradeoff in Distributed Computing with Stragging odes ifa Yan, Michèe Wigger LTCI, Téécom ParisTech Paris, France Emai: {qifa.yan, Sheng Yang L2S, CentraeSupéec Gif-sur-Yvette, France Emai: Xiaohu Tang Information Security and ationa Computing Grid Laboratory, Southwest Jiaotong University, , Chengdu, Sichuan, China Emai: arxiv: v1 [cs.it 23 Jan 2019 Abstract The optima storage-computation tradeoff is characterized for a MapReduce-ike distributed computing system with stragging nodes, where ony a part of the nodes can be utiized to compute the desired output functions. The resut hods for arbitrary output functions and thus generaizes previous resuts that restricted to inear functions. Specificay, in this work, we propose a new information-theoretica converse and a new matching coded computing scheme, that we ca coded computing for stragging systems CCS. I. ITRODUCTIO Distributed computing has emerged as one of the most important paradigms to speed up arge-scae data anaysis tasks such as machine earning. A we-known computing framework, MapReduce, can dea with tasks with arge data size. In such systems, the tasks are typicay decomposed into computing map and reduce functions, where the map functions can be computed by different nodes across the network, and the fina outputs are computed by combining the outputs of the map functions by means of reduce functions. Such architectures can be used, for exampe, to perform earning tasks in neura networks [1. Recenty, Li et a. proposed a scheme named coded distributed computing CDC, which through cever coding reduces the communication oad for data shuffing between the map and reduce phases [2. The CDC scheme achieves the optima storage-communication tradeoff, i.e., the smaest communication oad under a tota memory constraint. More recenty, this resut was extended in [3, [4 to account aso for the computation oad during the map phase. In this paper, we consider a variant of this probem where each node takes a random time to compute its desired map functions. In this case, waiting that a nodes have finished their computation can be too time consuming. Instead, the data shuffing and reduce computations are started as soon as any set of nodes, for a constant in {1,..., }, and the number of tota nodes, has terminated the map computations. One of the difficuties in such systems is that when assigning the map computations to the nodes, it is yet uncear which set of nodes wi perform the data shuffing and compute the reduce functions. In this sense, the assignment of map computations needs to be convenient irrespective of the set of nodes that computes the reduce functions. This setup with stragging nodes, hereafter referred to as the stragging system, has been introduced in [5. That work used maximum separabe codes MDS to set up coded computing. Simiar codes were aso used in [6, [7, and it was shown that they achieve the order optima storage-communcation tradeoff in such stragging systems when the goba task is to compute inear functions. Many important tasks in practice, such as computations in neura networks, are however non-inear. This motivates us to investigate the MapReduce framework with stragging nodes for genera map and reduce functions. In this work, we competey characterize the optima storage-communcation tradeoff for systems with stragging nodes. This optima tradeoff is achieved by our new scheme that we name coded computing for stragging systems CCS. We aso present an information-theoretica converse matching our new CCS scheme. Reated to the present work are aso distributed computing schemes with stragging nodes for master-worker network modes, where the worker nodes perform the map computations and the master node the reduce computations. Specificay, [8 [10 proposed coding schemes for systems with inear map and reduce functions and [11 proposed coding schemes for cacuating the derivative of ikeihood functions. otations: For positive integers n, k where k n, we wi use the notations C k n n! k!nk!, [n {1, 2,..., n}, and [k : n {k, k + 1,..., n}. The binary fied is denoted by F 2 and the n dimensiona vector space over F 2 is denoted by F n 2. The operator is used in the foowing way: for a set A, we use A to denote its cardinaity, whie for a signa X, we use X to denote its ength measured in bits. The notation Pr{} is used to denote probabiity of an event. II. SYSTEM MODEL Let,,, D, U, V, W be positive integers. Consider a system that aims to compute D output functions through distributed computing nodes, denoted by {1,..., }. Each output function φ d d [D takes a fies in the ibrary W {w 1,..., w } as inputs, where w n is a fie of size W bits, n [, and outputs a bit stream of ength U, i.e., u d φ d w 1,..., w F U 2, 1
2 where φ d : F W 2 F U 2. Assume that the computation of each output function φ d can be decomposed as: where φ d w 1,..., w h d f d,1 w 1,..., f d, w, 2 The map function f d,n : F W 2 F V 2 maps the fie w n into a binary stream of ength V, caed intermediate vaues IVA, i.e., v d,n f d,n w n F V 2, n [. 3 The reduce function h d : F V 2 F U 2 maps the IVAs {v d,n } n1 into the output stream u d h d v d,1,..., v d,. 4 Hence, the computation is carried out through three phases. 1 Map Phase: Each node k stores a subset of fies M k W, and tries to compute a the IVAs from the fies in M k, denoted by C k : C k {v d,n : d [D, w n M k }. 5 Each node has a random time to compute its corresponding IVAs. To imit atency of the system, the distributed computing scheme proceeds with the shuffe and reduce phase as soon as a fixed number of nodes has terminated the map computations. This subset of nodes wi be caed active set. otice that is a fixed system parameter. Moreover, for simpicity we assume that each subset of size is active with same probabiity. Define Ω t {T : T t}, t [, 6 and et denote the active set. Then, Pr { } 1 C, Ω. 7 The output functions φ 1,..., φ D are then uniformy assigned to the nodes in. Denote the indices of the output functions assigned to a given node k by D k. 1 Thus, D A {D k } k A forms a partition of [D, and each D k is of cardinaity D. Denote the set of a uniform partitions of [D by A. 2 Shuffe Phase: The nodes in proceed to exchange their computed IVAs. Each node k muticasts a signa X A k ϕ k C k, D A 8 to a other nodes in, where ϕ k : F C k V 2 A F XA 2 is the encoding function of node k. Thus each active node k receives the signas X A { } X A k : k errorfree. 3 Reduce Phase: With the received signas X A exchanged during the shuffe phase and the IVAs C k computed ocay during the map phase, node k restores a the IVAs {v d,1,..., v d, } d Dk k ψ k X, C k, D A, 9 1 Here we assume for simpicity that divides D. ote that otherwise we can aways add empty functions for the assumption to hod. where ψ k k X : F A k 2 F C k V 2 A F DV Subsequenty, it proceeds to compute 2. u d h d v d,1,..., v d,, d D k. 10 Remark 1: otice that a decomposition into map and reduce functions is aways possibe. In fact, triviay, one can set the map and reduce functions to be the identity and output functions respectivey, i.e., f d,n w n w n, and h d φ d, n [, d [D, in which case V W. However, to mitigate the communication cost in shuffe phase, one woud prefer a decomposition such that the ength of the IVAs is sma but suffices to compute the fina outputs. To measure the storage and computation costs, we introduce the foowing definitions: Definition 1 Storage Space: Storage space r is defined as the tota number of fies stored across the nodes normaized by the tota number of fies : k1 r M k. 11 Definition 2 Communication Load: Communication oad L is defined as the average tota number of bits sent in the shuffe phase, normaized by the tota number of bits of a intermediated vaues, i.e., L E [ k X A DV k, 12 where the expectation is taken over the active set. Definition 3 Storage-Communication Tradeoff: A pair of rea numbers r, L is feasibe if for any ɛ > 0, there exists an integer, a storage design {M k } k1 of storage space ess than r + ɛ, a set of uniform assignment of output functions {D } Ω, a coection of encoding functions {{ k} } ϕ with communication oad ess than L+ɛ, k Ω such that a the output functions φ 1,..., φ D can be computed successfuy. For a fixed [, we define the storagecommunication tradeoff as L r inf {L : r, L is achievabe}. 13 The goa of this paper is to characterize L r for r [ + 1, for each [. otice that, the interesting interva for r is [ + 1,. In fact, for r >, each node can triviay store a the fies. Moreover, a pair r, L can ony be feasibe if each fie is stored at east once in any subset of nodes. If r < + 1 then this is not possibe. III. MAI RESULT We summarize our main resut in the foowing theorem. Theorem 1: For a distributed computing system with nodes and any fixed [, for an integer storage space r [ + 1 :, L r 1 r min{r,1} 1 C r C 1 r1 C r+ 1
3 For a genera r [ +1,, L r is given by the ower convex enveope formed by the above points. Theorem 1 characterizes the storage-communication tradeoff { for a positive integers. Fig. 1 shows the curves L r : [ } for 10. otice that, when 1, the curve reduces to a singe point, 0, whie when, the curve corresponds to the optima tradeoff without stragging node obtained in [2. In fact, in the case, our proposed scheme becomes the CDC scheme in [2. For any j and any T Ω,r with j / T, eveny spit the IVAs V T,j into disjoint packets that we abe as {VT i,j : i T }. So, V T,j {V i T,j : i T }. 19 Each node k then muticasts the bit-wise XOR XT k VT k \{j} {k},j, T Ω,r s.t. k / T. 20 j T ode k can compute a these signas because it can compute a IVAs { V T,j : T Ω,r and k T }, see Reduce Phase: Any given node k can recover a IVAs {V T,k : T Ω r, k T } 21 ocay, see again 16. To obtain the missing IVAs {V T,k : T Ω r, k / T }, 22 Fig. 1. Storage-Communication Tradeoff L r for [ when 10. IV. ACHIEVABILITY In this section, we wi first present our proposed scheme, and then anayze its performance to prove the achievabiity part of Theorem 1. A. Coded Computing for Stragging Systems CCS Let r be an integer vaue in [ + 1 :. Consider a mutipe of C r. Partition the fies into Cr batches, each containing η r C fies. Each batch is then associated with r a subset T of of cardinaity r. Let W T denote the batch of η r fies associated with set T. Then, W {w 1,..., w } W T. 15 T Ω r We now describe the map, shuffe, and reduce procedures of the CCS scheme. 1 Map Phase: Each node k stores batches W T such that k T, i.e., M k W T, 16 T :T Ω r, k T and tries to compute a IVAs in 5. 2 Shuffe Phase: Let be the active set. Pick any uniform assignment of output functions D {D k } k. For any subset T of size r and any k, et V T,k {v d,n : d D k, w n W T }. 17 The shuffe phase is accompished in min{+1, r} rounds, where each round is indexed by an integer [ r + : min{r, 1}. Consider round, where we define a subsets of of size r with exacty active nodes: Ω,r {T : T r, T }. 18 it again proceeds in rounds r +,..., min{r, 1}. In round, it forms for each T Ω,r s.t. k / T and j T the IVA packet V j T,k V j T \{i} {k},i Xj T \{j} {k}. 23 otice that, i T \{j} 1 ode k can form the signa 23 since it obtained the signa X j T \{j} {k} from node j, and it can compute the IVAs { V j T \{i} {k},i : i T \{j} } are ocay. 2 ode k can decode a the IVAs in 22, since for any T Ω r such that k / T, r + T min{r, 1}, and thus, T must be an eement of Ω,r, for some [r + : min{r, 1}. Finay, it computes the output functions through 10. Remark 2: By 16, the storage design {M k } k1 does not depend on the parameter but ony on the avaiabe storage space r. The proposed storage design is thus universay optima irrespective of the size of the active set. In practice, the map phase can thus be carried out even without knowing how many nodes wi be participating in the reduce phase. B. Performance Anaysis By 16, each node k stores C r1 1 batches, each of which containing η r fies. Thus, the storage space is k1 M k 1 r1 ηr r. 24 When the active set is, denote the communication oad of the CCS scheme by L. For each [ r+ : min{r, 1}, by 19 and 20, there are in tota C Cr signas sent in the round, each of size ηr D V bits. ote that L does not depend on the reaization of the active set
4 but ony on its size. Hence, the average communication oad is L E [L 25 min{r,1} r+ C Cr ηr D DV V 26 1 r min{r,1} 1 C r C 1 r1 C r+ 1 We have proved the achievabiity when r [ + 1 :. For genera r [ + 1,, the ower convex enveope can be achieved with memory- and time- sharing. I : I s V. COVERSE Let {M k } k1 be a storage design and r, L be a storagecommunication pair achieved based on {M k } k1. For each s [ + 1 :, define M k M k, 28 k I k \I i.e., is the number of fies stored s times across a the nodes. Then by definition, satisfies I : I s+1 s+1 For any Ω and any [, define b M, M k k I 0, 29, 30 s r. 31 k \I M k, 32 i.e., b M, is the number of fies stored exacty times in the nodes of. Since any fie that is stored s times across the a nodes, has occurrences in exacty Cs C s subsets of Ω, we have Ω b M, + smax{,+1} C s C s. 33 Consider the case that the active set is, a the output functions are computed through the nodes in. For the sub-system with computing nodes in, under the storage {M k } k, the optima communication oad L M, has the foowing ower bound by [2, Lemma 1: L M, 1 b M,. 34 Then the communication oad [ k L E X k 35 DV [ k E X k DV Pr{ } 36 Ω L M, Pr{ } 37 Ω 1 C 1 a C 1 Ω 1 1 s+1 Ω + b M, b M, smax{,+1} min{s,} s+ C sc s C C sc s, 41 C where a foows from 33. Let Z x be the function defined over the interva [ + 1, by connecting the points s, Z s sequentiay, where min{s,} Z s s+ C s C s, 42 is defined for any s [ + 1 :. In the Appendix, we wi prove the foowing caim. Caim 1: The function Z x is a convex decreasing function over [ + 1,. a By 29 and 30, M,s are coefficients between 0 and 1 whose summation is 1. Hence by 41 and Caim 1, we have L 1 C 1 C Z s+1 Z s 43 s 44 s+1 Z r C. 45 In particuar, when r [ + 1 :, L min{r,} +r C r C r C 46 1 r min{r,1} 1 C r C 1 r1 C r+ 1 As a resut, by 45, 47 and Caim 1, we have proved the ower bound for any r [ + 1,.
5 ACOWLEDGMET The work of. Yan and M. Wigger has been supported by the ERC under grant agreement APPEDIX PROOF OF CLAIM 1 { We prove Caim 1 by verifying that, the sequence Z s} is a discrete monotonicay decreasing s+1 convex sequence, i.e, Z s + 1 Z s < 0, 48 Z s + 1 Z s > Z s Z s otice that by 42, min{s,} Z s a s+ min{s,} s+ C s C s min{s,} s+ C s C s 50 Cs C s C, 51 where in a, we used the equaity min{s,} s+ C s C s C. Thus the constant C shows up in both Z k s + 1 and s for a s [ + 1 : 1, then Z k a Z s + 1 Z s 52 min{s+1,} s+1+ min{s+1,} s+1+ min{s+1,} s+1+ min{s,1} b s+ min{s,} s+ min{s,1} s+ C s+1c s1 min{s,} s+ C s + Cs 1 C s1 Cs C s1 + C1 s1 Cs 1 C min{s,1} s1 s+ CsC 1 min{s,1} s1 + 1 s+ C sc 1 s1 + 1 C sc s CsC 1 s1 55 C sc 1 s < 0, 58 where in a, we appied the equaity Cn+1 m+1 Cm+1 n + Cn m and in b, we used the variabe change 1 in the first summation. Moreover, foowing simiar steps as above, by 57, for s [ + 2 : 1, we can derive Z s + 1 Z s Z s Z s 1 59 min{s,}1 +s1 min{s,}1 +s1 C s1c 1 s + 1 C s1 min{s,1} +s min{s1,2} +s1 min{s,1} +s min{s,1} s+ min{s,1} +s C 1 s1 + C2 s1 + 1 CsC 1 s1 + 1 C s1 + Cs1 1 C 1 s1 + 1 Cs1C 2 min{s,1} s1 C s s1 C1 s1 Cs1 1 C1 min{s,1} s1 C s 2C 1 s1 C1 s s1 C1 s > We have proved 48 and 49. As a resut, the piecewise inear function Z x connecting points { s, Z s} s+1 sequentiay is a convex function. REFERECES [1 Y. Liu, J. Yang, Y. Huang, L. Xu, S. Li, and M. i, MapReduce based parae neura networks in enabing arge scae machine earning," Comput. Inte. eurosci., [2 S. Li, M. A. Maddah-Ai,. Yu, and A. S. Avestimehr, A fundamenta tradeoff between computation and communication in distributed computing," IEEE Trans. Inf. Theory, vo. 64, no. 1, pp , Jan [3. Yan, S. Yang, and M. Wigger, Storage, computation, and communication: A fundamenta tradeoff in distributed computing," arxiv: [4 Y. H. Ezzedin, M. armoose, and C. Fragoui, Communication vs distributed computation: An aternative trade-off curve," in Proc. IEEE Inf. Theory Workshop ITW, aohsiung, Taiwan, pp , ov [5. Lee, M. Lam, R. Pedarsani, D. Papaiiopouos, and. Ramchandran, Speeding up distributed machine earning using codes," IEEE Trans. Inf. Theory, vo. 64, no. 3, pp , Mar [6 S. Li, M. A. Maddah-Ai, and A. S. Avestimehr, A unified coding framework for distributed computing with stragging servers," in Proc. IEEE Gobecom Works GC Wkshps, Washington, DC, USA, Dec [7 J. Zhang and O. Simeone, Improved atency-communication trade-off for map-shuffe-reduce systems with straggers," arxiv: [8. Yu, M. Maddah-Ai, and S. Avestimehr, Poynomia codes: An optima design for high-dimensiona coded matrix mutipication," in Proc. The 31st Annua Conf. eura Inf. Processing System IPS, Long Beach, CA, USA, May [9. Yu, M. Maddah-Ai, and S. Avestimehr, Stragger mitigation in distributed matrix mutipication: Fundamenta imits and optima coding," in Proc. IEEE Int. Symp. Inf. Theory, Vai, CO, USA, pp , Jun [10 T. Baharav,. Lee, O. Oca, and. Ramchandran, Stragger-proofing massive-scae distributed matrix mutipication with d-dimensiona product codes," in Proc. IEEE Int. Symp. Inf. Theory, Vai, CO, USA, pp , Jun [11. Raviv, I. Tamo, R. Tandon, and A. G. Dimakis, Gradient coding from cycic MDS codes and expander graphs," arxiv:
Centralized Coded Caching of Correlated Contents
Centraized Coded Caching of Correated Contents Qianqian Yang and Deniz Gündüz Information Processing and Communications Lab Department of Eectrica and Eectronic Engineering Imperia Coege London arxiv:1711.03798v1
More informationUnconditional security of differential phase shift quantum key distribution
Unconditiona security of differentia phase shift quantum key distribution Kai Wen, Yoshihisa Yamamoto Ginzton Lab and Dept of Eectrica Engineering Stanford University Basic idea of DPS-QKD Protoco. Aice
More informationarxiv: v1 [math.fa] 23 Aug 2018
An Exact Upper Bound on the L p Lebesgue Constant and The -Rényi Entropy Power Inequaity for Integer Vaued Random Variabes arxiv:808.0773v [math.fa] 3 Aug 08 Peng Xu, Mokshay Madiman, James Mebourne Abstract
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part IX The EM agorithm In the previous set of notes, we taked about the EM agorithm as appied to fitting a mixture of Gaussians. In this set of notes, we give a broader view
More informationNearly Optimal Constructions of PIR and Batch Codes
arxiv:700706v [csit] 5 Jun 07 Neary Optima Constructions of PIR and Batch Codes Hia Asi Technion - Israe Institute of Technoogy Haifa 3000, Israe shea@cstechnionaci Abstract In this work we study two famiies
More informationA Brief Introduction to Markov Chains and Hidden Markov Models
A Brief Introduction to Markov Chains and Hidden Markov Modes Aen B MacKenzie Notes for December 1, 3, &8, 2015 Discrete-Time Markov Chains You may reca that when we first introduced random processes,
More informationIterative Decoding Performance Bounds for LDPC Codes on Noisy Channels
Iterative Decoding Performance Bounds for LDPC Codes on Noisy Channes arxiv:cs/060700v1 [cs.it] 6 Ju 006 Chun-Hao Hsu and Achieas Anastasopouos Eectrica Engineering and Computer Science Department University
More informationCache Aided Wireless Networks: Tradeoffs between Storage and Latency
Cache Aided Wireess Networks: Tradeoffs between Storage and Latency Avik Sengupta Hume Center, Department of ECE Virginia Tech, Backsburg, VA 24060, USA Emai: aviksg@vt.edu Ravi Tandon Department of ECE
More informationLimited magnitude error detecting codes over Z q
Limited magnitude error detecting codes over Z q Noha Earief choo of Eectrica Engineering and Computer cience Oregon tate University Corvais, OR 97331, UA Emai: earief@eecsorstedu Bea Bose choo of Eectrica
More informationMARKOV CHAINS AND MARKOV DECISION THEORY. Contents
MARKOV CHAINS AND MARKOV DECISION THEORY ARINDRIMA DATTA Abstract. In this paper, we begin with a forma introduction to probabiity and expain the concept of random variabes and stochastic processes. After
More informationEfficient Generation of Random Bits from Finite State Markov Chains
Efficient Generation of Random Bits from Finite State Markov Chains Hongchao Zhou and Jehoshua Bruck, Feow, IEEE Abstract The probem of random number generation from an uncorreated random source (of unknown
More informationAppendix A: MATLAB commands for neural networks
Appendix A: MATLAB commands for neura networks 132 Appendix A: MATLAB commands for neura networks p=importdata('pn.xs'); t=importdata('tn.xs'); [pn,meanp,stdp,tn,meant,stdt]=prestd(p,t); for m=1:10 net=newff(minmax(pn),[m,1],{'tansig','purein'},'trainm');
More informationLinear Network Coding for Multiple Groupcast Sessions: An Interference Alignment Approach
Linear Network Coding for Mutipe Groupcast Sessions: An Interference Aignment Approach Abhik Kumar Das, Siddhartha Banerjee and Sriram Vishwanath Dept. of ECE, The University of Texas at Austin, TX, USA
More informationCodes between MBR and MSR Points with Exact Repair Property
Codes between MBR and MSR Points with Exact Repair Property 1 Toni Ernva arxiv:1312.5106v1 [cs.it] 18 Dec 2013 Abstract In this paper distributed storage systems with exact repair are studied. A construction
More informationImproved Converses and Gap-Results for Coded Caching
Improved Converses and Gap-Resuts for Coded Caching Chien-Yi Wang, Shirin Saeedi Bidokhti, and Michèe Wigger Abstract arxiv:702.04834v [cs.it] 6 Feb 207 Improved ower bounds on the average and the worst-case
More informationTarget Location Estimation in Wireless Sensor Networks Using Binary Data
Target Location stimation in Wireess Sensor Networks Using Binary Data Ruixin Niu and Pramod K. Varshney Department of ectrica ngineering and Computer Science Link Ha Syracuse University Syracuse, NY 344
More informationSeparation of Variables and a Spherical Shell with Surface Charge
Separation of Variabes and a Spherica She with Surface Charge In cass we worked out the eectrostatic potentia due to a spherica she of radius R with a surface charge density σθ = σ cos θ. This cacuation
More informationCoded Caching for Files with Distinct File Sizes
Coded Caching for Fies with Distinct Fie Sizes Jinbei Zhang iaojun Lin Chih-Chun Wang inbing Wang Department of Eectronic Engineering Shanghai Jiao ong University China Schoo of Eectrica and Computer Engineering
More informationASummaryofGaussianProcesses Coryn A.L. Bailer-Jones
ASummaryofGaussianProcesses Coryn A.L. Baier-Jones Cavendish Laboratory University of Cambridge caj@mrao.cam.ac.uk Introduction A genera prediction probem can be posed as foows. We consider that the variabe
More informationRate-Distortion Theory of Finite Point Processes
Rate-Distortion Theory of Finite Point Processes Günther Koiander, Dominic Schuhmacher, and Franz Hawatsch, Feow, IEEE Abstract We study the compression of data in the case where the usefu information
More informationCONGRUENCES. 1. History
CONGRUENCES HAO BILLY LEE Abstract. These are notes I created for a seminar tak, foowing the papers of On the -adic Representations and Congruences for Coefficients of Moduar Forms by Swinnerton-Dyer and
More informationArbitrary Throughput Versus Complexity Tradeoffs in Wireless Networks using Graph Partitioning
University of Pennsyvania SchoaryCommons Departmenta Papers (ESE) Department of Eectrica & Systems Engineering November 2006 Arbitrary Throughput Versus Compexity Tradeoffs in Wireess Networks using Graph
More informationMONTE CARLO SIMULATIONS
MONTE CARLO SIMULATIONS Current physics research 1) Theoretica 2) Experimenta 3) Computationa Monte Caro (MC) Method (1953) used to study 1) Discrete spin systems 2) Fuids 3) Poymers, membranes, soft matter
More informationImproved Min-Sum Decoding of LDPC Codes Using 2-Dimensional Normalization
Improved Min-Sum Decoding of LDPC Codes sing -Dimensiona Normaization Juntan Zhang and Marc Fossorier Department of Eectrica Engineering niversity of Hawaii at Manoa Honouu, HI 968 Emai: juntan, marc@spectra.eng.hawaii.edu
More informationThreshold Circuits for Multiplication and Related Problems
Optima-Depth Threshod Circuits for Mutipication and Reated Probems Chi-Hsiang Yeh Dept. of Eectrica & Computer Engineering Queen s University Kingston, Ontario, Canada, K7K 3N6 E.A. Varvarigos, B. Parhami,
More informationEfficiently Generating Random Bits from Finite State Markov Chains
1 Efficienty Generating Random Bits from Finite State Markov Chains Hongchao Zhou and Jehoshua Bruck, Feow, IEEE Abstract The probem of random number generation from an uncorreated random source (of unknown
More informationDistributed average consensus: Beyond the realm of linearity
Distributed average consensus: Beyond the ream of inearity Usman A. Khan, Soummya Kar, and José M. F. Moura Department of Eectrica and Computer Engineering Carnegie Meon University 5 Forbes Ave, Pittsburgh,
More informationMat 1501 lecture notes, penultimate installment
Mat 1501 ecture notes, penutimate instament 1. bounded variation: functions of a singe variabe optiona) I beieve that we wi not actuay use the materia in this section the point is mainy to motivate the
More informationAsynchronous Control for Coupled Markov Decision Systems
INFORMATION THEORY WORKSHOP (ITW) 22 Asynchronous Contro for Couped Marov Decision Systems Michae J. Neey University of Southern Caifornia Abstract This paper considers optima contro for a coection of
More informationA proposed nonparametric mixture density estimation using B-spline functions
A proposed nonparametric mixture density estimation using B-spine functions Atizez Hadrich a,b, Mourad Zribi a, Afif Masmoudi b a Laboratoire d Informatique Signa et Image de a Côte d Opae (LISIC-EA 4491),
More informationTight Approximation Algorithms for Maximum Separable Assignment Problems
MATHEMATICS OF OPERATIONS RESEARCH Vo. 36, No. 3, August 011, pp. 416 431 issn 0364-765X eissn 156-5471 11 3603 0416 10.187/moor.1110.0499 011 INFORMS Tight Approximation Agorithms for Maximum Separabe
More informationHaar Decomposition and Reconstruction Algorithms
Jim Lambers MAT 773 Fa Semester 018-19 Lecture 15 and 16 Notes These notes correspond to Sections 4.3 and 4.4 in the text. Haar Decomposition and Reconstruction Agorithms Decomposition Suppose we approximate
More informationAlgorithms to solve massively under-defined systems of multivariate quadratic equations
Agorithms to sove massivey under-defined systems of mutivariate quadratic equations Yasufumi Hashimoto Abstract It is we known that the probem to sove a set of randomy chosen mutivariate quadratic equations
More informationHow the backpropagation algorithm works Srikumar Ramalingam School of Computing University of Utah
How the backpropagation agorithm works Srikumar Ramaingam Schoo of Computing University of Utah Reference Most of the sides are taken from the second chapter of the onine book by Michae Nieson: neuranetworksanddeepearning.com
More informationAge of Information: The Gamma Awakening
Age of Information: The Gamma Awakening Eie Najm and Rajai Nasser LTHI, EPFL, Lausanne, Switzerand Emai: {eie.najm, rajai.nasser}@epf.ch arxiv:604.086v [cs.it] 5 Apr 06 Abstract Status update systems is
More informationSteepest Descent Adaptation of Min-Max Fuzzy If-Then Rules 1
Steepest Descent Adaptation of Min-Max Fuzzy If-Then Rues 1 R.J. Marks II, S. Oh, P. Arabshahi Λ, T.P. Caude, J.J. Choi, B.G. Song Λ Λ Dept. of Eectrica Engineering Boeing Computer Services University
More informationRelated Topics Maxwell s equations, electrical eddy field, magnetic field of coils, coil, magnetic flux, induced voltage
Magnetic induction TEP Reated Topics Maxwe s equations, eectrica eddy fied, magnetic fied of cois, coi, magnetic fux, induced votage Principe A magnetic fied of variabe frequency and varying strength is
More informationA. Distribution of the test statistic
A. Distribution of the test statistic In the sequentia test, we first compute the test statistic from a mini-batch of size m. If a decision cannot be made with this statistic, we keep increasing the mini-batch
More informationUniprocessor Feasibility of Sporadic Tasks with Constrained Deadlines is Strongly conp-complete
Uniprocessor Feasibiity of Sporadic Tasks with Constrained Deadines is Strongy conp-compete Pontus Ekberg and Wang Yi Uppsaa University, Sweden Emai: {pontus.ekberg yi}@it.uu.se Abstract Deciding the feasibiity
More informationarxiv: v1 [cs.db] 25 Jun 2013
Communication Steps for Parae Query Processing Pau Beame, Paraschos Koutris and Dan Suciu {beame,pkoutris,suciu}@cs.washington.edu University of Washington arxiv:1306.5972v1 [cs.db] 25 Jun 2013 June 26,
More informationReliability: Theory & Applications No.3, September 2006
REDUNDANCY AND RENEWAL OF SERVERS IN OPENED QUEUING NETWORKS G. Sh. Tsitsiashvii M.A. Osipova Vadivosto, Russia 1 An opened queuing networ with a redundancy and a renewa of servers is considered. To cacuate
More informationError-free Multi-valued Broadcast and Byzantine Agreement with Optimal Communication Complexity
Error-free Muti-vaued Broadcast and Byzantine Agreement with Optima Communication Compexity Arpita Patra Department of Computer Science Aarhus University, Denmark. arpita@cs.au.dk Abstract In this paper
More informationFRIEZE GROUPS IN R 2
FRIEZE GROUPS IN R 2 MAXWELL STOLARSKI Abstract. Focusing on the Eucidean pane under the Pythagorean Metric, our goa is to cassify the frieze groups, discrete subgroups of the set of isometries of the
More informationMinimizing Total Weighted Completion Time on Uniform Machines with Unbounded Batch
The Eighth Internationa Symposium on Operations Research and Its Appications (ISORA 09) Zhangiaie, China, September 20 22, 2009 Copyright 2009 ORSC & APORC, pp. 402 408 Minimizing Tota Weighted Competion
More informationAn Algorithm for Pruning Redundant Modules in Min-Max Modular Network
An Agorithm for Pruning Redundant Modues in Min-Max Moduar Network Hui-Cheng Lian and Bao-Liang Lu Department of Computer Science and Engineering, Shanghai Jiao Tong University 1954 Hua Shan Rd., Shanghai
More informationOn colorings of the Boolean lattice avoiding a rainbow copy of a poset arxiv: v1 [math.co] 21 Dec 2018
On coorings of the Booean attice avoiding a rainbow copy of a poset arxiv:1812.09058v1 [math.co] 21 Dec 2018 Baázs Patkós Afréd Rényi Institute of Mathematics, Hungarian Academy of Scinces H-1053, Budapest,
More informationarxiv:quant-ph/ v3 6 Jan 1995
arxiv:quant-ph/9501001v3 6 Jan 1995 Critique of proposed imit to space time measurement, based on Wigner s cocks and mirrors L. Diósi and B. Lukács KFKI Research Institute for Partice and Nucear Physics
More informationBayesian Learning. You hear a which which could equally be Thanks or Tanks, which would you go with?
Bayesian Learning A powerfu and growing approach in machine earning We use it in our own decision making a the time You hear a which which coud equay be Thanks or Tanks, which woud you go with? Combine
More informationOptimality of Inference in Hierarchical Coding for Distributed Object-Based Representations
Optimaity of Inference in Hierarchica Coding for Distributed Object-Based Representations Simon Brodeur, Jean Rouat NECOTIS, Département génie éectrique et génie informatique, Université de Sherbrooke,
More informationMelodic contour estimation with B-spline models using a MDL criterion
Meodic contour estimation with B-spine modes using a MDL criterion Damien Loive, Ney Barbot, Oivier Boeffard IRISA / University of Rennes 1 - ENSSAT 6 rue de Kerampont, B.P. 80518, F-305 Lannion Cedex
More informationImproved Approximation of Storage-Rate Tradeoff for Caching via New Outer Bounds
Improved Approximation of Storage-Rate Tradeoff for Caching via New Outer Bound Avik Sengupta, Ravi Tandon, T. Chare Cancy Hume Center & Dept. of Eectrica and Computer Engineering Dicovery Anaytic Center
More informationSchedulability Analysis of Deferrable Scheduling Algorithms for Maintaining Real-Time Data Freshness
1 Scheduabiity Anaysis of Deferrabe Scheduing Agorithms for Maintaining Rea- Data Freshness Song Han, Deji Chen, Ming Xiong, Kam-yiu Lam, Aoysius K. Mok, Krithi Ramamritham UT Austin, Emerson Process Management,
More informationDownlink Transmission of Short Packets: Framing and Control Information Revisited
1 Downin ransmission of Short Pacets: Frag and Contro Information Revisited Kasper Føe riingsgaard, Student Member, IEEE and Petar Popovsi, Feow, IEEE arxiv:1605.01829v1 cs.i 6 May 2016 Abstract Ceuar
More informationIEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 15, NO. 2, FEBRUARY
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 5, NO. 2, FEBRUARY 206 857 Optima Energy and Data Routing in Networks With Energy Cooperation Berk Gurakan, Student Member, IEEE, OmurOze,Member, IEEE,
More informationMIXING AUTOMORPHISMS OF COMPACT GROUPS AND A THEOREM OF SCHLICKEWEI
MIXING AUTOMORPHISMS OF COMPACT GROUPS AND A THEOREM OF SCHLICKEWEI KLAUS SCHMIDT AND TOM WARD Abstract. We prove that every mixing Z d -action by automorphisms of a compact, connected, abeian group is
More informationInductive Bias: How to generalize on novel data. CS Inductive Bias 1
Inductive Bias: How to generaize on nove data CS 478 - Inductive Bias 1 Overfitting Noise vs. Exceptions CS 478 - Inductive Bias 2 Non-Linear Tasks Linear Regression wi not generaize we to the task beow
More informationA Solution to the 4-bit Parity Problem with a Single Quaternary Neuron
Neura Information Processing - Letters and Reviews Vo. 5, No. 2, November 2004 LETTER A Soution to the 4-bit Parity Probem with a Singe Quaternary Neuron Tohru Nitta Nationa Institute of Advanced Industria
More informationStochastic Automata Networks (SAN) - Modelling. and Evaluation. Paulo Fernandes 1. Brigitte Plateau 2. May 29, 1997
Stochastic utomata etworks (S) - Modeing and Evauation Pauo Fernandes rigitte Pateau 2 May 29, 997 Institut ationa Poytechnique de Grenobe { IPG Ecoe ationae Superieure d'informatique et de Mathematiques
More information4 1-D Boundary Value Problems Heat Equation
4 -D Boundary Vaue Probems Heat Equation The main purpose of this chapter is to study boundary vaue probems for the heat equation on a finite rod a x b. u t (x, t = ku xx (x, t, a < x < b, t > u(x, = ϕ(x
More informationPrivate Secure Coded Computation
Private Secure Coded Computation Minchul Kim and Jungwoo Lee Seoul National University Department of Electrical and Computer Engineering 08826 Seoul, Korea Email: kmc1222@cml.snu.ac.kr, junglee@snu.ac.kr
More informationApplied Nuclear Physics (Fall 2006) Lecture 7 (10/2/06) Overview of Cross Section Calculation
22.101 Appied Nucear Physics (Fa 2006) Lecture 7 (10/2/06) Overview of Cross Section Cacuation References P. Roman, Advanced Quantum Theory (Addison-Wesey, Reading, 1965), Chap 3. A. Foderaro, The Eements
More informationRecursive Constructions of Parallel FIFO and LIFO Queues with Switched Delay Lines
Recursive Constructions of Parae FIFO and LIFO Queues with Switched Deay Lines Po-Kai Huang, Cheng-Shang Chang, Feow, IEEE, Jay Cheng, Member, IEEE, and Duan-Shin Lee, Senior Member, IEEE Abstract One
More informationResearch Article On the Lower Bound for the Number of Real Roots of a Random Algebraic Equation
Appied Mathematics and Stochastic Anaysis Voume 007, Artice ID 74191, 8 pages doi:10.1155/007/74191 Research Artice On the Lower Bound for the Number of Rea Roots of a Random Agebraic Equation Takashi
More informationHow to Optimally Allocate Resources for Coded Distributed Computing?
1 How to Optimally Allocate Resources for Coded Distributed Computing? Qian Yu, Songze Li, Mohammad Ali Maddah-Ali, and A. Salman Avestimehr Department of Electrical Engineering, University of Southern
More informationMulticasting Energy and Information Simultaneously
Muticasting Energy and Information Simutaneousy Ting-Yi Wu, Anshoo Tandon, Lav R. Varshney, and Mehu Motani Sun Yat-Sen University, wutingyi@mai.sysu.edu.cn Nationa University of Singapore, {anshoo.tandon@gmai.com,
More informationRestricted weak type on maximal linear and multilinear integral maps.
Restricted weak type on maxima inear and mutiinear integra maps. Oscar Basco Abstract It is shown that mutiinear operators of the form T (f 1,..., f k )(x) = R K(x, y n 1,..., y k )f 1 (y 1 )...f k (y
More informationData Search Algorithms based on Quantum Walk
Data Search Agorithms based on Quantum Wak Masataka Fujisaki, Hiromi Miyajima, oritaka Shigei Abstract For searching any item in an unsorted database with items, a cassica computer takes O() steps but
More informationCryptanalysis of PKP: A New Approach
Cryptanaysis of PKP: A New Approach Éiane Jaumes and Antoine Joux DCSSI 18, rue du Dr. Zamenhoff F-92131 Issy-es-Mx Cedex France eiane.jaumes@wanadoo.fr Antoine.Joux@ens.fr Abstract. Quite recenty, in
More informationPattern Frequency Sequences and Internal Zeros
Advances in Appied Mathematics 28, 395 420 (2002 doi:10.1006/aama.2001.0789, avaiabe onine at http://www.ideaibrary.com on Pattern Frequency Sequences and Interna Zeros Mikós Bóna Department of Mathematics,
More informationC. Fourier Sine Series Overview
12 PHILIP D. LOEWEN C. Fourier Sine Series Overview Let some constant > be given. The symboic form of the FSS Eigenvaue probem combines an ordinary differentia equation (ODE) on the interva (, ) with a
More informationSequential Decoding of Polar Codes with Arbitrary Binary Kernel
Sequentia Decoding of Poar Codes with Arbitrary Binary Kerne Vera Miosavskaya, Peter Trifonov Saint-Petersburg State Poytechnic University Emai: veram,petert}@dcn.icc.spbstu.ru Abstract The probem of efficient
More informationPower Control and Transmission Scheduling for Network Utility Maximization in Wireless Networks
ower Contro and Transmission Scheduing for Network Utiity Maximization in Wireess Networks Min Cao, Vivek Raghunathan, Stephen Hany, Vinod Sharma and. R. Kumar Abstract We consider a joint power contro
More information$, (2.1) n="# #. (2.2)
Chapter. Eectrostatic II Notes: Most of the materia presented in this chapter is taken from Jackson, Chap.,, and 4, and Di Bartoo, Chap... Mathematica Considerations.. The Fourier series and the Fourier
More informationTransport Cost and Optimal Number of Public Facilities
Transport Cost an Optima Number of Pubic Faciities Kazuo Yamaguchi Grauate Schoo of Economics, University of Tokyo June 14, 2006 Abstract We consier the number an ocation probem of pubic faciities without
More informationTraffic data collection
Chapter 32 Traffic data coection 32.1 Overview Unike many other discipines of the engineering, the situations that are interesting to a traffic engineer cannot be reproduced in a aboratory. Even if road
More informationOn the Achievable Extrinsic Information of Inner Decoders in Serial Concatenation
On the Achievabe Extrinsic Information of Inner Decoders in Seria Concatenation Jörg Kiewer, Axe Huebner, and Danie J. Costeo, Jr. Department of Eectrica Engineering, University of Notre Dame, Notre Dame,
More information8 Digifl'.11 Cth:uits and devices
8 Digif'. Cth:uits and devices 8. Introduction In anaog eectronics, votage is a continuous variabe. This is usefu because most physica quantities we encounter are continuous: sound eves, ight intensity,
More informationA Simple and Efficient Algorithm of 3-D Single-Source Localization with Uniform Cross Array Bing Xue 1 2 a) * Guangyou Fang 1 2 b and Yicai Ji 1 2 c)
A Simpe Efficient Agorithm of 3-D Singe-Source Locaization with Uniform Cross Array Bing Xue a * Guangyou Fang b Yicai Ji c Key Laboratory of Eectromagnetic Radiation Sensing Technoogy, Institute of Eectronics,
More informationLecture Note 3: Stationary Iterative Methods
MATH 5330: Computationa Methods of Linear Agebra Lecture Note 3: Stationary Iterative Methods Xianyi Zeng Department of Mathematica Sciences, UTEP Stationary Iterative Methods The Gaussian eimination (or
More informationOn the Goal Value of a Boolean Function
On the Goa Vaue of a Booean Function Eric Bach Dept. of CS University of Wisconsin 1210 W. Dayton St. Madison, WI 53706 Lisa Heerstein Dept of CSE NYU Schoo of Engineering 2 Metrotech Center, 10th Foor
More informationNew Efficiency Results for Makespan Cost Sharing
New Efficiency Resuts for Makespan Cost Sharing Yvonne Beischwitz a, Forian Schoppmann a, a University of Paderborn, Department of Computer Science Fürstenaee, 3302 Paderborn, Germany Abstract In the context
More informationStructural Control of Probabilistic Boolean Networks and Its Application to Design of Real-Time Pricing Systems
Preprints of the 9th Word Congress The Internationa Federation of Automatic Contro Structura Contro of Probabiistic Booean Networks and Its Appication to Design of Rea-Time Pricing Systems Koichi Kobayashi
More informationStatistical Learning Theory: A Primer
Internationa Journa of Computer Vision 38(), 9 3, 2000 c 2000 uwer Academic Pubishers. Manufactured in The Netherands. Statistica Learning Theory: A Primer THEODOROS EVGENIOU, MASSIMILIANO PONTIL AND TOMASO
More informationGlobal Optimality Principles for Polynomial Optimization Problems over Box or Bivalent Constraints by Separable Polynomial Approximations
Goba Optimaity Principes for Poynomia Optimization Probems over Box or Bivaent Constraints by Separabe Poynomia Approximations V. Jeyakumar, G. Li and S. Srisatkunarajah Revised Version II: December 23,
More informationThe Binary Space Partitioning-Tree Process Supplementary Material
The inary Space Partitioning-Tree Process Suppementary Materia Xuhui Fan in Li Scott. Sisson Schoo of omputer Science Fudan University ibin@fudan.edu.cn Schoo of Mathematics and Statistics University of
More informationFRST Multivariate Statistics. Multivariate Discriminant Analysis (MDA)
1 FRST 531 -- Mutivariate Statistics Mutivariate Discriminant Anaysis (MDA) Purpose: 1. To predict which group (Y) an observation beongs to based on the characteristics of p predictor (X) variabes, using
More informationThe Streaming-DMT of Fading Channels
The Streaming-DMT of Fading Channes Ashish Khisti Member, IEEE, and Star C. Draper Member, IEEE arxiv:30.80v3 cs.it] Aug 04 Abstract We consider the sequentia transmission of a stream of messages over
More informationc 2016 Georgios Rovatsos
c 2016 Georgios Rovatsos QUICKEST CHANGE DETECTION WITH APPLICATIONS TO LINE OUTAGE DETECTION BY GEORGIOS ROVATSOS THESIS Submitted in partia fufiment of the requirements for the degree of Master of Science
More informationConstruct non-graded bi-frobenius algebras via quivers
Science in China Series A: Mathematics Mar., 2007, Vo. 50, No. 3, 450 456 www.scichina.com www.springerink.com Construct non-graded bi-frobenius agebras via quivers Yan-hua WANG 1 &Xiao-wuCHEN 2 1 Department
More informationA Statistical Framework for Real-time Event Detection in Power Systems
1 A Statistica Framework for Rea-time Event Detection in Power Systems Noan Uhrich, Tim Christman, Phiip Swisher, and Xichen Jiang Abstract A quickest change detection (QCD) agorithm is appied to the probem
More information#A48 INTEGERS 12 (2012) ON A COMBINATORIAL CONJECTURE OF TU AND DENG
#A48 INTEGERS 12 (2012) ON A COMBINATORIAL CONJECTURE OF TU AND DENG Guixin Deng Schoo of Mathematica Sciences, Guangxi Teachers Education University, Nanning, P.R.China dengguixin@ive.com Pingzhi Yuan
More informationExpectation-Maximization for Estimating Parameters for a Mixture of Poissons
Expectation-Maximization for Estimating Parameters for a Mixture of Poissons Brandon Maone Department of Computer Science University of Hesini February 18, 2014 Abstract This document derives, in excrutiating
More informationThe basic equation for the production of turbulent kinetic energy in clouds is. dz + g w
Turbuence in couds The basic equation for the production of turbuent kinetic energy in couds is de TKE dt = u 0 w 0 du v 0 w 0 dv + g w q 0 q 0 e The first two terms on the RHS are associated with shear
More informationXSAT of linear CNF formulas
XSAT of inear CN formuas Bernd R. Schuh Dr. Bernd Schuh, D-50968 Kön, Germany; bernd.schuh@netcoogne.de eywords: compexity, XSAT, exact inear formua, -reguarity, -uniformity, NPcompeteness Abstract. Open
More informationJackson 4.10 Homework Problem Solution Dr. Christopher S. Baird University of Massachusetts Lowell
Jackson 4.10 Homework Probem Soution Dr. Christopher S. Baird University of Massachusetts Lowe PROBLEM: Two concentric conducting spheres of inner and outer radii a and b, respectivey, carry charges ±.
More informationSchedulability Analysis of Deferrable Scheduling Algorithms for Maintaining Real-Time Data Freshness
1 Scheduabiity Anaysis of Deferrabe Scheduing Agorithms for Maintaining Rea-Time Data Freshness Song Han, Deji Chen, Ming Xiong, Kam-yiu Lam, Aoysius K. Mok, Krithi Ramamritham UT Austin, Emerson Process
More informationBP neural network-based sports performance prediction model applied research
Avaiabe onine www.jocpr.com Journa of Chemica and Pharmaceutica Research, 204, 6(7:93-936 Research Artice ISSN : 0975-7384 CODEN(USA : JCPRC5 BP neura networ-based sports performance prediction mode appied
More informationHeavy-traffic Delay Optimality in Pull-based Load Balancing Systems: Necessary and Sufficient Conditions
Heavy-traffic Deay Optimaity in Pu-based Load Baancing Systems: Necessary and Sufficient Conditions XINGYU ZHOU, The Ohio State University JIAN TAN, The Ohio State University NSS SHROFF, The Ohio State
More informationParagraph Topic Classification
Paragraph Topic Cassification Eugene Nho Graduate Schoo of Business Stanford University Stanford, CA 94305 enho@stanford.edu Edward Ng Department of Eectrica Engineering Stanford University Stanford, CA
More informationhttps://doi.org/ /epjconf/
HOW TO APPLY THE OPTIMAL ESTIMATION METHOD TO YOUR LIDAR MEASUREMENTS FOR IMPROVED RETRIEVALS OF TEMPERATURE AND COMPOSITION R. J. Sica 1,2,*, A. Haefee 2,1, A. Jaai 1, S. Gamage 1 and G. Farhani 1 1 Department
More information