Privacy-Preserving Bayesian Network Parameter Learning

Transcription

1 4h WSEAS In. Conf. on COMUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS nd CYBERNETICS Mm, Flord, USA, November 7-9, 005 pp46-5) rvcy-reservng Byesn Nework rmeer Lernng JIANJIE MA. SIVAUMAR School of EECS, Wshngon Se Unversy, ullmn, WA Absrc: - rvcy s n mporn ssue n d mnng. Lernng Byesn nework BN) from prvcy sensve d hs been recen reserch opc. In hs pper, we propose o use pos rndomzon echnque o lern Byesn nework prmeers from dsrbued heerogeneous dbses. The only requred nformon from he d se s se of suffcen sscs for lernng Byesn nework prmeers. The proposed mehod esmes he suffcen sscs from he rndomzed d. We show boh heoreclly nd expermenlly h, even wh lrge level of rndomzon, our mehod cn lern he prmeers ccurely. ey-words: - Byesn Nework, rvcy-reservng D Mnng, Dsrbued Heerogeneous Dbses, os Rndomzon. Inroducon rvcy-preservng d mnng dels wh he problem of buldng ccure d mnng models over ggrege d, whle proecng prvcy he level of ndvdul records. There re wo mn pproches o prvcy-preservng d mnng. One pproch s o perurb or rndomze he d before sendng o he d mner. The perurbed or rndomzed d re hen used o lern or mne he models nd perns []. The oher pproch s o use secure mulpry compuon SMC) o enble wo or more pres o buld d models whou every pry lernng nyhng bou he oher pry s d [3]. rvcy-preservng Byesn nework BN) lernng s more recen opc. Wrgh nd Yng [9] dscuss prvcy-preservng BN srucure compuon on dsrbued heerogeneous dbses whle Meng e l. [7] hve consdered he prvcy-sensve BN prmeer lernng problem. The underlyng pproch n boh works s o conver he compuons requred for BN lernng no seres of nner produc compuons nd hen o use secure nner produc compuon mehod. The number of secure compuon operons ncreses exponenlly wh he possble confgurons of he problem vrbles. The curren work on prvcy-preservng BN lernng focuses on he mulpry mode whch requres h every pry hve some bly o compue. Besdes hs mode our pper consders model where here s d mner who cully does ll he compuons on behlf of he prcpng pres. SMC mehod hs he followng wo drwbcks: ) ssumes sem-hones mode whch s ofen unrelsc n he rel world ) requres lrge volumes of synchronzed compuons mong prcpng pres. Mos of he synchronzed compuons re overheds due o prvcy requremens. os rndomzon overcomes he drwbcks of SMC mehod by some rde off beween ccurcy nd prvcy. A mlcous pry who does no obey he proocol n SMC mehod cn esly ge some prve nformon of oher pres whle no pry s ble o ge exc prve nformon of oher pres f pos rndomzons re mplemened o ndvdul d records.. roblem Formulon The prvcy-preservng BN lernng nvolves dsrbued dbses, where he dbse s owned by severl pres. If he dbse s homogeneously dsrbued, prvcy-preservng BN Lernng s relvely esy snce every pry cn send d mner or oher pres) he se of suffcen sscs from hs pr of he dbse. rvcy of ndvdul records wll no be breched by sendng suffcen

2 4h WSEAS In. Conf. on COMUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS nd CYBERNETICS Mm, Flord, USA, November 7-9, 005 pp46-5) sscs o oher pres or d mner. The problem of prvcy-preservng BN lernng from heerogeneous dbse s h severl pres who ech own vercl poron of he dbse wn o lern globl BN for her muul benefs bu hey re concerned bou he prvcy of her sensve vrbles. In hs pper, we consder he BN prmeer lernng problem for he dscree vrble cse. Exensons o BN srucure lernng re possble nd would be repored n fuure publcon. We consder he followng wo models. Model I: There s no d mner; every pry hs o do some poron of lernng compuons, whch corresponds o he mul-pry model of SMC. Every pry sends her rndomzed d o hose pres who need hose d. Model II: There s d mner who does ll compuons for he prcpng pres. Ech pry smply sends ll s rndomzed d o he d mner. 3. rvcy Qunzon Consder dbse D wh n vrbles Λ }, { n where kes dscree vlues from he se S. The pos rndomzon for vrble s rndom) mppng R : S S, bsed on se of rnson probbles p = p = k = k ), where k lm m, kl S nd m l denoes he rndomzed) vrble vlue correspondng o vrble. The rnson probbly p lm s he probbly h vrble wh orgnl vlue k l s rndomzed o he vlue k m. os Rndomzon s so nmed becuse he rndomzon hppens fer d hve been colleced. Le = { p lm} denoe he mrx h hs p s s l, m) h enry, where s he crdnly of lm S. The condon h s nonsngulr hs o be mposed f we wn o esme he frequency dsrbuon of vrble from he rndomzed vrbles. In he followng, we gve ou some smple bu effecve pos rndomzon schemes on whch our expermens re bsed. If vrble kes bnry vlues, we cn use Bnry Rndomzon s shown n Fg. ). If he vrble s ernry, ernry symmerc chnnel s shown n Fg. b) cn be used. Fg. Rndomzon Schemes We cn pply he sme rndomzon schemes ndependenly o ll of he vrbles: unform rndomzon o he d se. Alernvely, we cn use non-unform rndomzon where dfferen pos rndomzon schemes re ppled o dfferen vrbles ndependenly. The non-unform rndomzon s effecve when dfferen vrbles hve dfferen sensvy levels. For exmple, we cn choose dfferen rndomzon prmeers p nd p o dfferen bnry vrbles for non-unform rndomzon f he prvcy requremens of he wo vrbles re dfferen. The non-unform rndomzon ncludes he specl cse when here s no prvcy requremen for some of he vrbles. From he bove, we cn see h f vrble kes vlues or cegores), he dmenson of wll p p p p ) p p p be. Wh lrger, more rndomzon s nroduced no vrble n generl. Ths s good from prvcy pon of vew. However, he vrnce of he esmor for frequency couns wll lso be lger under he sme smple sze. One soluon for hs problem s o pron he cegores no severl groups such h vlue n one group cn only be rndomzed o vlue n he sme group. In hs cse, mrx becomes block dgonl mrx. The problem of how mny groups should he vlues be proned no s mer of 3 p b) 3

3 4h WSEAS In. Conf. on COMUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS nd CYBERNETICS Mm, Flord, USA, November 7-9, 005 pp46-5) desgn choce. The pos rndomzon cn lso be mplemened o severl vrbles smulneously. For exmple, he vrbles nd j cn be rndomzed smulneously ccordng o rnson probbly p = l, = l = k, ). j j = k Rndomzng vrbles smulneously cn vod he possble nconssency of he dbse cused by rndomzon. We consder he noon of prvcy nroduced by Evfmevsk e l. [4] n erms of n mplfcon fcor γ. In [4], he mplfcon γ s proposed n frmework where every d record should be rndomzed wh fcor greer hn γ, before he d re sen o he d mner, o lm prvcy brech. However, n hs pper, we use mplfcon γ purely s wors-cse qunfcon of prvcy for desgned pos rndomzon scheme. I s proved n [4] h f he rndomzon operor s mos γ mplfyng, revelng = k wll cuse neher n upwrd ρ -o- ρ prvcy brech nor downwrd ρ -o- ρ prvcy brech f ρ ρ ρ ρ > γ. Clerly, he smller he vlue of γ, he beer s he wors cse prvcy. Idelly we would lke o hve γ =. The mos γ mplfcon provdes wors cse qunfcon of prvcy. However, does no provde ny nformon of prvcy n generl. Besdes γ, we use = mn #{ k = k' = k) > 0}, whch s he k mnmum number of possble cegores h cn be rndomzed o cegory k ' n desgned pos rndomzon, s noher qunfcon of prvcy. Ths ndces he prvcy preserved n generl. I s smlr o he defned n -nonymy n [8] bu n probblsc sense. If we group he cegores of vrble no severl groups, hen becomes smller n generl 4. rmeer Lernng Frmework For prmeer lernng, we ssume he srucure G s fxed nd known o every prcpng pry. For Model I, we use he defnon of cross vrble nd cross prens defned n [3]. records such h prens re n jh cegory. For ech pry : N jk s he number of s n kh cegory whle s ) Rndomze cross prens belong o s own pry ccordng o her respecve prvcy requremens usng pos rndomzon descrbed n Secon 3. Rndomzons re done ndependenly for ech combned) vrble nd ech record. ) Send rndomzed cross prens of pry o pry for pry j j ogeher wh he probbly rnson mrx used. 3) Lern prmeers for locl vrbles n pry. Ths sep does no nvolve rndomzed d. 4) Esme he suffcen sscs s own pry N jk s for ech cross vrble belongng o usng locl d nd rndomzed pren d from oher pres. 5) Compue he prmeers for cross vrbles usng he esmed suffcen sscs jk s. 6) Shre he prmeers wh ll oher pres. Vrbles n one pry re no rndomzed for s own clculons. Seps of lernng prmeers for Model II: For ech pry : ) Rndomze ll s sensve vrbles ccordng o her respecve prvcy requremens usng pos rndomzon descrbed n Secon 3. Rndomzons re done ndependenly for ech combned) vrble nd ech record. ) Send rndomzed d nd her correspondng probbly rnson mrces o he d mner. For he d mner: ) Esme he suffcen sscs N jk for ech

4 4h WSEAS In. Conf. on COMUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS nd CYBERNETICS Mm, Flord, USA, November 7-9, 005 pp46-5) node usng he rndomzed d from prcped pres. ) Esme he prmeers usng he esmed suffcen sscs. 3) Brodcs he prmeers o ll pres. The dels of esmon of suffcen sscs nd prmeer sep 4 nd 5 for Model I, Sep nd for d mner n Model II) from rndomzed d re descrbed n Secon Esmon of BN rmeers The problem of prvcy-preservng BN rmeer lernng cn be decomposed no se of esmon of N jk s for ech node nd gven fxed srucure G from he rndomzed d D. Consder he followng generl cse: Vrble wh crdnly hs Q pren nodes ), Λ, Q). The crdnly of q) s q) jk. These vrbles cn be rbrry verclly proned o dfferen pres n boh models. The rndomzon of ech combned) vrble cn lso be done by groupng he cegores of he vrbles no groups. We hve he followng dfferen cses for esmng N jk s from he rndomzed d D due o smulneous rndomzon: ) nd s prens re ll rndomzed ndependenly ech oher. b) Some prens of re rndomzed smulneously. c) s rndomzed smulneously wh some of s prens. d) s rndomzed smulneously wh non-pren vrbles. For b) nd c) bove, we cn consder he smulneously rndomzed vrbles s combned vrbles n esmng he suffcen sscs. For exmple, f vrble s rndomzed smulneously wh one of s prens ), N jk s equl o he number of records such h ; )) = k, j ), Q Q) j, where ; )) = ) = j,, s combned rndomzed d by consderng ; )) s sngle vrble wh crdnly ). For cse d), snce he curren N jk does no nvolve he vrble rndomzed smulneously wh, he lerner cn ge he mrgnl rnson probbly mrx from he gven rnson mrx of he combned vrble. From he bove rgumens, we conclude h he cses b), c), nd d) bove cn effecvely be consdered o be equvlen o cse ). Hence, whou loss of generly, we cn dscuss cse ) only. We denoe by ) s compound vrble for ll he prens of Vrble ) kes J = = N j N jk k = nd Q q= N s q). Hence dfferen vlues. J dmensonl vecor of N jk vlues, h s N ) = N, N, Λ, N, N, Λ NJ, where superscrp denoes mrx rnspose. N l) s n elemen of N. N jk, N j, nd N re defned smlrly s N jk, N j, nd N, respecvely bu for he rndomzed d D. esmors of jk, j, nd re N jk, N j, nd N respecvely. Gven he rnng d D wh N records of vrbles nd s Q prens n he bove generl cse, f hey re pos-rndomzed wh probbly rnson mrces, ),, respecvely, we hve he followng heorem. Theorem : E [ N D] = N, where Q) vrble. Thus, we cn esme he N jk s from he

5 4h WSEAS In. Conf. on COMUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS nd CYBERNETICS Mm, Flord, USA, November 7-9, 005 pp46-5) p = nd p p ) p Q) = Λ, denoes ronecker mrx produc. Moreover, J Cov[ N D] = N l) Vl, where Vl s l = J J mrx wh l, ) h elemen l l) l)) l = l V l l ) =. l) l ) l l roofs re omed here due o he pge lmons. Ineresed reders cn refer o longer verson of hs pper for dels [6]. The followng heorem esblshes he bs nd vrnce of he esmor N ˆ = ) N. Is proof s srgh-forwrd nd s omed. Theorem : N ˆ = ) N s n unbsed esmor for N nd Cov{ D} = ) Cov{ N D} ), where nd Cov{ N D} re s defned n Theorem. We use he esmed suffcen sscs o ge ML jk jk esme of he prmeer s θˆ ML jk = = j j j k = nd he MA esme of he prmeer s jk ˆ α + jk θ jk =, where we ssume he pror α + dsrbuon of θ j s Drchle wh prmeer α, α, Λ, α }. { j j jr 6. Expermenl Resuls In hs expermen, we use he Byesn Nework shown n Fg., where he vrbles re dsrbued over hree ses. All vrbles re bnry excep jk vrbles L nd B whch re ernry. The condonl probbles of he dfferen nodes re lso shown smples were genered from hs Byesn Nework o form he dse D. Ths d ws hen rndomzed ccordng o he scheme descrbed n Tble, where vrbles T, S, nd G were consdered no sensve nd hence no rndomzed. The correspondng mos γ mplfcon s lso shown n Tble. = for Bnry rndomzon whle =3 for ernry rndomzon. Tble shows pr of prmeers lern from he rndomzed d usng he lgorhm descrbed n Secon 4 for model II. The remnng pr cn be clculed by one mnus he gven pr. All he vlues n he Tble re verge over 5 runs, wh he correspondng sndrd devon ndced n prenhess. I s cler from he Tble h he proposed lgorhms cn ccurely lern he BN prmeers for boh scenros, even for modere levels of rndomzon. A T Se F E C A 0.7,0.3 T 0.,0.9,0.9,0. S 0.5,0.5 L 0.3,0.7,0.4,0.5,0.3,0.5 0.,0.6,0.8,0.4 F 0.5,0.9,0.75,0. E 0.5,0.8,0.5,0.5,0.3,0.4,0.75,0.,0.85,0.5,0.7,0.6 D 0.7,0.65,0.,0.4,0.8,0.35,0.3,0.35,0.9,0.6,0.,0.65 C 0.9,0.4,0.6,0.5,0.,0.6,0.4,0.75 B 0.8,0.5,0.,0.5,0.,0.35 G 0.,0.4,0.8,0.6 Fg. A Byesn Nework Less rndomzon occurs n Model I, so he resuls for Model I were beer hn hose for model II. We presen only he resuls for model II here. L Se 3 S Se D G B

6 4h WSEAS In. Conf. on COMUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS nd CYBERNETICS Mm, Flord, USA, November 7-9, 005 pp46-5) A,D Bnry symmerc p = p = 0. 5, γ = 3 L,B Ternry symmerc p = p = 0. 5 γ = E Bnry symmerc = p = 0. p γ = 4 Bnry symmerc = p = 0. C,F Bnry = 0. p γ = 4 p p = 0. 5 γ = 9 Tble : Rndomzon performed A ) T ) ) S ) )0.60.) L ) ) ) ) B )0.60.4) ) ) E ) )0.4.7)0.5.) 0.3.6)0.4.34) D 0.69.)0.65.3)0.3.3) )0.79.7) ) C )0.38.6)0.6.6)0.5.) F )0.9.) G ) ) Tble : Men nd sndrd devon 0 - ) over 5 runs of prmeers lern from he rndomzed d. 7. Concluson We hve proposed pos rndomzon echnque o lern prmeers of Byesn nework from dsrbued heerogeneous d. Our mehod esmes he suffcen sscs from he rndomzed d, whch re subsequenly used o lern he prmeers. Our expermens show h he pos rndomzon s n effcen, flexble nd esy-o-use mehod o lern Byesn nework from prvcy sensve d. os Rndomzon mehod cn be esly upgrded o lern BN Srucures from sensve d. I s cler h cn lso be ppled o rvcy-reservng decson ree lernng. References: [] D. Agrwl nd C. C. Aggrwl. On he Desgn nd Qunfcon of rvcy reservng D Mnng Algorhm, SIGMOD 00. [] R. Chen,. Svkumr, nd H. rgup, Collecve Mnng of Byesn Neworks from Dsrbued Heerogeneous D, nowledge nd Informon Sysems Journ vol. 6, 004. [3] C. Clfon, M. nrcoglu, J. Vdy,. Ln, nd M. Zhu. Tools for rvcy reservng Dsrbued D mnng. ACM SIGDD Explorons, vol. 4) 003. [4] A. Evfmevsk, J. Gehrke, nd R. Srkn. Lmng prvcy breches n prvcy preservng d mnng. In proceedngs of he ACM SIGMOD/OD Conference, June 003. [5] J. M. Gouweleeuw,. oomn, L.C.R.J. Wllenborg, nd.-. de Wolf. os Rndomson for Sscl Dsclosure Conrol: Theory nd Implemenon. Journl of offcl Sscs, Vol.4, 998. [6] J. M nd. Svkumr, rvcy-reservng Byesn Nework Lernng Usng os Rndomzon, n prepron), 005. [7] D. Meng,. Svkumr nd H. rgup. rvcy-sensve Byesn Nework rmeer Lernng. In he Fourh IEEE Inernonl Conference on D Mnng, 004. [8] L. Sweeney. k-nonymy: A model for proecng prvcy. Inernonl Journl on uncerny, Fuzzness nd nowledge-bsed Sysems, vol. 05):5 00. [9] R. Wrgh nd Z. Yng. rvcy reservng Byesn Nework Srucure Compuon on Dsrbued Heerogeneous D. roceedngs of he ACM SIGDD, 004.