Group-based active query selection. for rapid diagnosis in time-critical situations

Transcription

1 Group-bsed ctve query selecton for rpd dgnoss n tme-crtcl stutons *Gowthm Belll, Student Member, IEEE, Suresh K. Bhvnn, nd Clyton Scott, Member, IEEE Abstrct In pplctons such s ctve lernng or dsese/fult dgnoss, one often encounters the problem of dentfyng n unknown object whle mnmzng the number of yes or no questons (queres) posed bout tht object. Ths problem hs been commonly referred to s query lernng (wth membershp queres) or object/entty dentfcton n the lterture. We consder three extensons of ths fundmentl problem tht re motvted by prctcl consdertons n rel-world, tme-crtcl dentfcton tsks such s emergency response. Frst, we consder the problem where the objects re prttoned nto groups, nd the gol s to dentfy only the group to whch the object belongs. Second, we ddress the stuton where the queres re prttoned nto groups, nd n lgorthm my suggest group of queres to humn user, who then selects the ctul query. Thrd, we consder the problem of object dentfcton n the presence of persstent query nose, nd relte t to group dentfcton. To ddress these problems we show tht stndrd lgorthm for object dentfcton, known s the splttng lgorthm or generlzed bnry serch, my be vewed s generlzton of Shnnon-Fno codng. We then extend ths result to the group-bsed settngs, ledng to new lgorthms. The performnce of our greedy lgorthms s demonstrted through logrthmc pproxmton bound, nd through experments wth smulted dt nd dtbse used by frst responders for toxc chemcl dentfcton. Index Terms Actve lernng, decson trees, Shnnon-Fno codng, Generlzed bnry serch, persstent nose, submodulrty. G. Belll nd C. Scott re wth the Deprtment of Electrcl Engneerng nd Computer Scence, Unversty of Mchgn, Ann Arbor, nd S. K. Bhvnn s wth the Insttute for Trnsltonl Scences, Unversty of Texs Medcl Brnch, Glveston, TX. E-ml: gowthm@umch.edu, skbhvnn@gml.com, clyscot@umch.edu. Ths work ws supported n prt by NSF Awrds No nd , NIH Grnt No. ULRR nd CDC/NIOSH grnt No. R2 OH A2.

2 2 I. INTRODUCTION In emergency response pplctons, s well s other tme-crtcl dgnostc tsks, there s need to rpdly dentfy cuse by selectvely cqurng nformton from the envronment. For exmple, n the problem of toxc chemcl dentfcton, frst responder my queston vctms of chemcl exposure regrdng the symptoms they experence. Chemcls tht re nconsstent wth the reported symptoms my then be elmnted. Becuse of the mportnce of ths problem, severl orgnztons hve constructed extensve evdence-bsed dtbses (e.g., Hz-Mp ) tht record toxc chemcls nd the cute symptoms whch they re known to cuse. Unfortuntely, mny symptoms tend to be nonspecfc (e.g., vomtng cn be cused by mny dfferent chemcls), nd t s therefore crtcl for the frst responder to pose these questons n sequence tht leds to chemcl dentfcton n s few questons s possble. Ths problem hs been studed from mthemtcl perspectve for decdes, nd hs been descrbed vrously s query lernng (wth membershp queres) [], ctve lernng [2], object/entty dentfcton [3], [4], nd bnry testng [4], [5]. In ths work we refer to the problem s object dentfcton. The stndrd mthemtcl formulton of object dentfcton s often delzed reltve to mny rel-world dgnostc tsks, n tht t does not ccount for tme constrnts nd resultng nput errors. In ths pper we nvestgte lgorthms tht extend object dentfcton to such more relstc settngs by ddressng the need for rpd response, nd error-tolernt lgorthms. In these problems, there s set Θ = {θ,, θ M } of M dfferent objects nd set Q = {q,, q N } of N dstnct subsets of Θ known s queres. An unknown object θ s generted from ths set Θ wth certn pror probblty dstrbuton Π = (π,, π M ),.e., π = Pr(θ = θ ). The gol s to determne the unknown object θ Θ through s few queres from Q s possble, where query q Q returns vlue f θ q, nd 0 otherwse. An object dentfcton lgorthm thus corresponds to decson tree, where the nternl nodes re queres, nd the lef nodes re objects. Problems of ths nture rse n pplctons such s fult testng [6], [7], mchne dgnostcs [8], dsese dgnoss [5], [9], computer vson [0], [], pool-bsed ctve lernng [2], [2], [3] nd the dptve trvelng slesperson problem [4]. Algorthms nd performnce gurntees hve been extensvely developed n the lterture, s descrbed n Secton I-A below. In the context of toxc chemcl dentfcton, the objects re chemcls, nd the queres re symptoms. An object dentfcton lgorthm wll prompt the frst responder wth symptom. Once the presence or bsence of tht symptom s determned, new symptom s suggested by the lgorthm, nd so on,

3 3 untl the chemcl s unquely determned. In ths pper, we consder vrtons on ths bsc object dentfcton frmework tht re motvted by toxc chemcl dentfcton, nd re nturlly pplcble to other tme-crtcl dgnostc tsks. In prtculr, we develop theoretcl results nd new lgorthms for wht mght be descrbed s group-bsed ctve lernng. Frst, we consder the cse where Θ s prttoned nto groups of objects, nd t s only necessry to dentfy the group to whch the unknown object belongs. For exmple, the pproprte response to toxc chemcl my only depend on the clss of chemcls to whch t belongs (pestcde, corrosve cd, etc.). As our experments revel, n ctve query selecton lgorthm desgned to rpdly dentfy ndvdul objects s not necessrly effcent for group dentfcton. Second, we consder the problem where the set Q of queres s prttoned nto groups (resprtory symptoms, crdo symptoms, etc.). Insted of suggestng specfc symptoms to the user, we desgn n lgorthm tht suggests group of queres, nd llows the user the freedom to nput nformton on ny query n tht group. Although such system wll theoretclly be less effcent, t s motvted by the fct tht n prctcl pplcton, some symptoms wll be eser for gven user to understnd nd dentfy. Insted of suggestng sngle symptom, whch mght seem out of the blue to the user, suggestng query group wll be less bewlderng, nd hence led to more effcent nd ccurte outcome. Our experments demonstrte tht the proposed lgorthm bsed on query groups dentfes objects n nerly s few queres s fully ctve method. Thrd, we pply our lgorthm for group dentfcton to the problem of object dentfcton under persstent query nose. Persstent query nose occurs when the response to query s n error, but cnnot be re-smpled, s s often ssumed n the lterture. Such s the cse when the presence or bsence of symptom s ncorrectly determned, whch s more lkely n stressful emergency response scenro. Experments show our method offers sgnfcnt gns over lgorthms not desgned for persstent query nose. Our lgorthms re derved n common frmework, nd re bsed on renterpretton of stndrd object dentfcton lgorthm (the splttng lgorthm, or generlzed bnry serch) s generlzed form of Shnnon-Fno codng. We frst estblsh n exct formul for the expected number of queres requred to dentfy n object usng n rbtrry decson tree, nd show tht the splttng lgorthm effectvely performs greedy, top-down optmzton of ths objectve. We then extend ths formul to the cse of group dentfcton nd query groups, nd develop nlogous greedy lgorthms. In the process, we provde new nterpretton of mpurty-bsed decson tree nducton for mult-clss clssfcton. We lso develop logrthmc pproxmton bound for group dentfcton, usng the noton of submodulr

4 4 functons. We pply our lgorthms to both synthetc dt nd to the WISER dtbse (verson 4.2). WISER 2, whch stnds for Wreless Informton System for Emergency Responders, s decson support system developed by the Ntonl Lbrry of Medcne (NLM) for frst responders. Ths dtbse descrbes the bnry reltonshp between 298 toxc chemcls (corresponds to the number of dstngushble chemcls n ths dtbse) nd 79 cute symptoms. The symptoms re grouped nto 0 ctegores (e.g., neurologcl, crdo) s determned by NLM, nd the chemcls re grouped nto 6 ctegores (e.g., pestcdes, corrosve cds) s determned by toxcologst nd Hzmt expert. A. Pror nd relted work The problem of selectng n optml sequence of queres from Q to unquely dentfy n unknown object θ s equvlent to determnng n optml bnry decson tree, where ech nternl node n the tree corresponds to query, ech lef node corresponds to unque object from the set Θ nd the optmlty s wth respect to mnmzng the expected depth of the lef node correspondng to θ. In the specl cse when the query set Q s complete ( query set Q s sd to be complete f for ny S Θ there exsts query q Q such tht ether q = S or Θ\q = S), the problem of constructng n optml bnry decson tree s equvlent to constructon of optml vrble-length bnry prefx codes wth mnmum expected length. Ths problem hs been wdely studed n nformton theory wth both Shnnon [5] nd Fno [6] ndependently proposng top-down greedy strtegy to construct suboptml bnry prefx codes, populrly known s Shnnon-Fno codes. Lter Huffmn [7] derved smple bottom-up lgorthm to construct optml bnry prefx codes. A well known lower bound on the expected length of bnry prefx codes s gven by the Shnnon entropy of the probblty dstrbuton Π [8]. When the query set Q s not complete, n object dentfcton problem cn be consdered s constrned prefx codng wth the sme lower bound on the expected depth of tree. Ths problem hs lso been studed extensvely n the lterture wth Grey [3], [4] proposng dynmc progrmmng bsed lgorthm to fnd n optml soluton. Ths lgorthm runs n exponentl tme n the worst cse. Lter, Hyfl nd Rvest [9] showed tht determnng n optml bnry decson tree for ths problem s NP-complete. Therefter, vrous greedy lgorthms [5], [20], [2] hve been proposed to obtn suboptml bnry decson tree. The most wdely studed lgorthm, known s the splttng lgorthm [5] or generlzed bnry serch (GBS) [2], [2], selects query tht most evenly dvdes the probblty 2

5 5 mss of the remnng objects [2], [5], [2], [22]. Vrous bounds on the performnce of ths greedy lgorthm hve been estblshed n [2], [5], [2]. In ddton, severl vrnts of ths problem such s multwy or k-ry splts (nsted of bnry splts) [23], [24], [25] nd unequl query costs [3], [4], [25], [26] hve lso been studed n the lterture. Goodmn nd Smyth [22] observe tht the splttng lgorthm cn be vewed s generlzed verson of Shnnon-Fno codng. In Secton II, we demonstrte the sme through n lterntve pproch tht cn be generlzed to the group-bsed settngs, ledng to effcent lgorthms n these settngs 3. Golovn et l. [28] smultneously studed the problem of group dentfcton, nd lso proposed ner-optml lgorthm, whch s dscussed n more detl n Secton III-B. Though most of the bove work hs been devoted to object dentfcton n the del settng ssumng no nose, t s unrelstc to ssume tht the responses to queres re wthout error n mny pplctons. The problem of dentfyng n unknown object n the presence of query nose hs been studed n [2], [29], [30] where the queres cn be re-smpled or repeted. However, n certn pplctons, re-smplng or repetng query does not chnge the query response confnng the lgorthm to non-repetble queres. The work by Rény n [3] s regrded to be the frst to consder ths more strngent nose model, lso referred to s persstent nose n the lterture [32], [33], [34]. However, hs work hs focused on the pssve settng where the queres re chosen t rndom. Lernng under persstent nose model hs lso been studed n [32], [33], [35] where the gol ws to dentfy or lern Dsjunctve Norml Form (DNF) formule from nosy dt. The query (lbel) complexty of pool-bsed ctve lernng n the Probbly Approxmtely Correct (PAC) model n the presence of persstent clssfcton nose hs been studed n [34] nd ctve lernng lgorthms n ths settng hve been proposed n [34], [36]. Here, we focus on the problem of object dentfcton under the persstent nose model where the gol s to unquely dentfy the true object. A smlr problem studed n the gme-theoretc lterture s known s the Rény-Ulm s problem, where the gol s to dentfy n unknown number x from known set of numbers {,, n} usng s few bnry questons (of the form Is x member of S {,, n}? ) s possble, wth t most k errors n the obtned responses [37], [38], [39], [40]. Ths problem s smlr to the problem of desgnng mnmum length k-error correctng codes n communcton theory, where the query set Q s complete [4]. Hence, t cn be consdered s specl cse of the problem studed n ths pper. Fnlly, ths work ws motvted by erler work tht ppled GBS to WISER [42]. 3 A prelmnry verson of ths work ppered n NIPS 200 [27].

6 6 B. Notton We denote n object dentfcton problem by pr (B, Π) where B s bnry mtrx wth b j equl to f θ q j, nd 0 otherwse. We ssume tht the rows of B re dstnct,.e., we mke the ssumpton of unque dentfblty of every object n Θ. Ths s resonble snce objects tht hve smlr query responses for ll queres n Q,.e., objects tht re not dstngushble, cn lwys be grouped nto sngle met-object. A decson tree T constructed on (B, Π) hs query from the set Q t ech of ts nternl nodes wth the lef nodes termntng n the objects from the set Θ. At ech nternl node n the tree, the object set under consderton s dvded nto two subsets, correspondng to the objects tht respond 0 nd to the query, respectvely. For decson tree wth L leves, the lef nodes re ndexed by the set L = {,, L} nd the nternl nodes re ndexed by the set I = {L +,, 2L }. At ny nternl node I, let l(), r() denote the left nd rght chld nodes, where the set Θ Θ corresponds to the set of objects tht rech node, nd the sets Θ l() Θ, Θ r() Θ corresponds to the set of objects tht respond 0 nd to the query t node, respectvely. We denote by := {:θ Θ } π, the probblty mss of the objects under consderton t ny node n the tree. Also, t ny node, the set Q Q corresponds to the set of queres tht hve been performed long the pth from the root node up to node. We denote the Shnnon entropy of vector Π = (π,, π M ) by H(Π) := π log 2 π nd the Shnnon entropy of proporton π [0, ] by H(π) := π log 2 π ( π) log 2 ( π), where we use the lmt, lm π 0 π log 2 π = 0 to defne the lmtng cses. Fnlly, gven tree T, we use the rndom vrble K(T ) to denote the number of queres requred to dentfy n unknown object θ or the group of n unknown object θ usng the gven tree. II. GENERALIZED SHANNON-FANO CODING Before proceedng to the group-bsed settng, we frst present n exct formul for the stndrd object dentfcton problem. Ths result llows us to nterpret the splttng lgorthm or GBS s generlzed Shnnon-Fno codng. Furthermore, our proposed lgorthms for group-bsed settngs re bsed on generlztons of ths result. Frst, we defne prmeter clled the reducton fctor on the bnry mtrx/tree combnton tht provdes useful quntfcton on the expected number of queres requred to dentfy n unknown object.

7 7 Defnton. A reducton fctor t ny nternl node n decson tree s defned s ρ = mx(l(), r() )/ nd the overll reducton fctor of tree s defned s ρ = mx I ρ. Note from the bove defnton tht 0.5 ρ ρ nd we descrbe decson tree wth ρ = 0.5 to be perfectly blnced tree. Gven n object dentfcton problem (B, Π), let T (B, Π) denote the set of decson trees tht cn unquely dentfy ll the objects n the set Θ. For ny decson tree T T (B, Π), let {ρ } I denote the set of reducton fctors nd let d denote the depth of object θ n the tree. Then, the expected number of queres requred to dentfy n unknown object usng the gven tree s equl to M M E[K(T )] = Pr(θ = θ )E[K(T ) θ = θ ] = π d. = Theorem. The expected number of queres requred to dentfy n unknown object usng tree T T (B, Π) wth reducton fctors {ρ } I s gven by where := πθ r I πθr. E[K(T )] = H(Π) + H(Π) [ H(ρ )] = I I H(ρ ) Proof: The frst equlty s specl cse of Theorem 2 below. The second equlty follows from the observton E[K(T )] = M = π d = I. Hence replcng wth E[K(T )] n the frst equlty leds to the result. In the second equlty, the term I H(ρ ) denotes the verge entropy of the reducton fctors, weghted by the proporton of tmes ech nternl node s quered n the tree. Ths theorem re-tertes n erler observton tht the expected number of queres requred to dentfy n unknown object usng tree constructed on (B, Π) (where the query set Q s not necessrly complete set) s bounded below by ts entropy H(Π). It lso follows from the bove result tht tree ttns ths mnmum vlue (.e., E[K(T )] = H(Π)) ff t s perfectly blnced,.e., the overll reducton fctor ρ of the tree s equl to 0.5. From the frst equlty, the problem of fndng decson tree wth mnmum E[K(T )] cn be formulted s the followng optmzton problem: mn H(Π) + I [ H(ρ )]. (2) T T (B,Π) Snce Π s fxed, the optmzton problem reduces to mnmzng I [ H(ρ )] over the set of trees T (B, Π). Note tht the reducton fctor ρ depends on the query chosen t node n tree T. As mentoned erler, fndng globl optml soluton for ths optmzton problem s NP-complete. = ()

8 8 Insted, we my tke top down pproch nd mnmze the objectve functon by mnmzng the term [ H(ρ )] t ech nternl node, strtng from the root node. Snce s ndependent of the query chosen t node, ths reduces to mnmzng ρ (.e., choosng splt s blnced s possble) t ech nternl node I. The lgorthm cn be summrzed s shown n Algorthm below. Generlzed Bnry Serch (GBS) Intlzton : Let the lef set consst of the root node whle some lef node hs Θ > do for ech query q Q \ Q do Fnd Θ l() nd Θ r() produced by mkng splt wth query q Compute the reducton fctor ρ produced by query q end Choose query wth the smllest reducton fctor Form chld nodes l(), r() end Algorthm : Greedy decson tree lgorthm for object dentfcton Note tht when the query set Q s complete, Algorthm s smlr to Shnnon-Fno codng [5], [6]. The only dfference s tht n Shnnon-Fno codng, for computtonl resons, the queres re restrcted to those tht re bsed on thresholdng the pror probbltes π. Corollry. The stndrd splttng lgorthm/gbs s greedy lgorthm to mnmze the expected number of queres requred to unquely dentfy n object. Corollry 2 below follows from Theorem. It sttes tht gven tree T wth overll reducton fctor ρ <, the verge complexty of dentfyng n unknown object usng ths tree s O(log 2 M). Recently, Nowk [2] showed there re geometrc condtons (ncoherence nd neghborlness) tht lso bound the worst-cse depth of the tree to be O(log 2 M), ssumng unform pror on objects. These condtons mply tht the reducton fctors re close to 2 except possbly ner the very bottom of the tree where they could be close to. Becuse ρ could be close to for deeper nodes, the upper bound on E[K(T )] bsed on the overll reducton fctor ρ gven below could be very loose n prctce. Corollry 2. The expected number of queres requred to dentfy n unknown object usng tree T wth

9 9 overll reducton fctor ρ constructed on (B, Π) s bounded bove by E[K(T )] H(Π) H(ρ) log 2 M H(ρ) Proof: Usng the second equlty n Theorem, we get H(Π) E[K(T )] = I H(ρ ) H(Π) H(ρ) log 2 M H(ρ) where the frst nequlty follows from the defnton of ρ, ρ ρ 0.5, I nd the lst nequlty follows from the concvty of the entropy functon. In the sectons tht follow, we show how Theorem nd Algorthm my be generlzed, ledng to prncpled strteges for group dentfcton, object dentfcton wth group queres nd object dentfcton wth persstent nose. III. GROUP IDENTIFICATION We now move to the problem of group dentfcton, where the gol s not to determne the unknown object θ Θ, rther the group to whch the object belongs. Here, n ddton to the bnry mtrx B nd pror probblty dstrbuton Π on the objects, the group lbels for the objects re lso provded, where the groups re ssumed to be dsjont. Note tht f the groups re overlppng, t cn be reduced to the dsjont settng by fndng the smllest prtton of the objects such tht the group lbels re constnt on ech cell of the prtton. Then, group dentfcton lgorthm would dentfy precsely those groups to whch the object belongs. For exmple, n toxc chemcl dentfcton, frst responder my only need to know whether chemcl s pestcde, corrosve cd, or both. Hence, t could be resonble to reduce group dentfcton problem wth overlppng groups to tht of dsjont groups rsng out of ts prtton. Thus, we devote our ttenton to the problem of group dentfcton wth dsjont groups. We denote group dentfcton problem by (B, Π, y), where y = (y,, y M ) denotes the group lbels of the objects, y {,, m}. Let {Θ } m = be prtton of the object set Θ, where Θ denotes the set of objects n Θ tht belong to group. It s mportnt to note here tht the group dentfcton problem cnnot be smply reduced to n object dentfcton problem wth groups {Θ,, Θ m } s met-objects, snce the objects wthn group need not respond the sme to ech query. For exmple, consder the toy exmple shown n Fg. where the objects θ, θ 2 nd θ 3 belongng to group cnnot be consdered s one sngle met-object s these objects respond dfferently to queres q nd q 3. In ths context, we lso note tht GBS cn fl to fnd good soluton for group dentfcton problem s t does not tke the group lbels nto consderton whle choosng queres. Once gn, consder the

10 0 q q 2 q 3 Group lbel, y θ 0 θ 2 0 θ θ Fg.. Toy Exmple q 0 y = q 2 0 y = 2 y = Fg. 2. Decson tree constructed usng GBS for group dentfcton on toy exmple toy exmple shown n Fg. where just one query (query q 2 ) s suffcent to dentfy the group of n unknown object, wheres GBS requres 2 queres to dentfy the group when the unknown object s ether θ 2 or θ 4, s shown n Fg. 2. Hence, we develop new strtegy whch ccounts for the group lbels when choosng the best query t ech stge. Note tht when constructng tree for group dentfcton, greedy, top-down lgorthm termntes splttng when ll the objects t the node belong to the sme group. Hence, tree constructed n ths fshon cn hve multple objects endng n the sme lef node nd multple leves endng n the sme group. For tree wth L leves, we denote by L L = {,, L} the set of leves tht termnte n group. Smlr to Θ Θ, we denote by Θ Θ the set of objects tht belong to group t ny nternl node I n the tree. Also, n ddton to the reducton fctors defned n Secton II, we defne new set of reducton fctors clled the group reducton fctors t ech nternl node. Defnton 2. The group reducton fctor of group t ny nternl node n decson tree s defned s ρ = mx( l(), r() )/. Gven group dentfcton problem (B, Π, y), let T (B, Π, y) denote the set of decson trees tht cn unquely dentfy the groups of ll objects n the set Θ. For ny decson tree T T (B, Π, y), let ρ denote the reducton fctor nd let {ρ } m = denote the set of group reducton fctors t ech of ts nternl nodes. Also, let d j denote the depth of lef node j L n the tree. Then the expected number of queres requred to dentfy the group of n unknown object usng the gven tree s equl to E[K(T )] = Pr(θ Θ )E[K(T ) θ Θ ] =

11 = = j L j d j Theorem 2. The expected number of queres requred to dentfy the group of n object usng tree T T (B, Π, y) wth reducton fctors {ρ } I nd group reducton fctors {ρ } m =, I, s gven by E[K(T )] = H(Π y ) + I [ H(ρ ) + = H(ρ ) where Π y denotes the probblty dstrbuton of the object groups nduced by the lbels y,.e. Π y = (,, m). Proof: Specl cse of Theorem 7 below. See lso [27]. The bove theorem sttes tht gven group dentfcton problem (B, Π, y), the expected number of queres requred to dentfy the group of n unknown object s lower bounded by the entropy of the probblty dstrbuton of the groups. It lso follows from the bove result tht ths lower bound s cheved ff there exsts perfectly blnced tree (.e. ρ = 0.5) wth the group reducton fctors equl to t every nternl node n the tree. Also, note tht Theorem s specl cse of ths theorem where ech group hs sze ledng to ρ = for ll groups t every nternl node. Usng Theorem 2, the problem of fndng decson tree wth mnmum E[K(T )] cn be formulted s the followng optmzton problem: mn I [ H(ρ ) + ] m = π T T (B,Π,y) Θ H(ρ ). (4) Note tht here both the reducton fctor ρ nd the group reducton fctors {ρ } m = ] (3) depend on the query chosen t node. Also, the bove optmzton problem beng generlzed verson of the optmzton problem n (2) s NP-complete. Hence, we propose suboptml pproch to solve the bove optmzton problem where we optmze the objectve functon loclly nsted of globlly. We tke top-down pproch nd mnmze the objectve functon by mnmzng the term := [ H(ρ ) + ] m = H(ρ ) t ech nternl node, strtng from the root node. The lgorthm cn be summrzed s shown n Algorthm 2 below. Ths lgorthm s referred to s GISA (Group Identfcton Splttng Algorthm) n the rest of ths pper. Note tht the objectve functon n ths lgorthm conssts of two terms. The frst term [ H(ρ )] fvors queres tht evenly dstrbute the probblty mss of the objects t node to ts chld nodes (regrdless of the group) whle the second term of objects to one of ts chld nodes. H(ρ ) fvors queres tht trnsfer n entre group

12 2 Group Identfcton Splttng Algorthm (GISA) Intlzton : Let the lef set consst of the root node whle some lef node hs more thn one group of objects do for ech query q j Q \ Q do Compute {ρ } m = nd ρ produced by mkng splt wth query q j Compute the cost (j) of mkng splt wth query q j end Choose query wth the lest cost t node Form chld nodes l(), r() end Algorthm 2: Greedy decson tree lgorthm for group dentfcton A. Connecton to Impurty-bsed Decson Tree Inducton As bref dgresson, n ths secton we show connecton between the bove lgorthm nd mpurtybsed decson tree nducton. In prtculr, we show tht the bove lgorthm s equvlent to the decson tree splttng lgorthm used n the C4.5 softwre pckge [43]. Before estblshng ths result, we brefly revew the mult-clss clssfcton settng where mpurty-bsed decson tree nducton s populrly used. In the mult-clss clssfcton settng, the nput s trnng dt x,, x M smpled from some nput spce (wth n underlyng probblty dstrbuton) long wth ther clss lbels, y,, y M nd the tsk s to construct clssfer wth the lest probblty of msclssfcton. Decson tree clssfers re grown by mxmzng n mpurty-bsed objectve functon t every nternl node to select the best clssfer from set of bse clssfers. These bse clssfers cn vry from smple xs-orthogonl splts to more complex non-lner clssfers. The mpurty-bsed objectve functon s [ πθl() I(Θ ) I(Θ l() ) + π ] Θ r() I(Θ π r() ), (5) Θ whch represents the decrese n mpurty resultng from splt. Here I(Θ ) corresponds to the mesure of mpurty n the nput subspce t node nd corresponds to the probblty mesure of the nput subspce t node. Among the vrous mpurty functons suggested n lterture [44], [45], the entropy mesure used n the C4.5 softwre pckge [43] s populr. In the mult-clss clssfcton settng wth m dfferent clss

13 3 lbels, ths mesure s gven by where, I(Θ ) = m = log πθ (6) re emprcl probbltes bsed on the trnng dt. Smlr to group dentfcton problem, the nput here s bnry mtrx B wth b j denotng the bnry lbel produced by bse clssfer j on trnng smple, nd probblty dstrbuton Π on the trnng dt long wth ther clss lbels y. But unlke group dentfcton problem where the nodes n tree re not termnted untl ll the objects belong to the sme group, the lef nodes here re llowed to contn some mpurty n order to vod overfttng. The followng result extends Theorem 2 to the cse of mpure lef nodes. Theorem 3. The expected depth of lef node n decson tree clssfer T T (B, Π, y) wth reducton fctors {ρ } I nd clss reducton fctors {ρ } m =, I, s gven by E[K(T )] = H(Π y ) + ] [ H(ρ ) + H(ρ π ) I(Θ ) (7) I = Θ L where Π y denotes the probblty dstrbuton of the clsses nduced by the clss lbels y,.e., Π y = (,, m) nd I(Θ ) denotes the mpurty n lef node gven by (6). Proof: The proof s gven n Appendx A. The only dfference compred to Theorem 2 s the lst term, whch corresponds to the verge mpurty n the lef nodes. Theorem 4. At every nternl node n tree, mnmzng the objectve functon := H(ρ ) + [ m = H(ρ πθl() ] ) s equvlent to mxmzng I(Θ ) I(Θ l() ) + πθ r() I(Θ r() ) wth entropy mesure s the mpurty functon. Proof: The proof s gven n Appendx B. Therefore, greedy optmzton of (7) t nternl nodes corresponds to greedy optmzton of mpurty. Also, note tht optmzng (7) t lef ssgns the mjorty vote clss lbel. Therefore, we conclude tht mpurty-bsed decson tree nducton wth entropy s the mpurty mesure mounts to greedy optmzton of the expected depth of lef node n the tree. Also, Theorem 3 llows us to nterpret mpurty bsed splttng lgorthms for mult-clss decson trees n terms of reducton fctors, whch lso ppers to be new nsght.

14 4 B. Modfed GISA wth Ner-optml Performnce As mentoned n Secton I-A, the splttng lgorthm or GBS hs been shown to be ner-optml wth logrthmc pproxmton rto [2], [2], [3],.e., ( E[K( T )] O ln ) E[K(T )], π mn where π mn := mn π s the mnmum pror probblty of ny object, T s greedy tree constructed usng GBS nd T s n optml tree for the gven problem. Recently, Golovn et l. [3] ntroduced the noton of dptve submodulrty nd strong dptve monotoncty (refer Appendx C), nd showed tht greedy optmzton lgorthm wth these propertes cn be ner-optml nd cheve logrthmc pproxmton rto, wth GBS beng specfc nstnce of ths clss. Unfortuntely, the objectve functon of GISA,.e., π H(ρ ) H(ρ π ) (8) = does not stsfy these propertes. We present modfed verson of GISA tht cn be shown to be dptve submodulr nd strong dptve monotone, nd hence cn cheve logrthmc pproxmton to the optml soluton. The modfed lgorthm s to construct top-down, greedy decson tree where t ech nternl node, query tht mxmzes π l() π r() = π π π l() π r() (9) s chosen. Essentlly, the bnry entropy terms H(ρ ) nd H(ρ ) n (8) re pproxmted by the weghted Gn ndces, π 2 (ρ ( ρ )) nd ( π ) 2 (ρ ( ρ )), respectvely. Note tht n the specl cse where ech group s of sze, the query selecton crteron n (9) reduces to π l() π r(), thereby reducng modfed GISA to the stndrd splttng lgorthm. Gven group dentfcton problem (B, Π, y), recll tht T (B, Π, y) denotes the set of ll possble trees tht cn unquely dentfy the group of ny object from the set Θ. Then, let T denote tree wth the lest expected depth,.e., T rg mn E[K(T )], T T (B,Π,y) nd let T denote tree constructed usng modfed GISA. The followng theorem sttes tht the expected depth of T s logrthmclly close to tht of n optml tree.

15 5 Theorem 5. Let (B, Π, y) denote group dentfcton problem. For greedy decson tree T constructed usng modfed GISA, t holds tht ( ) ) E[K( T )] (2 ln + E[K(T )], (0) 3πmn where π mn := mn{π Π : π > 0} s the mnmum pror probblty of ny object. Proof: The proof s gven n Appendx C. In ddton, f the query costs re unequl, the query selecton crteron n modfed GISA cn be chnged to rg mx q / Q (q)/c(q), where (q) s s defned n (9), nd c(q) s the cost of obtnng the response to query q. Ths smple heurstc hs been shown to retn the ner-optml property [3],.e., ( ) ) c( T ) (2 ln + c(t ), 3πmn where T s greedy tree constructed usng the bove heurstc, nd T s tree wth mnmum expected cost. The cost of tree T s defned s c(t ) := E θ [c(t, θ)], where c(t, θ ) s the totl cost of the queres mde long the pth from the root node to the lef node endng n object θ. Golovn et l. [28] smultneously studed the problem of group dentfcton, nd, lke us, used t n the context of object dentfcton wth persstent nose. They proposed n extenson of the lgorthm n [46] for group dentfcton, nd showed logrthmc pproxmton smlr to us. However, ther result holds only when the prors π re rtonl. In ddton, the bound cheved by modfed GISA s mrgnlly tghter thn thers. IV. OBJECT IDENTIFICATION UNDER GROUP QUERIES In ths secton, we return to the problem of object dentfcton. The nput s bnry mtrx B denotng the reltonshp between M objects nd N queres, where the queres re grouped pror nto n dsjont ctegores, long wth the pror probblty dstrbuton Π on the objects. However, unlke the decson trees constructed n the prevous two sectons where the end user (e.g., frst responder) hs to go through fxed set of questons s dctted by the decson tree, here, the user s offered more flexblty n choosng the questons t ech stge. More specfclly, the decson tree suggests query group from the n groups nsted of sngle query t ech stge, nd the user cn choose query to nswer from the suggested query group. A decson tree constructed wth group of queres t ech stge hs multple brnches t ech nternl node, correspondng to the sze of the query group. Hence, tree constructed n ths fshon hs multple

16 6 Q Q q q 2 q 3 q 4 θ 0 0 θ 2 0 θ 3 0 Fg. 3. Toy Exmple 2 Fg. 4. queres 0 q θ 3 Q q 0.5 Q 2 q q 4 θ Q 2 0 q 3 θ θ 2 θ 2 θ θ 3 θ 2 Decson tree constructed on toy exmple 2 for object dentfcton under group leves endng n the sme object. Whle trversng ths decson tree, the user chooses the pth t ech nternl node by selectng the query to nswer from the gven lst of queres. Fg. 4 demonstrtes decson tree constructed n ths fshon for the toy exmple shown n Fg. 3. The crcled nodes correspond to the nternl nodes, where ech nternl node s ssocted wth query group. The numbers ssocted wth dshed edge correspond to the probblty tht the user wll choose tht pth over the others. The probblty of rechng node I n the tree gven θ Θ s gven by the product of the probbltes on the dshed edges long the pth from the root node to tht node, for exmple, the probblty of rechng lef node θ gven θ = θ n Fg. 4 s The problem now s to select the query ctegores tht wll dentfy the object most effcently, on verge. In ddton to the termnology defned n Sectons I-B nd II, we lso defne z = (z,, z N ) to be the group lbels of the queres, where z j {,, n}, j =,, N. Let {Q } n = be prtton of the query set Q, where Q denotes the set of queres n Q tht belong to group. Smlrly, t ny node n tree, let Q nd Q denote the set of queres n Q nd Q \ Q tht belong to group respectvely. Let p (q) be the pror probblty of the user selectng query q Q t ny node wth query group n the tree, where q Q p (q) =. In ddton, t ny node n the tree, the functon p (q) = 0, q Q, snce the user would not choose query whch hs lredy been nswered, n whch cse p (q) s renormlzed. In our experments we tke p (q) to be unform on Q. Fnlly, let z {,, n} denote the query group selected t n nternl node n the tree nd let p denote the probblty of rechng tht node gven θ Θ.

17 7 We denote n object dentfcton problem wth query groups by (B, Π, z, p). Gven (B, Π, z, p), let T (B, Π, z, p) denote the set of decson trees tht cn unquely dentfy ll the objects n the set Θ wth query groups t ech nternl node. For decson tree T T (B, Π, z, p), let {ρ (q)} q Q z denote the reducton fctors of ll the queres n the query group t ech nternl node I n the tree, where the reducton fctors re treted s functons wth nput beng query. Also, for tree wth L leves, let L L = {,, L} denote the set of leves termntng n object θ nd let d j denote the depth of lef node j L. Then, the expected number of queres requred to dentfy the unknown object usng the gven tree s equl to E[K(T )] = = M Pr(θ = θ )E[K(T ) θ = θ ] = M = π j L p j d j Theorem 6. The expected number of queres requred to dentfy n object usng tree T T (B, Π, z, p) s gven by E[K(T )] = H(Π) + p p z (q)h(ρ (q)) () I q Q z Proof: Specl cse of Theorem 7 below. Note from the bove theorem, tht gven n object dentfcton problem wth group queres (B, Π, z, p), the expected number of queres requred to dentfy n object s lower bounded by ts entropy H(Π). Also, ths lower bound cn be cheved ff the reducton fctors of ll the queres n query group t ech nternl node of the tree s equl to 0.5. In fct, Theorem s specl cse of the bove theorem where ech query group hs just one query. Gven (B, Π, z, p), the problem of fndng decson tree wth mnmum E[K(T )] cn be formulted s the followng optmzton problem: [ mn I p ] q Q p T T (B,Π,z,p) z z (q)h(ρ (q)). (2) Note tht here the reducton fctors ρ (q), q Q z nd the pror probblty functon p z (q) depends on the query group z {,, n} chosen t node n the tree. The bove optmzton problem beng generlzed verson of the optmzton problem n (2) s NP-complete. A greedy top-down locl optmzton of the bove objectve functon yelds suboptml soluton where we choose query group [ tht mnmzes the term (j) := ] q Q p j(q)h(ρ j (q)) t ech nternl node, strtng from the

18 8 root node. The lgorthm s summrzed n Algorthm 3 below s referred to s GQSA (Group Queres Splttng Algorthm) n the rest of ths pper. Group Queres Splttng Algorthm (GQSA) Intlzton : Let the lef set consst of the root node whle some lef node hs Θ > do for ech query group wth Q j do Compute the pror probbltes of selectng queres wthn group p j (q), q Q j t node Compute the reducton fctors for ll the queres n the query group {ρ (q)} q Q j Compute the cost (j) of usng query group j t node end Choose query group j wth the lest cost (j) t node Form the left nd the rght chld nodes for ll queres wth p j (q) > 0 n the query group end Algorthm 3: Greedy decson tree lgorthm for object dentfcton wth group queres Comment: In ths secton nd the one followng, we ssume tht the query groups re dsjont only for the ske of smplcty. However, we do not need ths ssumpton for the results n Theorem 6, nd Theorem 7 n the next secton, to hold. Smlrly, we ssume tht the pror probblty of choosng query from query group depends only on the group membershp. However, one could use more complex pror dstrbuton tht not only depends on the group membershp, but lso on the prevous queres nd ther responses. The results n Theorems 6 nd 7 do not chnge by these generlztons, s long s the pror dstrbuton s normlzed nd sums to t ech nternl node n the tree. Ths cn be redly observed from the proof of Theorem 7 n Appendx D. V. GROUP IDENTIFICATION UNDER GROUP QUERIES For the ske of completon, we consder here the problem of dentfyng the group of n unknown object θ Θ under group queres. The nput s bnry mtrx B denotng the reltonshp between M objects nd N queres, where the objects re grouped nto m groups nd the queres re grouped nto n groups. The tsk s to dentfy the group of n unknown object through s few queres from Q s possble where, t ech stge, the user s offered query group from whch query s chosen.

19 9 As noted n Secton III, decson tree constructed for group dentfcton cn hve multple objects termntng n the sme lef node. Also, decson tree constructed for group dentfcton wth query group t ech nternl node hs multple leves termntng n the sme group. Hence decson tree constructed n ths secton cn hve multple objects termntng n the sme lef node nd multple leves termntng n the sme group. Also, we use most of the termnology defned n Sectons III nd IV here. We denote group dentfcton problem wth query groups by (B, Π, y, z, p) where y = (y,, y M ) denotes the group lbels on the objects, z = (z,, z N ) denotes the group lbels on the queres nd p = (p (q),, p n (q)) denotes the pror probblty functons of selectng queres wthn query groups. Gven group dentfcton problem under group queres (B, Π, y, z, p), let T (B, Π, y, z, p) denote the set of decson trees tht cn unquely dentfy the groups of ll objects n the set Θ wth query groups t ech nternl node. For ny decson tree T T (B, Π, y, z, p), let {ρ (q)} q Q z the reducton fctor set nd let {{ρ (q)} m = } q Q z denote denote the group reducton fctor sets t ech nternl node I n the tree, where z {,, n} denotes the query group selected t tht node. Also, for tree wth L leves, let L L = {,, L} denote the set of leves termntng n object group nd let d j, p j denote the depth of lef node j L nd the probblty of rechng tht node gven θ Θ j, respectvely. Then, the expected number of queres requred to dentfy the group of n unknown object usng the gven tree s equl to E[K(T )] = Pr(θ Θ )E[K(T ) θ Θ ] = = = j L j p j d j Theorem 7. The expected number of queres requred to dentfy the group of n unknown object usng tree T T (B, Π, y, z, p) s gven by E[K(T )] = H(Π y ) + I p q Q z p z (q) [ H(ρ (q)) = H(ρ (q)) where Π y denotes the probblty dstrbuton of the object groups nduced by the lbels y,.e. Π y = (,, m) Proof: The proof s gven n Appendx D. Note tht Theorems, 2 nd 6 re specl cses of the bove theorem. Ths theorem sttes tht, gven group dentfcton problem under group queres (B, Π, y, z, p), the expected number of queres requred ] (3)

20 20 to dentfy the group of n object s lower bounded by the entropy of the probblty dstrbuton of the object groups H(Π y ). It lso follows from the bove theorem tht ths lower bound cn be cheved ff the reducton fctors nd the group reducton fctors of ll the queres n query group t ech nternl node re equl to 0.5 nd respectvely. The problem of fndng decson tree wth mnmum E[K(T )] cn be formulted s the followng optmzton problem: { mn I p [ q Q p T T (B,Π,y,z,p) z z (q) H(ρ (q)) ]} m = H(ρ (q)). (4) Note tht here the reducton fctors {ρ (q)} q Q z, the group reducton fctors {ρ (q)} q Q z for ll =,, m, nd the pror probblty functon p z (q) depends on the query group z {,, n} chosen t node n the tree. Once gn, the bove optmzton problem beng generlzed verson of the optmzton problem n (2) s NP-complete. A greedy top-down optmzton of the bove objectve functon yelds suboptml soluton where we choose query group tht mnmzes the term (j) := [ q Q p j(q) H(ρ j (q)) ] m = H(ρ (q)) t ech nternl node, strtng from the root node. The lgorthm s summrzed n Algorthm 4 below s referred to s GIGQSA (Group Identfcton under Group Queres Splttng Algorthm). Group Identfcton under Group Queres Splttng Algorthm (GIGQSA) Intlzton : Let the lef set consst of the root node whle some lef node hs more thn one group of objects do for ech query group wth Q j do Compute the pror probbltes of selectng queres wthn group, p j (q), q Q j t node Compute the reducton fctors for ll the queres n the query group {ρ (q)} q Q j Compute the group reducton fctors for ll the queres n the query group {ρ (q)} q Q j, =,, m Compute the cost (j) of usng query group j t node end Choose query group j wth the lest cost (j) t node Form the left nd the rght chld nodes for ll queres wth p j (q) > 0 n the query group end Algorthm 4: Greedy decson tree lgorthm for group dentfcton under group queres

21 2 VI. OBJECT IDENTIFICATION UNDER PERSISTENT NOISE We now consder the problem of rpdly dentfyng n unknown object θ Θ n the presence of persstent query nose, nd relte ths problem to group dentfcton. Query nose refers to errors n the query responses,.e., the observed query response s dfferent from the true response of the unknown object. For exmple, vctm of toxc chemcl exposure my not report symptom becuse of delyed onset of tht symptom. Unlke the nose model often ssumed n the lterture, where repeted queryng results n ndependent relztons of the nose, persstent query nose s more strngent nose model where repeted queres results n the sme response. Before we ddress ths problem, we need to ntroduce some ddtonl notton. Gven n object dentfcton problem (B, Π), let δ denote the mnmum Hmmng dstnce between ny two rows of the mtrx B. Also, we refer to the bt strng consstng of observed query responses s n nput strng. The nput strng cn dffer from the true bt strng (correspondng to the row vector of the true object n mtrx B) due to persstent query nose. However, note tht the number of query responses n error cnnot exceed ɛ := δ 2 for the unknown object to be unquely dentfed n the presence of nose. Gven ths restrcted nose settng, the gol of object dentfcton under persstent nose s to unquely dentfy the unknown object θ usng s few queres s possble. Ths problem cn be posed s group dentfcton problem s follows: Gven n object dentfcton problem (B, Π) wth M objects nd N queres tht s susceptble to ɛ errors, crete ( B, Π) wth M groups of objects nd N queres, where ech object group n ths new mtrx s formed by consderng ll possble bt strngs tht dffer from the orgnl bt strng n t most ɛ postons,.e., the sze of ech object group n B s ɛ e=0 ( N e ). Fg. 5(b) demonstrtes constructon of B for the toy exmple shown n Fg. 5() consstng of 2 objects nd 3 queres wth n ɛ =. Ech bt strng n the object set Θ of B corresponds to one of the possble nput strngs when the true object s θ nd t most ɛ errors occur. Also note tht, by defnton of ɛ, no two bt strngs n the mtrx B cn be the sme. Thus, the problem of rpdly dentfyng n unknown object θ from (B, Π) n the presence of t most ɛ persstent errors, reduces to the problem of dentfyng the group of the unknown object from ( B, Π). The probblty dstrbuton Π of the bt strngs n B depends on the pror Π nd the error model. In the followng secton, we descrbe one specfc error model tht rses commonly n pplctons such s ctve lernng, mge processng nd computer vson, nd demonstrte the computton of Π under tht error model. Gven tht ths problem cn be reduced to group dentfcton problem, the unknown object cn

22 22 q q 2 q 3 Π prone to error θ θ () q q 2 q 3 Π (p = 0.5) Π2 (p = 0.25) Θ Θ (b) Fg. 5. For the toy exmple shown n () consstng of 2 objects nd 3 queres wth n ɛ =, (b) demonstrtes the constructon of mtrx B. The probblty dstrbuton of the objects n B re generted usng the nose model descrbed n Secton VI-A, where only queres q 2 nd q 3 re ssumed to be prone to error. be rpdly dentfed n the presence of persstent query nose usng ny group dentfcton lgorthm ncludng GISA nd modfed GISA. In ddton, the ner-optml property of modfed GISA gurntees tht the expected number of queres requred to dentfy n unknown object under persstent nose s logrthmclly close to tht of n optml lgorthm, s stted n the result below. Corollry 3. Let (B, Π) denote n object dentfcton problem tht s susceptble to ɛ persstent errors. Let K denote the expected number of queres requred to dentfy n unknown object under persstent nose usng modfed GISA, nd let K denote the expected number of queres requred by n optml lgorthm. Then t holds tht ( ( ) ) K 2 ln + K, 3 πmn where π mn = mn{ π Π : π > 0}. Proof: The result follows from Theorem 5. A. Constnt nose rte We now consder nose model tht hs been used n the context of pool-bsed ctve lernng wth fulty orcle [30], [34], expermentl desgn [3], computer vson, nd mge processng [47], where the responses to some queres re ssumed to be rndomly flpped.

23 23 We wll descrbe generl verson of ths nose model. Gven N queres, consder the cse where frcton ν of them re prone to error. The query response to ech of these νn queres cn be n error wth probblty 0 p 0.5, where the errors occur ndependently. Then, the probblty of e errors occurrng s gven by Pr(e errors) = ( Nν e ɛ e =0 ) p e ( p) Nν e ( Nν e ) p e ( p) Nν e, 0 e ɛ where ɛ := mn(ɛ, Nν) denotes the mxmum number of persstent errors tht could occur. Note tht ths probblty model corresponds to truncted bnoml dstrbuton. Gven n object dentfcton problem (B, Π) tht s susceptble to ɛ errors, let B denote the extended bnry mtrx constructed s descrbed n Secton VI. The probblty dstrbuton Π of the objects n B cn be computed s follows. For n object belongng to group n B, f ts response to query tht s not prone to error dffers from the true response of object θ n B, then the probblty π of tht object n B s 0. On the other hnd, f ts response dffers n e ɛ queres tht re prone to error, then ts probblty s gven by ɛ e =0 p e ( p) Nν e ( Nν ) π. e p e ( p) Nν e Fg. 5(b) shows the probblty dstrbuton of the objects n B usng the probblty model descrbed bove wth p = 0.5 ( Π ) nd p = 0.25 ( Π2 ) for the toy exmple shown n Fg. 5() where only queres q 2 nd q 3 re prone to error. However, one possble concern wth ths pproch for object dentfcton under persstent nose could be memory relted ssue of explctly mntnng the mtrx B due to the combntorl exploson n ts sze. Interestngly, for the nose model descrbed here, the relevnt qunttes for query selecton n GBS, GISA nd modfed GISA (.e., the reducton fctors) cn be effcently computed wthout explctly constructng the mtrx B, descrbed n detl n Appendx E. VII. EXPERIMENTS We perform three sets of experments, demonstrtng our lgorthms for group dentfcton, object dentfcton usng query groups, nd object dentfcton wth persstent nose. In ech cse, we compre the performnces of the proposed lgorthms to stndrd lgorthms such s the splttng lgorthm, usng synthetc dt s well s rel dtset, the WISER dtbse. The WISER dtbse s toxc chemcl dtbse descrbng the bnry reltonshp between 298 toxc chemcls nd 79 cute symptoms. The symptoms re grouped nto 0 ctegores (e.g., neurologcl, crdo) s determned by NLM, nd the

24 24 chemcls re grouped nto 6 ctegores (e.g., pestcdes, corrosve cds) s determned by toxcologst nd Hzmt expert. A. Group dentfcton Here, we consder group dentfcton problem (B, Π) where the objects re grouped nto m groups gven by y = (y,, y M ), y {,, m}, wth the tsk of dentfyng the group of n unknown object from the object set Θ through s few queres from Q s possble. Frst, we consder rndom dtsets generted usng rndom dt model nd compre the performnces of GBS, GISA nd modfed GISA for group dentfcton n these rndom dtsets. Then, we compre the performnce of these lgorthms on the WISER dtbse. In both these experments, we ssume unform pror probblty dstrbuton on the objects. ) Rndom Dtsets: We consder rndom dtsets of the sme sze s the WISER dtbse, wth 298 objects nd 79 queres where the objects re grouped nto 6 clsses wth the sme group szes s tht n the WISER dtbse. We ssocte ech query n rndom dtset wth two prmeters, γ w [0.5, ] whch reflects the correlton of the object responses wthn group, nd γ b [0.5, ] whch cptures the correlton of the object responses between groups. When γ w s close to 0.5, ech object wthn group s eqully lkely to exhbt 0 or s ts response to the query, wheres, when γ w s close to, most of the objects wthn group re hghly lkely to exhbt the sme response to the query. Smlrly, when γ b s close to 0.5, ech group s eqully lkely to exhbt 0 or s ts response to the query, where group response corresponds to the mjorty vote of the object responses wthn group, whle, s γ b tends to, most of the groups re hghly lkely to exhbt the sme response. Gven (γ w, γ b ) pr for query n rndom dtset, the object responses for tht query re creted s follows ) Generte Bernoull rndom vrble, x 2) For ech group {,, m}, ssgn bnry lbel b, where b = x wth probblty γ b 3) For ech object n group, ssgn b s the object response wth probblty γ w Gven the correlton prmeters (γ w (q), γ b (q)) [0.5, ] 2, q Q, rndom dtset cn be creted by followng the bove procedure for ech query. Conversely, we descrbe n Secton VII-A2 on how to estmte these prmeters for gven dtset. Fg. 6 compres the men E[K(T )] for GBS, GISA nd modfed GISA n 00 rndomly generted dtsets (for ech vlue of d nd d 2 ), where the rndom dtsets re creted such tht the query prmeters re unformly dstrbuted n the rectngulr spce governed by d, d 2 s shown n Fg. 7. Ths

25 25 8 Expected # of queres GBS Modfed GISA GISA Entropy bound, H(Π y ) 3 0. d d Fg. 6. Expected number of queres requred to dentfy the group of n object usng GBS, GISA nd modfed GISA on rndom dtsets generted usng the proposed rndom dt model. Note tht GISA nd modfed GISA cheve lmost smlr performnce on these dtsets, wth GISA performng slghtly better thn modfed GISA. demonstrtes the mproved performnce of GISA nd modfed GISA over GBS n group dentfcton. Especlly, note tht E[K(T )] tends close to the entropy bound H(Π y ) usng both GISA nd modfed GISA s d 2 ncreses. Ths s due to the ncrement n the number of queres n the fourth qudrnt of the prmeter spce s d 2 ncreses. Specfclly, s the correlton prmeters γ w, γ b tends to nd 0.5 respectvely, choosng tht query elmntes pproxmtely hlf the groups wth ech group beng ether completely elmnted or completely ncluded,.e. the group reducton fctors tend to for these queres. Such queres re preferble n group dentfcton wth both GISA nd modfed GISA beng specfclly desgned to serch for those queres ledng to ther strkngly mproved performnce over GBS s d 2 ncreses. 2) WISER Dtbse: Tble I compres the expected number of queres requred to dentfy the group of n unknown object n the WISER dtbse usng GISA, modfed GISA, GBS nd rndom serch, where the group entropy n the WISER dtbse s gven by H(Π y ) = The tble reports the 95% symmetrc confdence ntervls bsed on rndom trls, where the rndomness n GISA, modfed GISA nd GBS s due to the presence of multple best splts t ech nternl node. However, the mprovement of both GISA nd modfed GISA over GBS on WISER s less thn ws observed for mny of the rndom dtsets dscussed bove. To understnd ths, we developed method