Hardware Acceleration of the Pair HMM Algorithm for DNA Variant Calling

Transcription

1 Hardware Acceleration o the Pair H Algorithm or NA Variant Calling Sitao Huang, Gowthami Jayashri anikandan, Anand Ramachandran, Kyle Rupnow, Wen-mei W. Hwu, eming Chen University o llinois at Urbana-Champaign, USA Advanced igital Sciences Center, Singapore

2 Genomic Variation and utations Humans have two sets o billion bases in their genomes No two humans have identical genome sequences About 0. % o genomes are not identical These dierences lead to people Having dierent susceptibility or resistance to diseases Responding dierently to the same medication There are also somatic variations that lead to cancer

3 The mportance o utations and Variant Calling The study o mutations is important (e.g. in cancer study) They create cancer They enable cancer to survive They enable cancer to spread They enable cancer to kill Variant calling is a set o analytics that tries to identiy mutations in a sequenced genome compared to a standard reerence Variant Calling is critical in cancer research and clinical applications GATK's HaplotypeCaller is one o the most popular variant calling tools available today.

4 Accelerating the Pair H in GATK Why Pair H Needs to Be Accelerated? Pair H computations constitute the bottleneck o HaplotypeCaller The ull HaplotypeCaller is time consuming Full HaplotypeCaller run on 80xWGS PCR-Free NA878 dataset: days on single CPU Traversal + Assembly Genotyping 598s 79s (.86%) (6.7%) Pair-H (orward algorithm) 45s (70.4%) Why Using Hardware (FPGA)? Proiling result o a typical HaplotypeCaller run on CPU Parallelism in pair H could be better utilized by the ine-grained processing elements in FPGA FPGA is good at processing streaming applications (alignment algorithms nature)

5 Pair H (candidate to be veriied) (data rom sequencing machine) nput: two sequences SS h and SS rr (SS h : haplotype SS rr : read) Goal: ind a similarity score o SS h and SS rr SS rr GTAA SS h AGGTC One possible alignment: SS rr SS h {aa tt } --GTAA AGGTC- Another possible alignment: SS rr -G-TAA SS h AGGT-C {aa tt } There are many action sequences mapping SS rr to SS hh. Similarity score is deined over a pair Hidden arkov odel 4

6 Pair H ynamic Programming Similarly: ( i, j) = a ( i, j) = a Coeicients rom model ( i, j) = prior ( i, j ) + a ( i, j) + a ( i, j ) ( i, j) ( a ( i, j ) + a ( i, j ) + a ( i, j ) ) 0 Read Sequence C G G A Output: Complexity: ssssssssss SS h, SS rr = NN h, NN rr + NN h, NN rr + NN h, NN rr O ( h r Nh Nr ) # haplotype sequences # read sequences 5

7 How to Accelerate? : Processing Elements Process rontier elements at the same time to maximize parallelism Number o s Needed = atrix Height What i matrix height is larger than number o s FPGA can host? 6

8 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Read Sequence C G G G A A N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 7

9 Ring* Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Haplotype Sequence A G G T A C 0 Read Sequence C G G G A A 4 Busy N dle * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

10 Ring* Connects the irst and the last with ivide matrix rows to groups Haplotype Sequence A G G T A C 0 Read Sequence C G G G A A 4 Processing Element () Ring: N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

11 Ring* Connects the irst and the last with ivide matrix rows to groups Haplotype Sequence A G G T A C 0 Read Sequence C G G G A A 4 Processing Element () Ring: N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

12 Ring* Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Haplotype Sequence A G G T A C 4 0 Read Sequence C G G G A A N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

13 Ring* Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Haplotype Sequence A G G T A C 4 0 Read Sequence C G G G A A N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

14 Ring* Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Haplotype Sequence A G G T A C 4 0 Read Sequence C G G G A A N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

15 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups 4 Read Sequence C G G G A A Processing Element () Ring: N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

16 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: 4 Read Sequence C G G G A A N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

17 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: 4 Read Sequence C G G G A A N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

18 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Read Sequence C G G G A A N 4 * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

19 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Read Sequence C G G G A A N 4 * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

20 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: Read Sequence C G G G A A N 4 * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

21 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: N 4 Read Sequence C G G G A A * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 8

22 Ring* Haplotype Sequence A G G T A C 0 Connects the irst and the last with ivide matrix rows to groups Processing Element () Ring: N * Zilong Wang, Sitao Huang, Lanjun Wang, Hao Li, Yu Wang, and Huazhong Yang. Accelerating subsequence similarity search based on dynamic time warping distance with FPGA. n Proceedings o the AC/SGA international symposium on Field Programmable Gate Arrays, pages 5 6. AC, 0. 4 Read Sequence C G G G A A 8

23 Challenges in esigning or Pair H v ( i- ) v ( i- ) structure is designed according to the data dependencies in the algorithm Each passes its intermediate computing result to the next v( i ) v ( i ) 9

24 Challenges in esigning or Pair H Cont. Floating point operations Long latency Need sophisticated FS Complicated arithmetic operations in P Elements in three P matrices depend on each other ( i, j) = a ( i, j) = a ( i, j) = prior ( i, j ) + a ( i, j) + a ( i, j ) ( i, j) i, j + i, j adm + amm i, j prior i, j ami i, j + aii i, j (i-, j) i, j (i, j) amd i, j add Arithmetic Operations Within a (Original) ( a ( i, j ) + a ( i, j ) + a ( i, j ) ) (i-, j-) (i, j-) + i, j i, j 0

25 Optimization : Shorten critical path in arithmetic operations (i-, j) (i-, j-) Using dierent input operands i, j + i, j adm a t i, j amm b t i, j a t i, j + b t i, j i, j prior i, j i, j + i, j adm + amm i, j i, j prior ami i, j + aii i, j i, j (i, j) amd (i, j-) i, j add + i, j i, j Optimized Calculation Arithmetic Operations Within a (Original)

26 Optimization : Pipelining and resource sharing Hide the loating-point arithmetic operations latency mprove throughput

27 Optimization : Tuning ring size and number o rings Same amount o HW resource can accommodate more shorter rings (calculating multiple matrices) Shorter rings have ewer idle s Only /N dle s!

28 Experiment Result : Comparison to Other mplementations Compared to CPU, vector processor, GPU, multicore, previous FPGA implementations Using 0s dataset Arria 0 has more logic and SP resources. t also has hard loating-point SP block * * Theoretical runtime lower bound (assuming no idle ) or 64 s: 4.7ms * Altera. Accelerating genomics research with OpenCL and FPGAs, 06. 4

29 Experiment Result : mpact o Ring Size Shorter rings beneit rom higher utilization and smaller initialization overhead 8s/ring x 8 rings 6s/ring x 4 rings s/ring x rings 64s/ring x ring 5

30 Summary Pair H orward algorithm is computationintensive. t is the bottleneck o HaplotypeCaller. Ring-based hardware structure exhibits lexibility in coniguration and high data reuse. ring structure based pair H implementation can achieve signiicant speedup compared to the sotware implementation, and it also outperorms the published best hardware implementation. 6

31 HaplotypeCaller BACKUP SLES

32 Emission and Transition Probabilities BACKUP SLES Q base : Base Error Rate Q i : Base nsertion Probability Q d : Base eletion Probability Q g : Gap Continuation Penalty

33 What s in? (i-, j) (i-, j-) BACKUP SLES (i, j) (i, j-) i, j + i, j adm amm a t i, j + b t i, j i, j prior ami i, j + aii i, j amd i, j add + i, j a t i, j b t i, j i, j i, j i, j

34 Why sequence alignment? BACKUP SLES Comparing genes or regions rom dierent species to ind important regions determine unction uncover evolutionary orces Assembling ragments to sequence NA Compare individuals to looking or mutations

35 Problem Statement BACKUP SLES nput: two sequences SS h and SS rr (SS h : haplotype SS rr : read) Goal: ind a similarity score o SS h and SS rr (candidate to be veriied) (data rom sequencing machine) SS rr SS h GTAA AGGTC Similarity score is deined over a pair Hidden arkov odel

36 Pair H Action Sequence BACKUP SLES Action(elete, nsert, atch/ismatch) sequence {aa tt } s.t. SS rr {aa tt } SSh One possible alignment: SS rr SS h {aa tt } --GTAA AGGTC- Another possible alignment: SS rr -G-TAA SS h AGGT-C {aa tt } There are many action sequences mapping SS rr to SS hh.

37 Pair H - Probability BACKUP SLES Each action sequence is associated with a probability: PP aa tt = pp(aa tt aa tt ) tt atch/ ismatch

38 BACKUP SLES Similarity score ynamic Programming ssssssssss SS h, SS rr = PP aa tt = NN h, NN rr {aa tt } aa tt : SS h SSrr + NN h, NN rr + NN h, NN rr Last action: delete Last action: match / mismatch Last action: insert SS rr [0: jj ] SS h [0: ii] aa tt SS rr G?T?AA SS h AGGTA- {aa tt }????? ( i, j) = a + a + a ( i, j ) ( i, j ) ( i, j ) probability dependency (arkov)

39 Recursion Similarly: ( i, j) = a ( i, j) = a ( i, j) = prior ( i, j ) + a ( i, j) + a ( i, j ) ( i, j) ( a ( i, j ) + a ( i, j ) + a ( i, j ) ) BACKUP SLES 0 Read Sequence C G G A Output: Complexity: ssssssssss SS h, SS rr = NN h, NN rr + NN h, NN rr + NN h, NN rr O ( h r Nh Nr ) # haplotype sequences # read sequences