Imaging using GPU. V-K Veligatla, Kapteyn Institute P. Labropoulos, ASTRON and Kapteyn Institute L. Koopmans, Kapteyn Institute

Transcription

1 Imaging using GPU V-K Veligatla, Kapteyn Institute P. Labropoulos, ASTRON and Kapteyn Institute L. Koopmans, Kapteyn Institute

2 Introduction What is a GPU? Why another Imager? Large amount of data to be processed. W-projection is a requirement. Issues to Consider Wide field imaging Full Stokes Speed Experiment with New approaches

3 Basic Imaging Equation V u, v, w = I l, m B l, m 1 l m i ul vm w 1 l m 1 e 2 2 For low frequency wide-field imaging the assumption w~0 does not hold If w=0, then visibilities = FFT(image) But this is not always the case hence WProjection. (S. Bhatnagar and T.J.Cornwell) dl dm

4 W-Projection V u, v, w = I l, m B l, m 1 l m 2 2 e 2 i ul vm w 1 l m dl dm Assuming W is constant for a given plane of UV values V u, v, w = e 2 i [w 1 l m 1 ] 2 2 I l, m B l,m 1 l m 2 Correction CorrectionTerm Term 2 e 2 i ul vm dl dm

5 Block Diagram Imaging+ correcting W planes Corrected Image

6 Wproj Convolution function W=0 W=64 W=127

7 Results (Simulation) Normal Imaging W-Projected

8 Results (Simulation) Normal Imaging W-Projected

9 3C196 W-Corrected

10 Full Stokes (Sim) II QQ U U V V

11 Full Stokes 3C196 I Q U V

12 LOFAR Eor GPU Cluster Configuration 2 x Tesla M1060 (GPU) Computing capability Processing 602 MHz 4GB Memory 16 x Intel(R) Xeon(R) CPU E GHz 12 GB Ram 80 nodes

13 W-Projection Performance Planes 128 Planes 256 Planes 200 T i me I n S ec ond s GpuImager CasaImager

14 Without W-Projection T i me I n S ec ond s GpuImager CasaImager

15 3C196 8 sq. degrees 256 Planes 80 Nodes 244 sub-bands In About 5 mins

16 3C196 (zoomed) 0.5 mjy rms noise at the edge of the image

17 Future Plans We are working on implementation of MVDR on the gpu. Experimenting with Weighting and Convolution schemes in the UV plane. Incorporating a very fast calibration scheme into the imager so both are done together

18 Thank You

19 Introduction What is a GPU? Why another Imager? Large amount of data to be processed. W-projection is a requirement. Issues to Consider Wide field imaging Full Stokes Speed Experiment with New approaches

20 Why use GPU Hundreds of Processors Very well suited for matrix operations. Well suited for parallel tasks. With the latest APIs (CUDA, OpenCL) easy to program.

21 Basic Imaging Equation V u, v, w = I l, m B l, m 1 l m i ul vm w 1 l m 1 e In general W~0 (U,V,W=0). This implies visibilities = FFT(image) 2 But this is not always the case hence WProjection. (T.J.Cornwell et all) 2 dl dm

22 W-Projection V u, v, w = I l, m B l, m 1 l m i ul vm w 1 l m 1 2 e 2 dl dm Assuming W is constant for a given plane of UV values 2 i[w 1 l 2 m 2 1 ] V u, v, w = e I l, m B l, m 1 l m i ul vm e G l, m, w V u, v, w =G l, m, w V u, v, w=0 IFFT V u, v, w =G l, m, w IFFT V u, v, w=0 Ref:Tim Cornwell and Liu dl dm

23 Block Diagram Imaging+ correcting W planes Corrected Image

24 W Correction W=0 W=64 W=127

25 Results (Simulation) Normal Imaging W-Projected

26 Results (Simulation) Normal Imaging W-Projected

27 3C196 W-Corrected

28 Full Stokes (Sim) II QQ U U V V

29 Full Stokes 3C196 I Q U V

30 LOFAR Eor GPU Cluster Configuration 2 x Tesla M1060 (GPU) Compute 1.3 capable MHz 4GB Memory 16 x Intel(R) Xeon(R) CPU 12 GB Ram We have 80 Such Nodes

31 W-Projection Performance Planes 128 Planes 256 Planes 200 T i me I n S ec ond s GpuImager CasaImager

32 Without W-Projection T i me I n S ec ond s GpuImager CasaImager

33 3C196 GPU Imager W-Projection 256 Planes 80 Nodes In About 5 mins

34 3C196 (zoomed)

35 LOFAR Eor GPU Cluster Configuration 2 x Tesla C1060 (GPU) Compute 1.3 capable MHz 4GB Memory 8 x Intel(R) Xeon(R) CPU 12 GB Ram We have 80 Such Nodes

36 Results (Simulation) Normal Imaging W-Projected

37 3C196 W-Corrected

38 W-Projection Performance Planes 128 Planes 256 Planes 200 T i me I n S ec ond s GpuImager CasaImager

39 3C196 GPU Imager W-Projection 256 Planes 80 Nodes In About 5 mins

40 Minimum Variance Distortionless Response Minimizes side lobe gains (cause beam to vary spatially) Computationally intensive Parallel computation possible. Dirty Image Equation: H I k l, m = A k l, m R k A k l, m I k l, m Pixel Intensity at l,m at time instance k A k l, m Antenna steering vector Rk Array correlation matrix (ACM).

41 MVDR The weights in dirty image A(l,m) are replaced with W(l,m) Weight set to minimize influence of interfering sources. W =argmin w W H R W Under the constraint W H A=1 H 1 A R H W = H 1 A R A H I k l, m =W R W Ref: R. Levanda and Amir Leshem, Liu at al., Stoica et al.

42 Challenges H 1 A R W = H 1 A R A H Inverse of Visibility Matrix ACM close to singular. Diagonal loading, (non-linear) dimensionality reduction L-curve type of analysis to select diagonal loading strength

43 Challenges: beamformer power vs time itle:/home/vkrishna/documents/experi/ Creator:MATLAB, The MathWorks, Inc. Vers CreationDate:07/07/ :17:26 anguagelevel:2 Flagged/missing data => Higher Diagonal Loading Factor Higher regularization factor => higher the noise in that epoch Cholesky Factorization to determine level of Regularization.

44 Noisy data

45 Results

46 CPU vs GPU CPU vs GPU 350 GPU CPU 300 Threads equivalent to #processors on system Each Thread processes part of final image Time in Minutes CPU implementation: GPU Implementation: Multiple threads process a single pixel Each thread calculates component of vector Size of Image 2048

47 Optimization Varying Block Size Varying Register Count Multiprocessor Warp Occupancy Multiprocessor Warp Occupancy Registers Per Thread Threads Per Block

48

49

50

51

52

53 Summary Since we produce snapshots we can also correct them for image plane effects on the fly Currently implemented a station beam library Highly scalable New methods of Imaging can be explored

54 Thank You