JOVIAL JUPYTER OVERRIDE FOR ASTRONOMICAL LIBRARIES. MAURICIO ARAYA, CAMILO VALENZUELA, HUMBERTO FARÍAS, MAURICIO SOLAR, et al.

Transcription

1 JOVIAL JUPYTER OVERRIDE FOR ASTRONOMICAL LIBRARIES MAURICIO ARAYA, CAMILO VALENZUELA, HUMBERTO FARÍAS, MAURICIO SOLAR, et al.

2 WHAT IS JOVIAL? Jupyter-based service for astronomical analysis Integrate astronomical libraries in web-based platform Cloud service of the Chilean Virtual Observatory Status: alpha, beta (Dec)

3 WHAT JOVIAL IS NOT

4 WHAT JOVIAL IS NOT Yet another astronomical library (we already try that last year)

5 WHAT JOVIAL IS NOT Yet another astronomical library (we already try that last year) Yet another astronomical GUI

6 WHAT JOVIAL IS NOT Yet another astronomical library (we already try that last year) Yet another astronomical GUI (too many good competitors)

7 WHAT JOVIAL IS NOT Yet another astronomical library (we already try that last year) Yet another astronomical GUI (too many good competitors) Yet another data discovery App

8 WHAT JOVIAL IS NOT Yet another astronomical library (we already try that last year) Yet another astronomical GUI (too many good competitors) Yet another data discovery App (other VOs are way ahead!)

9 WHAT JOVIAL IS NOT Yet another astronomical library (we already try that last year) Yet another astronomical GUI (too many good competitors) Yet another data discovery App (other VOs are way ahead!)

10 JUPYTER Web-based application IPython Code snippets (several languages) Markup language (markdown) Interactivity Inline visualisation Autocompletion Widgets IPython Notebooks Jupyter

11 JUPYTER Web-based application IPython Code snippets (several languages) Markup language (markdown) Interactivity Inline visualisation Autocompletion Widgets IPython Notebooks Jupyter

12 IN OUR DATA CENTER We have allocated a dedicated data centre for astronomy We have already data access and batch processing capabilities We want to make resources used for interactive data analysis SSH As a Cloud Service VO Services MPI VM VM VM VM VM VM VM VM Cluster Manager 1 2 N Computing Nodes Virtualisation System M Service Nodes

13 What we should put here?

14 What we should put here?

15 1. Python-based EASY PEASY LIBRARIES 2. Numpy-based 3. Matplotlib-based 4. Astropy-based 5. Notebook-aware 6. Advanced UI AstroML GWCS AplPy Astrodendro Astroquery PyVO PyData Astropy

16 1. Python-based EASY PEASY LIBRARIES 2. Numpy-based 3. Matplotlib-based 4. Astropy-based 5. Notebook-aware 6. Advanced UI AstroML GWCS AplPy Astrodendro Astroquery PyVO PyData Astropy

17 1. Python-based EASY PEASY LIBRARIES 2. Numpy-based 3. Matplotlib-based 4. Astropy-based 5. Notebook-aware 6. Advanced UI AstroML GWCS AplPy Astrodendro Astroquery PyVO PyData Astropy

18 INTEGRATING CASA CASANOVA patchelf strategy Both CASA core and tasks work in JOVIAL CASA 4.7 includes MPI support! CASANOVA CASA MPICAS A AstroML GWCS AplPy Astrodendro Astropy Astroquery PyVO PyData

19 INTEGRATING CASA CASANOVA patchelf strategy Both CASA core and tasks work in JOVIAL CASA 4.7 includes MPI support! CASANOVA CASA MPICAS A AstroML GWCS AplPy Astrodendro Astropy Astroquery PyVO PyData

20 INTEGRATING CUPID We wanted CUPID to run in Jupyter CPython wrappers (e.g. AstKeyMap to PythonDictionary) Cython wrappers (e.g. ClumpFind algorithm) CASANOVA Cython Wrappers AstroML GWCS AplPy Astrodendro Astroquery PyVO CASA MPICAS A CUPID Starlink Wrappers PyData Astropy

21 INTEGRATING CUPID We wanted CUPID to run in Jupyter CPython wrappers (e.g. AstKeyMap to PythonDictionary) Cython wrappers (e.g. ClumpFind algorithm) CASANOVA Cython Wrappers AstroML GWCS AplPy Astrodendro Astroquery PyVO CASA MPICAS A CUPID Starlink Wrappers PyData Astropy

22 ADDING OUR LIBRARY ASYDO (ADASS-14) ROI Detect (A&C-16) Automatic Stacking BubbleClumps Volumetric Visualisation Pythonic Magical API ACALIB API NDD/Table Wrappers CASANOVA CASA MPICAS A Cython Wrappers CUPID Starlink Wrappers ROI Detect ASYDO Mayavi AstroML Astrodendro AplPy GWCS Astropy PyData Astroquery PyVO

23 ADDING OUR LIBRARY ASYDO (ADASS-14) ROI Detect (A&C-16) Automatic Stacking BubbleClumps Volumetric Visualisation Pythonic Magical API ACALIB API NDD/Table Wrappers CASANOVA CASA MPICAS A Cython Wrappers CUPID Starlink Wrappers ROI Detect ASYDO Mayavi AstroML Astrodendro AplPy GWCS Astropy PyData Astroquery PyVO

24 ADDING OUR LIBRARY ASYDO (ADASS-14) ROI Detect (A&C-16) Automatic Stacking BubbleClumps Volumetric Visualisation Pythonic Magical API ACALIB API NDD/Table Wrappers CASANOVA CASA MPICAS A Cython Wrappers CUPID Starlink Wrappers ROI Detect ASYDO Mayavi AstroML Astrodendro AplPy GWCS Astropy PyData Astroquery PyVO

25 WORK IN PROGRESS ACALIB API NDD/Table Wrappers ADMIT CASANOVA CASA MPICAS A UWS MPI Cython Wrappers More Slink CUPID Starlink Wrappers ROI Detect ASYDO Mayavi AstroML GWCS AplPy PyData Astrodendro Astropy VM VM VM VM VM VM Astroquery VoSpace PyVO VM Glue Put your library here VM Cluster Manager 1 2 N Computing Nodes Ovirt M Service Nodes

26 FUTURE WORK Beta and production Interactivity (IPython widgets) Jolly (Jovial Local Laboratory) JupyterLab (next evolution) Tutorials for astronomers See you in ADASS 17 in Santiago!

27 JOVIAL JUPYTER OVERRIDE FOR ASTRONOMICAL LIBRARIES MAURICIO ARAYA, CAMILO VALENZUELA, HUMBERTO FARÍAS AND MAURICIO SOLAR

28 AS A CLOUD SERVICE User Experience No installation/upgrade Remote access/processing Flexible (code) Graphical UI Popular! Opportunities Abstraction of HPC User community Less data transfer Documented pipelines