Integration of Physics and Statistics in Imaging Via Deep Data

Transcription

1 Integration of Physics and Statistics in Imaging Via Deep Data Sergei V. Kalinin Institute for Functional Imaging of Materials Guiding the design of materials tailored for functionality 1 ORNL is managed by UT-Battelle for the US Department of Energy

2 More than imaging J.J. Guo et al., Nat. Comm. 5, 5389 (2014) Nature 515, 487 (2014) Atomic positions can be determined to <10-pm precision Bond length: Chemical reactivity, catalytic activity Bond angles: Magnetism and transport Configurations and repeating elements? 2

3 Imaging Dynamic matter: information dimension Static matter Functional matter Dynamic matter Controlled matter Big data Unsupervised learning Correlative learning Image recognition In-situ Control Electronic Structure Ab Initio dynamics Theory Molecular Dynamics Multiscale 3

4 Imaging: What do the atoms do? Our scientific paradigm is shifting Classical concept Synthesis Characterization Theory Computation Expanding to include Data mining Correlative functional imaging Local theory-experiment matching of multi-dimensional (multimodal), spatially and temporally resolved information 4 Institute for Functional Imaging of Materials (IFIM) Establish synergy between imaging disciplines Bridge physical imaging with theory via big data and data analytics to design new materials Leverage ORNL strengths in Physics and chemistry on the atomic scale in real space Mesoscale structure and functional probing Big data and predictive theories

5 Approach 0. Getting big data: making imaging tools a part of data infrastructure Physics: Why something happens 1. Big data: How does it happen? Unsupervised learning, clustering, and visualization Biggest hurdle: Language/ elementary tools 2. Deep data: How can we understand? Physics informed data analytics/ supervised methods Biggest hurdles: Mathematical framework, scalability of computational tools 3. Smart data: How can we do better? Feedback and expert/ai systems Biggest hurdles: Don t know where to start, but it is possible 5

6 Chemistry Materials Science Mass Spectrometry Optical Imaging Atom Probe Tomography Global Security Biology Chemical Imaging INSTITUTE FOR FUNCTIONAL IMAGING OF MATERIALS Electron Microscopy 6 Biomedical Technology Scanning Probe Microscop y Environment Neutron Imaging Level 0: Getting big data 1. Synergy of microscopies 2. Enabling technologies 3. Novel probes

7 Data Generation and Utilization in SPM We realized we are doing big data Kalinin, Jesse, Proksch, Information Acquisition & Processing in Scanning Probe Microscopy, RD Mag 2008 Single frequency methods: Band excitation: SPM tip confines electric/thermal field in material and probes associated responses Fundamental physics of stimulus-induced transformation requires high (3,4,5) dimensional measurements -> large data volumes/analysis times Need approaches to visualize and reduce data (big data) and extract relevant information (deep data)

8 G-mode: Full Information Recovery Instrumental limit: photodetector bandwidth (~10 MHz) x DAQ performance (32 Bit) Single frequency/heterodyne: lock-in compression to ~ 1 khz Band excitation: 10 2 bins at ~ 1 khz = 100 khz G-mode: full streaming at ~10 MHz Applications: Fast ferroelectric loop imaging (x7,000 compared to standard method) Full dynamics in Kelvin Probe Force Microscopy (x1,000 to classical method) W 2 spectroscopic imaging (no classical analogs) A. Belianinov et al., Nat. Comm 6, 6550 (2015) Future: Fast force-distance curve imaging Detection of spurious and transient phenomena Variable density imaging

9 Data Generation in Electron Microscopy electron beam Sub-Å probe To scan coils Specimen Advanced DAQ One dimensional excitation signal Fast Direct Electron Detection Complex detection signals 0D: bright/dark field intensity 1D: energy loss spectra 2D: ptychography/diffraction Can be realized on 2D (image) and 3D (focal series, tilt series) spatial grids 9

10 Data Generation in Electron Microscopy 10 Instrumental limit: Electron flux ( e/s) x detector performance (1 Bit/electron) Detectors: Information per electron? Storage, visualization, and curating

11 Ptychographic Imaging The standard STEM is exquisitely tuned to capture material structure (in projection) However, the transmitted electrons carry far more information than is captured by monolithic detectors: sub-atomic diffraction pattern Grain boundary in BiFeO 3 11 Capture full information stream Emulate monolitic detectors: any geometry Multivariate analysis

12 Material Sculpting and Electrochemical writing? D. Eigler Long long time ago SPM facility far far away A. Borisevich S. Jesse Q. He Can we use induced single atom dynamics to fabricate bulk 3D structures? Synergy of e-beam writing and advanced feedback and control 3D atomic fabrication: quantum computing, spintronics, etc. R. Ishikawa et al., Nano Lett. 14, 1903 (2014)

13 Level 1: Big Data Unsupervised Learning and classification 13

14 Imaging: A. Borisevich Q. He Sample: V. Guilants Image analysis: S. Jesse A. Belianinov

15 Normal Modes: Analysis of Nearest Neighbors Principal Component Analysis A i ( ω ) = a w ( ω ) j PCA transforms the data such that the greatest variance by any projection lies on the first coordinate ik k j Chemistry Physics K-means Clustering k-means clustering aims to partition the n observations into k sets (k n) S = {S 1, S 2,, S k } so as to minimize the within-cluster sum of squares kk arg min xx jj μμ ii 2 ii=1 xx jj SS ii

16 Image Analytics: Enabling the Discovery Multivariate analysis of atomic shapes and nearest neighborhoods Local physics and chemistry from connectivity and distortions Image based analysis for quantitative improvement of Molybdenum Vanadium based complex oxide catalysts for propane ammoxidation Q He, J Woo, A Belianinov, VV Guliants, A Borisevich; ACS nano, DOI: /acsnano.5b00271, (2015)

17 Local property mapping STM image of (11) at L-He Superconductive gap map di/dv(a.u.) 3 nm Filtered STM Image a b 0 Local crystallography Bias (V) 15 a The defect preserves lattice continuity, but is associated with change in molar volume and lattice parameter - Guinier-Preston zone. Superconductivity is suppressed at the defect.

18 Local structure-property coupling Surface atomic structure Tunneling spectral image Structure descriptors: 1. Atom height 2. Molar volume 3.. Electronic property descriptors: 1. PCA components of spectra 2. Superconductive gap

19 Atomic-Scale Structure and Functionality Structure Image Positions Physics Identify & Classify Structure Analysis Phase 2 Phase 1 Genomic Library Properties Spectra 3D 4D 5D Register & Deconvolute Multivariate Analysis Physics and chemistry on single defect level Need new language: 1. What are structural descriptors? 2. How do we define local symmetry, phases and ferroic variants? 3. How do we introduce and quantify translational symmetry? Scattering methods: completeness of library Macroscopic properties: averaging rules What do we learn: 1. Structure-property relationship on single atom, molecule, and defect level 2. Libraries of structure-property relationships 3. Feedback to theory through microscopic degrees of freedom

20 Imaging to materials by design Classical approach: Synthesis Characterization Theory Bulk Crystal Chemical Space Functional Properties T m γ Y P Ρ 20 Big data from imaging: Stochastic library φ (x,y,v, ) Y (x,y,v, ) P (x,y,v, ) Future: Libraries of preferred local configurations: what is relevant Structural + functional imaging: stochastic combinatorial libraries Theory based prediction Need: Functional probes High-resolution structural imaging Theoretical models Big/Deep/smart data

21 Level 2: Deep Data Transition from correlative to causative analysis 1. Theoretical microscope 2. Physics-constrained un-mixing 3. Inverse problems 21

22 Fundamental Science via Local Degrees of Freedom Atomistic Imaging Atomistic Simulation Theoretical microscope Bayesian inference Local functionalities calculated from observables Improved theory Can we complement experiment by theory to visualize invisible degrees of freedom and extract functionalities of interest? Can we refine and improve theory by factoring in experimental data (via Bayesian inference) Can we develop approach to extract relevant macroscopic parameters from experiment and simulations (e.g. via Fischer information)

23 Vasudevan et al submitted Physics-constrained un-mixing Real space Spectral space Current-voltage curves at each location y x Voltage (V) 5 0 V p time (s) 1 0 4D dataset I = f(x, y, V, V p ) Current (na) Fitting to physical models Eigenvector 1 Loading 1 CFO Extracting Physical meaning Eigenvector 2 Loading 2 Needs R 1 R 2 R 2 R 1 Bottom electrode BFO A Strelcov et al, ACS Nano 2014, 2015 Need: Un-mixing with user-defined constraints on the endmembers or loading maps Combined spatial and spectral unmixing Incorporate physics (symmetry, non-negativity, material parameter/models, etc.)

24 Mesoscale Structure and Dynamics: Inverse Problem Pt nanoparticle growth under e-beam Source image Binary image Detected particles Mesoscale dynamics Reaction/transport Ginzburg-Landau Theory Molecular Dynamics R. Unocic Growth controlled by Pt transport: Particle boundary conditions: Local growth velocity: c t dc dn = D c boundary = qc v growth = kc Can we learn: Free energy expansions Thermodynamics Universality classes Frozen disorder Reaction/diffusion kinetics Simulated concentration field COMSOL Multiphysics PDA solution Simulation workflow Particles boundary Concentration Matlab Exp. boundary detection Data comparison

25 Need: Supporting real-time image analytics STEM/EELS, SPM High throughput image capture Multi-modal: -High-angle annular dark field (HAADF) detector - Electron energy loss spectroscopy O(1000) of images per experiment Transfer files to HPC storage and convert data to HDF5 format Data motion via BBCP/GridFTP Data conversion from DM3 stacks to HDF5 slices Standard data format (HDF5) Data layout conducive to HPC algorithms Massively parallel Image processing/ feature detection Denoise Image Identify atoms Identify lattice Thousands of images/sec Built on MPI+Fortran Near linear scaling some limitations as file counts get extremely large Massively parallel electronic structure calculation Massively parallel study of hundreds to thousands of individual configurations Calculation of electronic structure Based on Density Functional Theory 0(1000) of configurations Minutes Seconds Seconds Minutes

26 Level 3: Smart Data - Supervised Learning - Context - History

27 From Human Expert to Automatic Systems Expert Control Synthesis of expertise: factor in human expert knowledge PFM Signal (a.u.) Decision making Experimental data Bias (V) Future: Automated analysis of routine data Identification of anomalies Initial training of new practitioners Data centers: information based on knowledge Automatic Expert System Timeline User Model Context search: published results data mining/social networks

28 Smart data: Google car, cancer screening, expert systems Understanding Data Statistical measures of orientation of texture for the detection of architectural distortion in prior mammograms of interval-cancer J. Electron. Imaging. 2012;21(3): doi: /1.jei

29 Classical Instrumental Research Paradigm Instrument Control/data acquisition Researcher Community Social networking/education Publications/citations 1. Only small fraction of data stream from the instrumentation is captured 2. Only small fraction of captured data is analyzed, interpreted, and put in the context 3. Human-machine interaction during acquisition is often slow and can be non-optimal 4. Human interpretation of data is limited: bias and ignoring serendipity 5. Information propagation and concept evolution in scientific community is extremely slow and affected by non-scientific factors

30 Cloud-Based Imaging: Integrated Instrumental Network 1. Multiple geographically-distributed data generation node 2. Full capture of instrumental data stream 3. Coordination of protocols and data/metadata across the cloud 4. Cloud-based processing and dimensionality reduction 5. Community-wide analytics

31 Institute for Functional Imaging of Materials Goal: guide the design of materials tailored for functionality via probing, understanding, and designing local structure-property relationships on atomic and nanometer level Means: Synergy and coordination between imaging disciplines Linking theory and imaging on the level of microscopic degrees of freedom via data analytics Big, deep, and smart data in materials exploration and design New analysis Big data Imaging Theory New probes Static Unsupervised learning Functional Correlative learning Electronic Structure Ab Initio Dynamic Image recognition Controlled In-situ control Molecular Dynamics Multiscale New control 31

32 32 Thank you for coming!