Nanotechnology-inspired Information Processing Systems of the Future Lav R. Varshney University of Illinois at Urbana-Champaign August 31, 2016 Cross-cutting Panel 3
Putting intelligence in (trillions of) things, wearables, and phones On-device Augmented Intelligence Grounded in our physical environment Operating at the speed of thought Running perpetually MITIGATE INFORMATION OVERLOAD (ATTENTION AND CONTEXT-DEPENDENT PRIORITY) FACILITATE CREATIVITY (HETEROASSOCIATIVE MEMORY) SOLVE PROBLEMS (BEYOND CLASSIFICATION) Augment human intelligence by providing us with just the right amount of contextual information when we need it [Image courtesy of Nic Lane, Bell Labs]
Spices, silks, alloys, drug cocktails, science, military strategy/tactics Creativity is the generation of artifacts or ideas that are novel and high-quality [Joint Force Quarterly] [IEEE Spectrum]
A current approach to connect nanotech with applications Devices Circuits/Architectures Systems
A Shannon-inspired approach to connect nanotech with applications CHARACTERIZATION Devices DESIDERATA Circuits/Architectures Systems Theory
How can theory help us in matching application with device? APPLICATION APPLICATION DEVICE (Shannon, 1948) Specificity Sometimes applications are perfectly matched to devices uncoded, circuit-free, delay-free transmission is optimal Generality Always applications can be mapped to intermediate representation to become matched to devices block codes allow any application to run on any device via bit representation
Theory of Specificity: matching condition Theorem An information system with a source p U u and a channel p Y X y x is perfectly matched (i.e. optimal with uncoded transmission) if the basic physical resource of the technology satisfies: b x = D p Y X x p Y and the basic notion of fidelity for the application satisfies: d u, v = log p U V u v Interpretation If Bayesian surprise (cf. creativity) is the basic physical resource of the device and logarithmic loss (cf. inference) is the basic notion of fidelity in the application, perfect matching follows. Redundancy in source is exactly redundancy needed to protect against channel Gaussian source over power-constrained AWGN channel under quadratic distortion
Theory of Generality: intermediate representation and architecture (Fano, 1961) Dear Prof. Varshney, Fig. 1.1 is used to specify what part of the process of communication the textbook is about, namely the transmission of messages chosen from a finite set (representable by binary numbers). In other words, the book is about the boxes labeled "channel encoder" and "channel decoder". The fact that the set of possible messages is finite is basic to information theory, and sometime forgotten. Best wishes for 2014, Bob Fano
Predictions of synaptic microarchitecture Experimentally verified predictions Stochastic characterization of device Optimize storage capacity per unit volume Synaptic connectivity should be sparse Synapses should be small and noisy on average Heavy-tailed synaptic strength distribution Synapses may be discrete-valued (Matsazuki et al., 2001) (Murthy et al., 2001) Relationship between volume and efficacy
Need stochastic benchmarking of nanotech devices to connect to apps Error-delay-energy tradeoff of spintronic logic Limit theorems and optimal architectural principles using information-theoretic optimization Heterogeneous energy allocation is very powerful! [Patil, Shanbhag, Manipatruni, Nikonov, and Young] Ferroelectric FET nanofunctions Rules of thumb for circuit design and specific design algorithms for actual design [Khan, Chatterjee, Duarte, Lu, Sachid, Khandelwal, Ramesh, Hu, Salahuddin]
A Shannon-inspired approach to connect nanotech with applications CHARACTERIZATION Devices What every engineer needs is a good set of limit theorems DESIDERATA Circuits/Architectures Systems Theory FUNDAMENTAL LIMITS [Spielberg, 1989]