Realization and test of an Artificial Neural Network based on Superconducting Quantum Interference Devices

Size: px

Start display at page:

Download "Realization and test of an Artificial Neural Network based on Superconducting Quantum Interference Devices"

Kerry Reeves
5 years ago
Views:

1 Realization and test of an Artificial Neural Network based on Superconducting Quantum Interference Devices Chiarello F 1, Carelli P 2, Castellano M G 1, Torrioli G 1 1 IFN-CNR, via Cineto Romano 42, Rome, Italy. 2 DSFC, Università dell'aquila, Via Vetoio, Coppito L'Aquila, Italy. fabio.chiarello@ifn.cnr.it Abstract. We propose a scheme for the realization of an artificial neural network based on Superconducting Quantum Interference Devices (SQUIDs). In order to demonstrate the operation of this scheme we designed and successfully tested at 350mK a small network trained to implement an XOR gate. These measurements are performed in the classical regime, but the scheme is entirely based on elements tested in quantum computing applications (flux qubits, flux discriminator, superconducting couplings) and it is suitable for future implementation of solid state quantum neural networks. 1. Introduction Artificial neural networks (ANN) are effective tools to deal with complex problems such as data mining, handwriting and images recognition, bioinformatics, etc. [1 3]. Some authors suppose that the introduction of quantum mechanical principles in artificial neural networks (Quantum Neural Networks) [4 6] could greatly enhance our computing possibilities as it appears to be for quantum computing [7,8], and there are some examples of the possible implementation of such idea [9]. Superconducting circuits can be used for the realization of classical artificial neural networks [10 13], exploiting the flexibility and rapidity of superconducting electronics [14,15] which allows clock rates not possible with conventional silicon electronics (from tens up to hundreds of GHz) at the cost of the use of cryogenic technologies. On the other hand superconducting devices allows the realization of effective quantum computing elements (qubits and quantum gates) [16 18] and for these reasons are promising candidates for a possible implementation of solid state quantum neural networks. In the present work we propose and demonstrate the effective and robust operation of an artificial neural network realized by coupled SQUIDs (Superconducting Quantum Interference Devices). We consider a very simple, but nontrivial feedforward network, trained in order to implement an XOR gate. The present results are all obtained in the classical regime, but it is interesting to notice that the used SQUIDs, couplings and readout devices, are the same used for quantum computing applications. In other words, we are testing in the classical regime an artificial neural network the building blocks of which are designed to work as quantum devices. This is a very preliminary but necessary step towards the realization of a superconducting quantum neural network. In section 2 we describe the artificial neural network and the devices we used. In section 3 we present the experimental setup and discuss the experimental results. 1

2 2. SQUID Artificial Neural Network An Artificial Neural Network can be described by an oriented graph where each vertex k is an artificial neuron with an internal state x k (fig. 1a). Any oriented edge connects a neuron k to a second neuron l with an associated weight w kl. The evolution of the k th neuron from its initial state x k to the new state y k is affected by the signals coming from the coupled neurons: their states x l are multiplied by the weights w lk and summed together (plus an eventual bias value b k ), and the final state is obtained by applying an appropriate activation function f(x) to this result (Fig.1b): yk f = wlkxl + bk. (1) l The activation function f(x) depends on the required network architecture and can be a linear or a threshold function, a sigmoid and so on. Once an appropriate architecture has been chosen, the network must be trained in order to execute a specific task and this is done by properly modifying weights w lk and biases b k in order to have correct responses to applied input patterns. Fig.1. (a) Example of an artificial neural network. (b) Scheme of a single neuron. In this paper we demonstrate the possibility to realize a general artificial neural network with a system of SQUIDs. For this purpose we need a network as simple as possible but still presenting nontrivial characteristics in order to ensure a general use. A good and typical candidate is a network of five neurons disposed in three layers, trained in order to implement an XOR gate. This can be accomplished by choosing a network with the architecture of Fig.1a with three distinct layers: an input layer (neurons 1 and 2 ), a hidden layer (neurons 3 and 4 ) and an output layer (neuron 5 ). One possible set of values suitable for the required task is the following: weights w 13 = w 24 = w 35 = w 45 = +w and w 14 = w 23 = w; biases b 1 =b 2 =0, b 3 = b 4 = w and b 5 =+w, where w is a positive constant, but the network is tolerant to small variations of these values. The neurons activation function is a threshold function with f(x) = 1 for x < 0 and f(x)=+1 for x > 0. In this case the output neuron will respond with a +1 state if the input values are opposite, and with -1 if the input values have the same sign. In our scheme the neurons are implemented by SQUIDs (Superconducting Quantum Interference Devices) coupled by superconducting flux transformers. The magnetic fluxes in the SQUIDs are used as neurons states. The weights w ik are fixed by the coupling strengths (and verses) of the superconducting transformers (Fig.2). In this case the couplings are fixed so that the network is already trained. However it is possible to design a more complex circuitry where the magnetic couplings are adjustable by means of tunable couplings [19,20], in order to have a flexible network that can be trained during the running. 2

3 The neurons of each layer have different functions, so in order to simplify the circuitry we use different devices suited for each different task. We describe these different devices and their arrangement according to the scheme given in Fig.2, which is essentially the application of the scheme in Fig.1a. The neuron in the output layer (neuron 5 ) must sum the output fluxes of neurons 3 and 4 plus the bias B2, apply the step function to the resulting amount and finally return the result when required by an appropriate readout signal (clock CK2). We implement this neuron by a dc SQUID used as flux discriminator (Fig.2), a device typically used for superconducting qubits readout [21]. A dc SQUID consists of a superconducting loop of inductance l interrupted by two (nominally) identical Josephson junctions, each of critical current J and capacitance C, which can be biased by a current I applied to its terminals. The device responds with a voltage V with a behavior controlled by the 15 magnetic flux Φ in applied to its loop. When 2 Jl Φ0 /( 2π ) (where Φ 0 = h/ ( 2e) Wb is the flux quantum) the device behaves approximately as a single Josephson junction with capacitance C = C and tunable critical current given by: 0 2 in I0 = 2Jcos π Φ. (2) Φ0 If we apply a current smaller than the critical value ( I < I 0 ) the device remains in the superconductive state with a zero voltage response (V=0), but if I overcomes the critical value ( I > I 0 ) there is a transition to the voltage state (V >0). This device can be used as a magnetic flux detector by applying a current ramp and measuring the critical value I 0 for which the transition to the voltage state occurs, and then obtaining the magnetic flux value Φ in (modulo Φ 0 by inverting eq.2). In order to use the device as the final neuron we apply a periodic sequence of short current ramps used as clock readout signals (CK2). The final digital result is obtained by comparing the analogue output with an appropriate threshold value. Fig.2. Scheme of the SQUID neural network: current generators as input layer neurons 1 and 2, double SQUIDs as hidden layer neurons 3 and 4, and a dc SQUID as output layer neuron 5. 3

4 The two neurons in the hidden layer (neurons 3 and 4 ) must sum the input magnetic fluxes (plus the bias B1) and apply a step function in correspondence of a clock signal CK1, as done by neuron 5, but moreover they must memorize the result as a flux state until the operation in the next layer will be performed. The ideal device for this task is the so called double SQUID, a superconducting loop of inductance L interrupted by a dc SQUID similar to that described for the output layer, here used as tunable Josephson junction (Fig.3a). Here we have two distinct flux controls: the large loop of the double SQUID can be biased by a magnetic flux Φ x used as input, while the small loop of the dc SQUID can be biased by a second flux Φ c [22 24]. The total magnetic flux Φ in the large loop is the output response of the device. Fig.3. (a) Scheme of the double SQUID with the applied magnetic fluxes Φ x and Φ c. (b) The flux Φ x modifies the potential symmetry. In the double well case the device behaves as a memory element. (c) The flux Φ c controls the barrier height. By completely removing the barrier the flux state is reset. (d) By returning to the double well case the state is re-trapped in one of the two wells depending on the total input flux Φ x, implementing in this way the step function and the memorization. ( ) In the limit 2 Jl Φ0 / 2π the double SQUID can be described by a mechanical equivalent with a particle of effective mass C 0 =2C moving along the Φ direction in the potential: ( ) 2 Φ Φx Φ 0 Φ U = I0 ( Φc ) cos π, (3) 2L 2π Φ0 where the critical current I 0 = 2J cos(π Φ c /Φ 0 ) can assume also negative values. For Φ x = 0 the potential is symmetric and presents a single well in Φ = 0 when I0( Φ ) / ( 2 c > Φ0 π L), and two identical wells separated by a barrier otherwise. Therefore the flux Φ c controls the potential shape, with a modification between the double-well to the single-well shape. On the contrary the flux Φ x modifies the potential symmetry, and in particular the sign of Φ x determines which of the two wells will be the lowest (Fig.3b). We can now describe the use of this device as a neuron. In the double-well case the state is blocked in one of the two wells, protected by the barrier and practically independent on the input flux Φ x if this is not too large (Fig.3b). This is the memory state, where the output flux Φ is quite constant and independent on the sum of the input values. If we remove the barrier arriving till to the single-well case (by applying a pulse on the control flux Φ c ) the flux state is reset (Fig.3c); by returning back to the double well case the system is blocked (memorized) again in one of the two minima which will be chosen according to the sign of the total input flux Φ x, realizing in this way the step function. Notice that the same device, the double SQUID, has been successfully used as a flux qubit with a manipulation similar to that described here [25 27] but in the quantum regime. The two neurons in the input layer (neurons 1 and 2 ) must only provide the input magnetic flux and consist of on-chip planar coils fed by room temperature current generators. 4

5 3. Experimental setup, results and discussions We have designed a series of chips based on the scheme in Fig.2, which have been fabricated at the MIT Lincoln Laboratory facility by using the deep submicron (DSM) with fully-planarized Nb-(Al- AlOx)-Nb trilayer process, and tested at IFN-CNR Rome in a He 3 cryostat Heliox at 350mK. The nominal parameters of the dc SQUID are J 8 μa, C 10 ff and l 10 ph; for the double SQUIDs we have identical values for the inner dc SQUIDs, and a large inductance L 230 ph. The network is feed by two bias currents (B1 and B2), by two input currents (I1 and I2), and by a couple of clock signals (CK1 and CK2) with a repetition rate of 2 khz. The clock CK1 consists of a series of short current pulses used to reset the states of the two double SQUIDs (neurons in the hidden layer). The readout clock pulse CK2 is shifted after the pulse on CK1 of about 200μs, and has the shape of short ramps sweeping the current across the critical value in order to realize the dc SQUID readout. The characterization of the system obtained by changing the flux biases [26] allows to determine the couplings between feed coils and the SQUID loops from the periodicity of the SQUIDs characteristics, obtaining M B2 = 2.5 ph for the coupling between B2 and the dc SQUID, M in = 6.2 ph for the coupling between I1, I2, B1 and the relative large loops in the SQUIDs, M ck = 3.8 ph for the coupling between CK1, CK2 and the relative small loops in the double SQUIDs. We measured a transforming ratio of 0.7% for the coupling between the double SQUIDs and the dc SQUID. These values are in good agreement with the design parameters. This characterization also allows to optimize the values for the biases B1 and B2 and the pulses CK1 and CK2, although the system continues to operate correctly for variations up to 25% away from the optimal values. In order to demonstrate the correct operation of the XOR circuit we apply the clock sequence and acquire the output voltage; by repeating this for different input values of I1 and I2 we obtain the color map in Fig.4, where the measured output critical current is plotted as a function of the two input currents values I1 and I2. We observe large regions where the system works correctly (about 25% of tolerance in the inputs space), indicating the robustness of the network for variations of the input values. Fig.4. Colour map of the XOR neural network response to the inputs I1 and I2. There are large regions of correct operation, where the response is 1 only in case of opposite inputs. 5

6 Fig.5 reports an example of network operation versus time, with the input I1 (upper plot) periodically oscillating between -80μA (logic state 0 ) and +80μA (logic state 1 ), and the input I2 (middle plot) between -150μA (logic state 0 ) and +40μA (logic state 1 ) with a shifted phase (these values correspond to the dashed lines in fig.4). In the lower graph we plot the measured output critical current (continuous line), the value of which fluctuates around 13.15μA for the logic state 0 and around 13.45μA for logic state 1. The digital output is obtained by using a step function on the acquired output current with a threshold at 13.3μA, and the result is correct in 95% of cases. The red dotted curve shows the expected value corresponding to a correct XOR operation. Fig.5. Operation of the XOR neural network for varying I1 and I2 inputs. The red dotted line in the lower plot is the expected result. In our case the readout dc SQUID (neuron 5) must discriminate a difference of about ΔΦ 7 mφ 0 in order to distinguish between the double SQUID flux states (which are separated by about a flux quantum), because the relative couplings are approximately 0.7%. The dc SQUID presents a gray zone where the measurement is uncertain because of thermal noise; the amplitude ΔΦ of this zone can be measured by applying a fixed bias flux and evaluating the standard deviation of the measured value, and we obtain ΔΦ 3 mφ 0 at 350 mk, which is smaller than ΔΦ and allows the observed 95% discrimination, and ΔΦ 9 mφ 0 at 4.2 K, which gives only a 70% discrimination. The observed fluctuation on I out, the reduced fidelity of 95% as well as the necessity to work at T=350mK and not at a more comfortable Helium temperature T = 4.2 K, are therefore all due to the small coupling between SQUIDs (0.7%), fixed in the design phase. The doubling of the coupling strength (very simple to obtain starting from the design phase) will completely remove this problem in future devices. The observed results are obtained with a clock of 2 khz, limited by the readout electronics. We expect that the main limit in the device speed is mainly due to relaxation times, which are essentially related to the parallel of the junctions capacitances C and of the junctions leaks R, with τ RC. In our case C 60 ff and we suppose R < 100 kω corresponding to τ < 10 ns, so that we expect an operating rate below 100 MHz. This operating rate can be greatly enhanced thanks to the same techniques used for classical superconducting electronics, for example by introducing shunting resistances, and one can expect rates similar to that of analogous superconducting electronics, which goes from 10 GHz to 100 6

7 GHz [14,15]. Such a fast neural network can be a very interesting tool in applications where it is necessary to deal with a large amount of data in very short times and in the same time the cryogenic environment is not a burden, as it is for triggering events in high-energy physics experiments, or for the multiplexing of large amounts of data traffic in telephony, internet or video communications. A completely different interest involves quantum neural network where there is the opposite problem: it is necessary to have a very small dissipation in order to avoid decoherence and increase the number of correct quantum operations. In this case the upper limit for the rate, typically below 1 GHz, is due to the requirement of adiabaticity for the manipulation of the quantum states. The particular configuration here studied is important from a classical point of view because it allows demonstrating the operation of a small but nontrivial classical neural network. On the contrary it will be necessary to develop and test a different proper quantum architecture in order to really prove the operation as a quantum neural network. 4. Conclusions We have designed and tested an artificial neural network based on a series of SQUIDs in order to implement an XOR gate, a very simple system yet having a non trivial behaviour. The network has been successfully operated in the classical regime at the base temperature of 350mK, showing good operation margins on control parameters. All the building blocks of this scheme are elements already tested for quantum computing applications: two neurons are flux qubits and one is a standard flux discriminator. Once demonstrated the operation of this simple scheme at least in the classical regime, for the next future two different developments can be envisaged: from one side one can improve the performances of the classical system by introducing shunting resistances and optimizing the design in order to raise the operating speed and obtain an ultrafast classical neural network; on the other side one can test the system in the quantum regime with a proper quantum architecture in order to realize a solid state quantum neural network. Acknowledgments This work was supported by Italian MIUR under the PRIN2008 C3JE43 project. We thanks W. Oliver and the MIT Lincoln Laboratory LTSE team for device fabrication. References [1] Haykin S S 1994 Neural networks: a comprehensive foundation (Macmillan) [2] Hassoun M H 1995 Fundamentals of artificial neural networks (MIT Press) [3] Anderson J A 1995 An Introduction to Neural Networks (MIT Press) [4] Bonnell G and Papini G 1997 Quantum neural network International Journal of Theoretical Physics [5] Gupta S and Zia R K P 2001 Quantum Neural Networks Journal of Computer and System Sciences [6] Kouda N, Matsui N, Nishimura H and Peper F 2005 Qubit neural network and its learning efficiency Neural Comput & Applic

8 [7] Nielsen M A and Chuang I L 2011 Quantum Computation and Quantum Information: 10th Anniversary Edition (Cambridge University Press) [8] Pérez-Delgado C A and Kok P 2011 Quantum computers: Definition and implementations Phys. Rev. A [9] Neigovzen R, Neves J L, Sollacher R and Glaser S J 2009 Quantum pattern recognition with liquid-state nuclear magnetic resonance Phys. Rev. A [10] Harada Y and Goto E 1991 Artificial neural network circuits with Josephson devices Magnetics, IEEE Transactions on [11] Mizugaki Y, Nakajima K, Sawada Y and Yamashita T 1994 Implementation of new superconducting neural circuits using coupled SQUIDs Applied Superconductivity, IEEE Transactions on [12] Rippert E D and Lomatch S 1997 A multilayered superconducting neural network implementation Applied Superconductivity, IEEE Transactions on [13] Onomi T, Maenami Y and Nakajima K 2011 Superconducting Neural Network for Solving a Combinatorial Optimization Problem IEEE Transactions on Applied Superconductivity [14] Kimura T, Kajino K, Watanabe M, Horii Y, Inoue M and Fujimaki A 2009 Operation of Toggle Flip-Flop Circuits Up to 500 GHz Based on Vertically-Stacked High-Temperature Superconductor Josephson Junctions IEEE Transactions on Applied Superconductivity [15] Filippov T, Dorojevets M, Sahu A, Kirichenko A, Ayala C and Mukhanov O Bit Asynchronous Wave-Pipelined RSFQ Arithmetic-Logic Unit IEEE Transactions on Applied Superconductivity [16] Clarke J and Wilhelm F K 2008 Superconducting quantum bits Nature [17] DiCarlo L, Chow J M, Gambetta J M, Bishop L S, Johnson B R, Schuster D I, Majer J, Blais A, Frunzio L, Girvin S M and Schoelkopf R J 2009 Demonstration of two-qubit algorithms with a superconducting quantum processor Nature [18] Reed M D, DiCarlo L, Nigg S E, Sun L, Frunzio L, Girvin S M and Schoelkopf R J 2012 Realization of three-qubit quantum error correction with superconducting circuits Nature [19] Castellano M G, Chiarello F, Leoni R, Simeone D, Torrioli G, Cosmelli C and Carelli P 2005 Variable transformer for controllable flux coupling Appl. Phys. Lett [20] Bialczak R C, Ansmann M, Hofheinz M, Lenander M, Lucero E, Neeley M, O Connell A D, Sank D, Wang H, Weides M, Wenner J, Yamamoto T, Cleland A N and Martinis J M 2011 Fast Tunable Coupler for Superconducting Qubits Phys. Rev. Lett

9 [21] Cosmelli C, Carelli P, Castellano M G, Chiarello F, Leoni R and Torrioli G 2001 A hysteretic dc SQUID reading the flux states of an rf SQUID IEEE TRANSACTIONS ON APPLIED SUPERCONDUCTIVITY [22] Han S, Lapointe J and Lukens J E 1989 Thermal activation in a two-dimensional potential Phys. Rev. Lett [23] Chesca B 1994 On the theory of the symmetrical double SQUID Physica C: Superconductivity [24] Castellano M G, Chiarello F, Leoni R, Mattioli F, Torrioli G, Carelli P, Cirillo M, Cosmelli C, de Waard A, Frossati G, Grønbech-Jensen N and Poletto S 2007 Catastrophe Observation in a Josephson-Junction System Phys. Rev. Lett [25] Poletto S, Chiarello F, Castellano M G, Lisenfeld J, Lukashenko A, Carelli P and Ustinov A V 2009 A tunable rf SQUID manipulated as flux and phase qubits Physica Scripta T [26] Poletto S, Chiarello F, Castellano M G, Lisenfeld J, Lukashenko A, Cosmelli C, Torrioli G, Carelli P and Ustinov A V 2009 Coherent oscillations in a superconducting tunable flux qubit manipulated without microwaves NEW JOURNAL OF PHYSICS 11 [27] Castellano M G, Chiarello F, Carelli P, Cosmelli C, Mattioli F and Torrioli G 2010 Deep-well ultrafast manipulation of a SQUID flux qubit New J. Phys

Fabio Chiarello IFN-CNR Rome, Italy

Fabio Chiarello IFN-CNR Rome, Italy Italian National Research Council Institute for Photonics and Nanotechnologies Elettronica quantistica con dispositivi Josephson: dagli effetti quantistici macroscopici al qubit Fabio Chiarello IFN-CNR