Refine
Document Type
Way of publication
- Open Access (3)
Language
- English (39)
Keywords
- RRAM (37)
- neural network (12)
- HfO2 (11)
- Multilevel switching (10)
- memristive device (8)
- memristive switching (4)
- Neural network (3)
- resistive switching (3)
- In-Memory Computing (2)
- variability (2)
Institute
BTU
We explore the stochastic switching of oxide-based memristive devices by using the Stanford model for circuit simulation. From measurements, the device-to-device (D2D) and cycle-to-cycle (C2C) statistical variation is extracted. In the low-resistive state (LRS) dispersion by D2D variability is dominant. In the high-resistive state
(HRS) C2C dispersion becomes the main source of fluctuation. A statistical procedure for the extraction of parameters of the compact model is presented. Thereby, in a circuit simulation the typical D2D and C2C fluctuations of the current–voltage (I-V) characteristics can be emulated by extracting statistical parameters of key model parameters. The statistical distributions of the parameters are used in a Monte Carlo simulation to reproduce the I-V D2D and C2C dispersions which show a good agreement to the measured curves. The results allow the simulation of the on/off current variation for the design of memory cells or can be used to emulate the synaptic behavior of these devices in artificial neural networks realized by a crossbar array of memristors.
In this work, three different RRAM compact models implemented in Verilog-A are analyzed and evaluated in order to reproduce the multilevel approach based on the switching capability of
experimental devices. These models are integrated in 1T-1R cells to control their analog behavior by means of the compliance current imposed by the NMOS select transistor. Four different resistance
levels are simulated and assessed with experimental verification to account for their multilevel capability. Further, an Artificial Neural Network study is carried out to evaluate in a real scenario the viability of the multilevel approach under study.
Resistive random access memory (RRAM)-based hardware accelerators are playing an important role in the implementation of in-memory computing (IMC) systems for artificial intelligence applications. The latter heavily rely on vector-matrix multiplication (VMM) operations that can be efficiently boosted by RRAM devices. However, the stochastic nature of the RRAM technology is still challenging real hardware implementations. To study the accuracy degradation of consecutive VMM operations, in this work we programed two RRAM subarrays composed of 8x8 one-transistor-one-resistor (1T1R) cells following two different distributions of conductive levels. We analyze their robustness against 1000 identical consecutive VMM operations and monitor the inherent devices’ nonidealities along the test. We finally quantize the accuracy loss of the operations in the digital domain and consider the trade-offs between linearly distributing the resistive states of the RRAM cells and their robustness against nonidealities for future implementation of IMC hardware systems.
The stochastic nature of the resistive switching (RS) process in memristive devices makes device-to-device (DTD) and cycle-to-cycle (CTC) variabilities relevant magnitudes to be quantified and modeled. To accomplish this aim, robust and reliable parameter extraction methods must be employed. In this work, four different extraction methods were used at the production level (over all the 108 devices integrated on 200-mm wafers manufactured in the IHP 130-nm CMOS technology) in order to obtain the corresponding collection of forming, reset, and set switching voltages. The statistical analysis of the experimental data (mean and standard deviation (SD) values) was plotted by using heat maps, which provide a good summary of the whole data at a glance and, in addition, an easy manner to detect inhomogeneities in the fabrication process.
This work presents a quasi-static electrical characterization of 1-transistor-1-resistor memristive structures designed following hardness-by-design techniques integrated in the CMOS fabrication process to assure multi-level capabilities in harsh radiation environments. Modulating the gate voltage of the enclosed layout transistor connected in series with the memristive device, it was possible to achieve excellent switching capabilities from a single high resistance state to a total of eight different low resistance states (more than 3 bits). Thus, the fabricated devices are suitable for their integration in larger in-memory computing systems and in multi-level memory applications.
Index Terms—radiation-hard, hardness-by-design, memristive
devices, Enclosed Layout Transistor, in-memory computing
The impact of artificial intelligence on human life
has increased significantly in recent years. However, as the
complexity of problems rose aswell, increasing system features
for such amount of data computation became troublesome due
to the von Neumann’s computer architecture. Neuromorphic
computing aims to solve this problem by mimicking the parallel
computation of a human brain. For this approach, memristive
devices are used to emulate the synapses of a human brain. Yet,
common simulations of hardware based networks require time
consuming Monte-Carlo simulations to take into account the
stochastic switching of memristive devices. This work presents
an alternative concept making use of the convolution of the
probability distribution functions (PDF) of memristor currents
by its equivalent multiplication in Fourier domain. An artificial
neural network is accordingly implemented to perform the
inference stage with handwritten digits.
In this work, we present an integrated read and programming circuit for Resistive Random Access Memory (RRAM) cells. Since there are a lot of different RRAM technologies in research and the process variations of this new memory technology often spread over a wide range of electrical properties, the proposed circuit focuses on versatility in order to be adaptable to different cell properties. The circuit is suitable for both read and programming operations based on voltage
pulses of flexible length and height. The implemented read method is based on evaluating the voltage drop over a measurement resistor and can distinguish up to eight different states, which are coded in binary, thereby realizing a digitization of the analog memory value. The circuit was fabricated in the 130 nm CMOS process line of IHP. The simulations were done using a physics-based, multi-level RRAM model. The measurement results prove the functionality of the read circuit and the programming system and demonstrate that the read system can distinguish up to eight different states with an overall resistance ratio of 7.9.
The use of resistive random-access memory (RRAM) for in-memory computing (IMC) architectures has significantly improved the energy-efficiency of artificial neural networks (ANN) over the past years. Current RRAM-technologies are physically limited to a defined unambiguously distinguishable number of stable states and a maximum resistive value and are compatible with present complementary metal-oxide semiconductor (CMOS)-technologies. In this work, we improved the accuracy of current ANN models by using increased weight resolutions of memristive devices, combining two or more in-series RRAM cells, integrated in the back end of line (BEOL) of the CMOS process. Based on system level simulations, 1T2R devices were fabricated in IHP's 130nm SiGe:BiCMOS technology node, demonstrating an increased number of states. We achieved an increase in weight resolution from 3 bit in ITIR cells to 6.5 bit in our 1T2R cell. The experimental data of 1T2R devices gives indications for the performance and energy-efficiency improvement in ITNR arrays for ANN applications.
Influence of variability on the performance of HfO2 memristor-based convolutional neural networks
(2021)
A study of convolutional neural networks (CNNs) was performed to analyze the influence of quantization and variability in the network synaptic weights. Different CNNs were considered accounting for the number of convolutional layers, size of the filters in the convolutional layer, number of neurons in the final network layers and different sets of quantization levels. The conductance levels of fabricated 1T1R structures based on HfO2 memristors were considered as reference for four or eight level quantization processes at the inference stage of the CNNs, which were previous trained with the MNIST dataset. We also included the variability of the experimental conductance levels that was found to be Gaussian distributed and was correspondingly modeled for the synaptic weight implementation.
We have performed different simulation experiments in relation to hardware neural networks (NN) to analyze the role of the number of synapses for different NN architectures in the network accuracy, considering different datasets. A technology that stands upon 4-kbit 1T1R ReRAM arrays, where resistive switching devices based on HfO2 dielectrics are employed, is taken as a reference. In our study, fully dense (FdNN) and convolutional neural networks (CNN) were considered, where the NN size in terms of the number of synapses and of hidden layer neurons were varied. CNNs work better when the number of synapses to be used is limited. If quantized synaptic weights are included, we observed thatNNaccuracy decreases significantly as the number of synapses is reduced; in this respect, a trade-off between the number of synapses and the NN accuracy has to be achieved. Consequently, the CNN architecture must be carefully designed; in particular, it was noticed that different datasets need specific architectures according to their complexity to achieve good results. It was shown that due to the number of variables that can be changed in the optimization of a NN hardware implementation, a specific solution has to be worked in each case in terms of synaptic weight levels, NN architecture, etc.
In Memory Computing (IMC) is one of the most promising candidates
for data-intensive computing accelerators of machine learning (ML). A key ML algorithm for dimensionality reduction and classification is principal component analysis (PCA), which heavily relies on matrixvector multiplications (MVM) for which classic von Neumann architectures are not optimized. Here, we provide the experimental
demonstration of a new IMCbased PCA algorithm based on power iteration and deflation executed in a 4-kbit array of resistive switching random-access memory (RRAM). The classification accuracy of the Wisconsin Breast Cancer data set reaches 95.43%, close to floatingpoint implementation. Our simulations indicate a 250× improvement in energy efficiency compared to commercial GPUs, thus supporting IMC for energy-efficient ML in modern data-intensive computing.
RRAM technology is a promising candidate for implementing efficient AI accelerators with extensive multiply-accumulate operations. By scaling RRAM devices to the synaptic crossbar array, the computations can be realized in situ, avoiding frequent weights transfer between the processing units and memory. Besides, as the computations are conducted in the analog domain with high flexibility, applying multilevel input voltages to the RRAM devices with multilevel conductance states enhances the computational efficiency further. However, several non-idealities existing in emerging RRAM technology may degrade the reliability of the system. In this paper, we measured and investigated the impact of read disturb on RRAM devices with different input voltages, which incurs conductance drifts and introduces errors. The measured data are deployed to simulate the RRAM based AI inference engines with multilevel states.
This paper presents a read circuit design for resistive memory cells based on current mirrors. The circuit utilizes high-precision current mirrors and reference cells to determine the state of resistive memory using comparators. It offers a high degree in adaptability in terms of both resistance range and number of levels. Special emphasis was put on device protection to prevent accidental programming of the memory during read operations. The realized circuit can resolve eight states with a resolution of up to 1 k Ω, realizing a digitization of the analog memory information. Furthermore, the integration in a complete memory macro is shown. The circuit was realized in a 130 nm-process but can easily be adapted to other processes and resistive memory technologies.
AbstractBlooming and pruning is one of the most important developmental mechanisms of the biological brain in the first years of life, enabling it to adapt its network structure to the demands of the environment. The mechanism is thought to be fundamental for the development of cognitive skills. Inspired by this, Chialvo and Bak proposed in 1999 a learning scheme that learns from mistakes by eliminating from the initial surplus of synaptic connections those that lead to an undesirable outcome. Here, this idea is implemented in a neuromorphic circuit scheme using CMOS integrated HfO2-based memristive devices. The implemented two-layer neural network learns in a self-organized manner without positive reinforcement and exploits the inherent variability of the memristive devices. This approach provides hardware, local, and energy-efficient learning. A combined experimental and simulation-based parameter study is presented to find the relevant system and device parameters leading to a compact and robust memristive neuromorphic circuit that can handle association tasks.
The use of thin layers of amorphous hafnium oxide has been shown to be suitable for the manufacture of Resistive Random-Access memories (RRAM). These memories are of great interest because of their simple structure and non-volatile character. They are particularly appealing as they are good candidates for substituting flash memories. In this work, the performance of the MIM structure that takes part of a 4 kbit memory array based on 1-transistor-1-resistance (1T1R) cells was studied in terms of control of intermediate states and cycle durability. DC and small signal experiments were carried out in order to fully characterize the devices, which presented excellent multilevel capabilities and resistive-switching behavior.
One of the objectives fostered in medical science is the so-called precision medicine, which requires the analysis of a large amount of survival data from patients to deeply understand treatment options. Tools like Machine Learning and Deep Neural Networks are becoming a de-facto standard. Nowadays, computing facilities based on the Von Neumann architecture are devoted to these tasks, yet rapidly hitting a bottleneck in performance and energy efficiency. The In-Memory Computing (IMC) architecture emerged as a revolutionary approach to overcome that issue. In this work, we propose an IMC architecture based on Resistive switching memory (RRAM) crossbar arrays to provide a convenient primitive for matrix–vector multiplication in a single computational step. This opens massive performance improvement in the acceleration of a neural network that is frequently used in survival analysis of biomedical records, namely the DeepSurv. We explored how the synaptic weights mapping strategy and the programming algorithms developed to counter RRAM non-idealities expose a performance/energy trade-off. Finally, we assessed the benefits of the proposed architectures with respect to a GPU-based realization of the same task, evidencing a tenfold improvement in terms of performance and three orders of magnitude with respect to energy efficiency.
End-to-end modeling of variability-aware neural networks based on resistive-switching memory arrays
(2022)
Resistive-switching random access memory (RRAM) is a promising technology that enables advanced applications in the field of in-memory computing (IMC). By operating the memory array in the analogue domain, RRAM-based IMC architectures can dramatically improve the energy efficiency of deep neural networks (DNNs). However, achieving a high inference accuracy is challenged by significant variation of RRAM conductance levels, which can be compensated by (i) advanced programming techniques and (ii) variability-aware training (VAT) algorithms. In both cases, however, detailed knowledge and accurate physics-based statistical models of RRAM are needed to develop programming and VAT methodologies. This work presents an end-to-end approach to the development of highly-accurate IMC circuits with RRAM, encompassing the device modeling, the precise programming algorithm, and the VAT simulations to maximize the DNN classification accuracy in presence of conductance variations.
A crucial step in order to achieve fast and low-energy switching operations in resistive random access memory (RRAM) memories is the reduction of the programming pulse width. In this study, the incremental step pulse with verify algorithm (ISPVA) was implemented by using different pulse widths between 10 μ s and 50 ns and assessed on Al-doped HfO 2 4 kbit RRAM memory arrays. The switching stability was assessed by means of an endurance test of 1k cycles. Both conductive levels and voltages needed for switching showed a remarkable good behavior along 1k reset/set cycles regardless the programming pulse width implemented. Nevertheless, the distributions of voltages as well as the amount of energy required to carry out the switching operations were definitely affected by the value of the pulse width. In addition, the data retention was evaluated after the endurance analysis by annealing the RRAM devices at 150 °C along 100 h. Just an almost negligible increase on the rate of degradation of about 1 μ A at the end of the 100 h of annealing was reported between those samples programmed by employing a pulse width of 10 μ s and those employing 50 ns. Finally, an endurance performance of 200k cycles without any degradation was achieved on 128 RRAM devices by using programming pulses of 100 ns width