Quantum Computer development
Quantum computers harness the power to revolutionize industries like cryptography, database management and medicine. However, many challenges still need to be overcome before the potential of this new technology can be realized. One such challenge is determining how to reliably store information within a quantum computer. This is something me and a team of five had the privilege to explore. Specifically, we investigated ways of improving quantum error correction algorithms with the help of deep reinforcement learning. Below follows a summary of our work, along with references to the paper we produced.
An introduction to quantum computer logicIn order to fully understand the workings and rationale that underpin a quantum computer, it is useful to first consider information handling within traditional computers.
The basic unit of information in computing and digital communications is known as the bit. It represents a logical state with one of two possible values, most commonly depicted as either “1”or”0”. Bits can be physically represented in many ways. In most modern computing devices, they are represented by electrical voltages, pulses of electrical current, or by the electrical states of flip-flop circuits. Moreover, bits can be stored in digital devices or other physical systems that exists in either of two distinct states. These may be the two stable states of a flip-flop, two positions of an electrical switch or even sequences of DNA molecules.
Similarly to regular bits, qubits or quantum bits are the basic units of quantum information. A qubit is a two-state (or two-level) quantum-mechanical system, one of the simplest quantum systems displaying the peculiarity of quantum mechanics. This can for example be realized by the spin of an electron, in which the two levels can be taken as spin up and spin down. Another example is the polarization of a single photon, in which the two states can be taken to be the vertical polarization and the horizontal polarization. In a classical system, a bit would have to be in one state or the other. However, quantum mechanics allows the qubit to be in a coherent superposition of both states simultaneously. This property is of fundamental importance to the utility of a quantum computer.
In order to utilize any computational machine, one needs the ability to store information. As a result of decoherence and other quantum noise, errors in the quantum information stored in a quantum computer will naturally start to accumulate. Quantum error correction is therefore essential if one is to perform fault-tolerant quantum computations that can deal not only with noise, but also with faulty quantum gates, faulty quantum preparation, and faulty measurements.
In classical computing, one normally hedges against these kinds of errors by introducing redundancy in the system. This enables identification of where the issue arose, as well as the possibility of correcting the distorted piece of information. However, the use of quantum computers renders this method unfeasible. This is because an identical copy of a superposed quantum state cannot be created without collapsing the original state: and in doing so loosing the information which was contained.
One workaround utilizes the distinction between logical and physical qubits. A logical qubit is an abstraction which represents the smallest unit of information in a quantum computational system. A physical qubit is a physical implementation of these logical abstractions. The method involves creating a stabilizer formalism that projects a large number of physical qubits into smaller number of entangled logical qubits. These types of constructions are called quantum error correcting codes - and the specific one considered in our project was Kitaev’s surface code on a torus.
Kitaev’s surface code uses syndrome measurements to diagnose which error corrupts an encoded state. A syndrome measurement tells as much as possible about the error that has happened, but nothing at all about the value that is stored in the logical qubit—as measuring the logical qubit would destroy any information stored.
Bit flip errors and syndrome
Our work was focused on so called bit-flip errors, which are specific kinds of errors that can affect the quantum state. On the surface code of the material used for storage, these errors gives rise to a pairs of defects as represented by red lines in (a). The error correction needs to suggest a correction string connecting the two defects. Two examples of such strings can be seen in blue in (b). If the connecting string is properly selected, the correction string and the error string form a trivial loop (c); thus returning the qubit to the original state. If instead the correction string and the error string together make up a non-trivial loop that winds the torus (d), the error syndrome has been eliminated at the expense of chancing the qubit state (corresponding to a logical bit-flip). This would therefore represent a failed attempt at correcting the error. The task of bit flip error correction can therefore be renormalized as finding a correction string such that the the correction string and the error string form a trivial loop. What makes this task challenging is that the surface code outlined in the figure cannot be directly observed without collapsing the quantum state. Instead, one relies on the syndrome, which is uniquely produced by the set of bit-flip errors. The syndrome is illustrated by the circles in the figure, and finding the right combination of bit flips can be thought of as finding a correct path to “pair up” the defects of the syndrome.
There has been numerous attempts at finding an algorithm which is able to reliably perform such an error correction based on the limited information offered by the syndrome. Deterministic versions of these error correction algorithms do a decent job of reverting these states. However, an algorithm whose performance is guaranteed to be optimal is yet to be found. Our research focused on evaluating whether one can construct superior algorithms by means of deep reinforcement learning.
The decoder presented in our paper is a neural network-based agent, optimized using reinforcement learning to observe toric code syndromes and suggesting recovery chains for them step by step. The agent is given the option of moving any defect one plaquette in any direction (left, right, up, or down), corresponding to performing a bit flip on one of the physical qubits enclosing the plaquette containing the defect. It makes use of a deep convolutional neural network to approximate Q values of actions given a syndrome. The utility of these actions can be represented by arrows like in the figure. In the end, we found that our machine learning solution was competitive against the traditional algorithms which are currently advocated. Due to these promising findings, our entire team was offered funding by Chalmers University of Technology to pursue further research. At this time I had already accepted an internship position in the US, but a number of my colleagues further advanced our research. The results of their efforts was published in this paper.