5 ms·

Quantinuum is already tackling NLP question-answering tasks, it runs on quantum hardware. The quantum categorial crowd is adamant they'll surpass ChatGPT (the b

Quantinuum is already tackling NLP question-answering tasks, it runs on quantum hardware. The quantum categorial crowd is adamant they'll surpass ChatGPT (the big promise iirc is that it will scale linearly with context length, GPT scales quadratically). For instance see this tweet by Quantinuum head of research and this burgeoning field's main rock: https://twitter.com/coecke/status/1655695990739927040?s=20 https://twitter.com/coecke/status/1655695990739927040?s=20 Coecke went from supervising dozen of thesis at Oxford Quantum (logic) Group to preparing summer camps for high school pupils this year. It's also taking off socially/academically, and observing the field evolving we might have a quantum equivalent of ChatGPT before or at the same time we get implementations of Shor's algorithm (source: my own intuition). See this for instance: https://arxiv.org/abs/2210.11523 https://arxiv.org/abs/2210.11523

cgranade 3y ago
The problem you'll run into for any application of quantum computing to large language models is that quantum computers just aren't very good at big data applications. There's two reasons for that: - Current devices, as well as devices likely to be built in the near- to medium-term are quite limited in the number of qubits that they implement. The current record for the most fault-tolerant qubits in a single device is 1. That's a hell of a lot better than where the field was at a couple years ago, but it's far from the huge amount of data that needs to be processed for LLM training and evaluation. - Even if you have enough qubits to store training data, looking them up on a quantum device is still challenging due to what's sometimes called the qRAM problem. It's not trivial to make a quantum oracle that returns the data stored at a given index, and it's still an area of ongoing research to figure out how to do that. That's part of why you see quantum algorithms being developed less for big data tasks and more for big compute tasks like chemistry. There, the program might be very large, but size of the input that has to be stored within the quantum devices and the size of the output you measure back out are both quite small, even down to a single floating-point number in some cases. (source: I've worked in quantum computing for about twenty years now.)
- pmoriarty 3y ago
  "The current record for the most fault-tolerant qubits in a single device is 1." Can you comment on Google's[1] and IBM's[2] announcements of 70 and 433 qubit quantum computers? Is this just marketing hype? Are the qubits not fault tolerant? Is fault tolerance really necessary to get useful results? [1] - https://www.telegraph.co.uk/business/2023/07/02/google-quantum-computer-breakthrough-instant-calculations/ https://www.telegraph.co.uk/business/2023/07/02/google-quant... [2] - https://www.technologyreview.com/2023/05/25/1073606/ibm-wants-to-build-a-100000-qubit-quantum-computer/ https://www.technologyreview.com/2023/05/25/1073606/ibm-want...
  - cgranade 3y ago
    That's why I emphasized one _logical_ qubit. I'll definitely argue that fault tolerance is necessary to achieve useful results, as you say, but there is some argument in the research community on that. Even setting that discussion aside, there's absolutely no way to run something like LLM training dirctly on physical qubits (unless there was an improvement in error rates that's probably on the order of 10^15 to 10^18), even if you had both enough to do so and had a good qRAM implementation.
    - StackOverlord 3y ago
      What do you think of > In some of its applications, the original > Zeng-Coecke algorithm relies on the existence of a quantum random access memory (QRAM) [22], > which is not yet known to be efficiently implementable in the absence of fault tolerant scalable quantum > computers [1, 7]. Here we take a different approach, using the classical ansatz parameters to encode the ¨ > distributional embedding and avoiding the need for QRAM entirely. The cost function for the parameter > optimisation is informed by a corpus, already parsed and POS-tagged by classical means. Source: Quantum Natural Language Processing on Near-Term Quantum Computers https://arxiv.org/abs/2005.04147 https://arxiv.org/abs/2005.04147 Following my intuition, i.e. as an outsider that has been watching the progress of quantum NLP since 2012, I see the current academic situation in quantum computing as in the process of merging two branches, one being the traditional quantum computing field with concerns stemming and application thought in mathematics, computing theory, physics(and upwards chemistry->biochemistry->biology), the other branch being a fork carried out by Coecke (quantum logic), Abramsky (computer science) and Sadrzadeh (epistemic logic) who saw in categorial formalisms of quantum logic a way to mix compositional (syntax, logical rules) and distributional (statistics, "bag-of-neighbor-words") representations of meaning. In this regard they bring new methods but also new applications of quantum computing, with a focus on NLP, as language given this "natural tensor structure [20, 35, 23] [...] can be considered quantum-native [48, 2, 8]." (same paper).

cgranade 3y ago

The problem you'll run into for any application of quantum computing to large language models is that quantum computers just aren't very good at big data applications. There's two reasons for that: - Current devices, as well as devices likely to be built in the near- to medium-term are quite limited in the number of qubits that they implement. The current record for the most fault-tolerant qubits in a single device is 1. That's a hell of a lot better than where the field was at a couple years ago, but it's far from the huge amount of data that needs to be processed for LLM training and evaluation. - Even if you have enough qubits to store training data, looking them up on a quantum device is still challenging due to what's sometimes called the qRAM problem. It's not trivial to make a quantum oracle that returns the data stored at a given index, and it's still an area of ongoing research to figure out how to do that. That's part of why you see quantum algorithms being developed less for big data tasks and more for big compute tasks like chemistry. There, the program might be very large, but size of the input that has to be stored within the quantum devices and the size of the output you measure back out are both quite small, even down to a single floating-point number in some cases. (source: I've worked in quantum computing for about twenty years now.)

pmoriarty 3y ago
"The current record for the most fault-tolerant qubits in a single device is 1." Can you comment on Google's[1] and IBM's[2] announcements of 70 and 433 qubit quantum computers? Is this just marketing hype? Are the qubits not fault tolerant? Is fault tolerance really necessary to get useful results? [1] - https://www.telegraph.co.uk/business/2023/07/02/google-quantum-computer-breakthrough-instant-calculations/ https://www.telegraph.co.uk/business/2023/07/02/google-quant... [2] - https://www.technologyreview.com/2023/05/25/1073606/ibm-wants-to-build-a-100000-qubit-quantum-computer/ https://www.technologyreview.com/2023/05/25/1073606/ibm-want...
- cgranade 3y ago
  That's why I emphasized one _logical_ qubit. I'll definitely argue that fault tolerance is necessary to achieve useful results, as you say, but there is some argument in the research community on that. Even setting that discussion aside, there's absolutely no way to run something like LLM training dirctly on physical qubits (unless there was an improvement in error rates that's probably on the order of 10^15 to 10^18), even if you had both enough to do so and had a good qRAM implementation.
  - StackOverlord 3y ago
    What do you think of > In some of its applications, the original > Zeng-Coecke algorithm relies on the existence of a quantum random access memory (QRAM) [22], > which is not yet known to be efficiently implementable in the absence of fault tolerant scalable quantum > computers [1, 7]. Here we take a different approach, using the classical ansatz parameters to encode the ¨ > distributional embedding and avoiding the need for QRAM entirely. The cost function for the parameter > optimisation is informed by a corpus, already parsed and POS-tagged by classical means. Source: Quantum Natural Language Processing on Near-Term Quantum Computers https://arxiv.org/abs/2005.04147 https://arxiv.org/abs/2005.04147 Following my intuition, i.e. as an outsider that has been watching the progress of quantum NLP since 2012, I see the current academic situation in quantum computing as in the process of merging two branches, one being the traditional quantum computing field with concerns stemming and application thought in mathematics, computing theory, physics(and upwards chemistry->biochemistry->biology), the other branch being a fork carried out by Coecke (quantum logic), Abramsky (computer science) and Sadrzadeh (epistemic logic) who saw in categorial formalisms of quantum logic a way to mix compositional (syntax, logical rules) and distributional (statistics, "bag-of-neighbor-words") representations of meaning. In this regard they bring new methods but also new applications of quantum computing, with a focus on NLP, as language given this "natural tensor structure [20, 35, 23] [...] can be considered quantum-native [48, 2, 8]." (same paper).