7 ms·

Parsing Chemistry

54 points by kencausey 11mo ago

whitten 11mo ago
Does the SMILE (or Simplified Molecular Input Line Entry System) code have an EBNF definition ? https://en.wikipedia.org/wiki/Simplified_Molecular_Input_Line_Entry_System https://en.wikipedia.org/wiki/Simplified_Molecular_Input_Lin... Claims there is a context free grammar.
- fred_tandemai 11mo ago
  [dead]
- dalke 11mo ago
  That's "SMILES". Yes. Here is the yacc grammar for the SMILES parser in the RDKit. https://github.com/rdkit/rdkit/blob/master/Code/GraphMol/SmilesParse/smiles.yy https://github.com/rdkit/rdkit/blob/master/Code/GraphMol/Smi... There's also one from OpenSMILES at http://opensmiles.org/opensmiles.html#_grammar http://opensmiles.org/opensmiles.html#_grammar . It has a shift/reduce error (as I recall) that I was not competent enough to fix. I prefer to parser almost completely in the lexer, with a small amount of lexer state to handle balanced parens, bracket atoms, and matching ring closures. See https://hg.sr.ht/~dalke/opensmiles-ragel https://hg.sr.ht/~dalke/opensmiles-ragel and more specifically https://hg.sr.ht/~dalke/opensmiles-ragel/browse/opensmiles.rl?rev=tip https://hg.sr.ht/~dalke/opensmiles-ragel/browse/opensmiles.r... .
  - dalke 10mo ago
    Oh, I should have pointed out my Python lexer-driven parser at https://hg.sr.ht/~dalke/smiview/browse/smiview.py https://hg.sr.ht/~dalke/smiview/browse/smiview.py The lexer: https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L337 https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L3... The lexer state transitions: https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L364 https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L3...
- dekhn 11mo ago
  I wrote a very simple SMILES parser using pyparsing https://github.com/dakoner/smilesparser/tree/master https://github.com/dakoner/smilesparser/tree/master I wouldn't say it's intended for production work, but it has been useful in situations where I didn't want to pull in rdkit.
  - dalke 10mo ago
    I see you include the dot disconnect "." as part of the Bond definition. You also define Chain as: Chain <<= pp.Group(pp.Optional(Bond) + pp.Or([Atom, RingClosure])) I believe this means your grammar allows the invalid SMILES C=.N
the__alchemist 11mo ago
Note: There are two standardized formats for this called SMILES and SELFIES. SMILES is much better supported, but SELFIES is more robust. I'm integrating them into some bio and chem software I'm working on. You can do things like look up, using PubChem's API, similar molecules etc to a SMILES string. I believe most molecule editors can load and save SMILES.
- dachrillz 11mo ago
  What about inchi? Isn’t that a common way of describing molecules as well?
  - the__alchemist 11mo ago
    Good point!
  - fred_tandemai 11mo ago
    InChI isn't really meant to be used as a format to store 2D molecules say for rendering but rather serves as a unique descriptive chemical identifier. InChI has many flavors but the Standard InChI yields one unique identifier for multiple forms (tautomers) of the same molecule.
- jugoetz 11mo ago
  SMILES and SELFIES are molecular graph representations and aren't meant to solve the "parse this sum formula" problem. SELFIES are for genAI. If you ask a VAE to generate SMILES, it will spit out some strings that are invalid - can't happen with SELFIES, that is the one application where they are robust.
  - dekhn 11mo ago
    It's still being argued if you really need SELFIES, or if SMILES autoencoders can be trained to only generate valid molecules, or if generating invalid molecules is useful (I'm in camp SELFIES, but I also want better ways to represent and learn on graphical chemical structures, ratehr than serialized strings).
logifail 11mo ago
Does this do structural formulae too? Was thinking of InChI[0] but on Googling SMILES and SELFIES I found this[1] talk, this[2] paper and my goodness I've been down a few rabbit holes since... [0] https://en.wikipedia.org/wiki/International_Chemical_Identifier https://en.wikipedia.org/wiki/International_Chemical_Identif... [1] https://www.inchi-trust.org/wp/wp-content/uploads/2019/12/18.-IUPAC-SMILES-update-and-breakout_InChI2019-20190821.pdf https://www.inchi-trust.org/wp/wp-content/uploads/2019/12/18... [2] https://pubs.rsc.org/en/content/articlehtml/2022/dd/d1dd00013f https://pubs.rsc.org/en/content/articlehtml/2022/dd/d1dd0001...
- jugoetz 11mo ago
  No, in Python you can use rdkit (https://github.com/rdkit/rdkit https://github.com/rdkit/rdkit) for that
toast_x 11mo ago
this is insanely cool
- Jaxan 11mo ago
  … It is just a parser? Sure the parser is written very succinctly and that’s neat. But parser generators for other languages can do it similarly.
brilee 11mo ago
Does this handle, e.g., water of hydration CaSO4 . 2H2O? states of matter H2O(g)? does it preserve subunit information, as in (C6H5)CH2COOH? Writing a parser for basic formulae is such a tiny tiny part of the actual problem... deciding the scope of what you want to handle and how is the real problem
mwt 11mo ago
This code is jibberish to me, but it appears the target is just parsing how many atoms are in a molecule string of some representation. That's cool, but to do just about anything useful in chemistry we need the bond graph (and often more - bond orders stereochemistry, plus much more for biopolymers).
- the__alchemist 11mo ago
  That was my initial reaction too, but I suspect this is has utility in applications other than what you and I are looking for. From context, I gather this may be for thermodynamic arithmetic, or reaction product arithmetic.
  - mwt 11mo ago
    I'd be really interested to know of anybody making money with those topics (and doesn't already have their own domain-specific practice for the problem)
    - fred_tandemai 11mo ago
      Cheminformatics is such an example. Heavily used in computational drug discovery.
      - chermi 11mo ago
        Computational biology/cheminformatics has probably been on the most frustrating investments pharma companies have made in the past 20 years. There's been waves of optimism with many hires, then a slump after reality doesn't match optimistic expections, and so on. This time it may actually be different, and I myself am in that camp. I'm particularly excited by the discoveries in sampling methods that aren't just molecular dynamics. And the cellular foundation models for pre-screening drug interactions - they aren't quite there yet, but give it time.
      - mwt 10mo ago
        The cheminformatics I do (mostly drug discovery/biophysics) definitely requires bonds!
fred_tandemai 11mo ago
[dead]

54 points by kencausey 11mo ago

Does the SMILE (or Simplified Molecular Input Line Entry System) code have an EBNF definition ? https://en.wikipedia.org/wiki/Simplified_Molecular_Input_Line_Entry_System https://en.wikipedia.org/wiki/Simplified_Molecular_Input_Lin... Claims there is a context free grammar.

fred_tandemai 11mo ago
[dead]
dalke 11mo ago
That's "SMILES". Yes. Here is the yacc grammar for the SMILES parser in the RDKit. https://github.com/rdkit/rdkit/blob/master/Code/GraphMol/SmilesParse/smiles.yy https://github.com/rdkit/rdkit/blob/master/Code/GraphMol/Smi... There's also one from OpenSMILES at http://opensmiles.org/opensmiles.html#_grammar http://opensmiles.org/opensmiles.html#_grammar . It has a shift/reduce error (as I recall) that I was not competent enough to fix. I prefer to parser almost completely in the lexer, with a small amount of lexer state to handle balanced parens, bracket atoms, and matching ring closures. See https://hg.sr.ht/~dalke/opensmiles-ragel https://hg.sr.ht/~dalke/opensmiles-ragel and more specifically https://hg.sr.ht/~dalke/opensmiles-ragel/browse/opensmiles.rl?rev=tip https://hg.sr.ht/~dalke/opensmiles-ragel/browse/opensmiles.r... .
- dalke 10mo ago
  Oh, I should have pointed out my Python lexer-driven parser at https://hg.sr.ht/~dalke/smiview/browse/smiview.py https://hg.sr.ht/~dalke/smiview/browse/smiview.py The lexer: https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L337 https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L3... The lexer state transitions: https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L364 https://hg.sr.ht/~dalke/smiview/browse/smiview.py?rev=tip#L3...
dekhn 11mo ago
I wrote a very simple SMILES parser using pyparsing https://github.com/dakoner/smilesparser/tree/master https://github.com/dakoner/smilesparser/tree/master I wouldn't say it's intended for production work, but it has been useful in situations where I didn't want to pull in rdkit.
- dalke 10mo ago
  I see you include the dot disconnect "." as part of the Bond definition. You also define Chain as: Chain <<= pp.Group(pp.Optional(Bond) + pp.Or([Atom, RingClosure])) I believe this means your grammar allows the invalid SMILES C=.N

the__alchemist 11mo ago

Note: There are two standardized formats for this called SMILES and SELFIES. SMILES is much better supported, but SELFIES is more robust. I'm integrating them into some bio and chem software I'm working on. You can do things like look up, using PubChem's API, similar molecules etc to a SMILES string. I believe most molecule editors can load and save SMILES.

dachrillz 11mo ago
What about inchi? Isn’t that a common way of describing molecules as well?
- the__alchemist 11mo ago
  Good point!
- fred_tandemai 11mo ago
  InChI isn't really meant to be used as a format to store 2D molecules say for rendering but rather serves as a unique descriptive chemical identifier. InChI has many flavors but the Standard InChI yields one unique identifier for multiple forms (tautomers) of the same molecule.
jugoetz 11mo ago
SMILES and SELFIES are molecular graph representations and aren't meant to solve the "parse this sum formula" problem. SELFIES are for genAI. If you ask a VAE to generate SMILES, it will spit out some strings that are invalid - can't happen with SELFIES, that is the one application where they are robust.
- dekhn 11mo ago
  It's still being argued if you really need SELFIES, or if SMILES autoencoders can be trained to only generate valid molecules, or if generating invalid molecules is useful (I'm in camp SELFIES, but I also want better ways to represent and learn on graphical chemical structures, ratehr than serialized strings).

logifail 11mo ago

Does this do structural formulae too? Was thinking of InChI[0] but on Googling SMILES and SELFIES I found this[1] talk, this[2] paper and my goodness I've been down a few rabbit holes since... [0] https://en.wikipedia.org/wiki/International_Chemical_Identifier https://en.wikipedia.org/wiki/International_Chemical_Identif... [1] https://www.inchi-trust.org/wp/wp-content/uploads/2019/12/18.-IUPAC-SMILES-update-and-breakout_InChI2019-20190821.pdf https://www.inchi-trust.org/wp/wp-content/uploads/2019/12/18... [2] https://pubs.rsc.org/en/content/articlehtml/2022/dd/d1dd00013f https://pubs.rsc.org/en/content/articlehtml/2022/dd/d1dd0001...

jugoetz 11mo ago
No, in Python you can use rdkit (https://github.com/rdkit/rdkit https://github.com/rdkit/rdkit) for that

toast_x 11mo ago

this is insanely cool

Jaxan 11mo ago
… It is just a parser? Sure the parser is written very succinctly and that’s neat. But parser generators for other languages can do it similarly.

brilee 11mo ago

Does this handle, e.g., water of hydration CaSO4 . 2H2O? states of matter H2O(g)? does it preserve subunit information, as in (C6H5)CH2COOH? Writing a parser for basic formulae is such a tiny tiny part of the actual problem... deciding the scope of what you want to handle and how is the real problem

mwt 11mo ago

This code is jibberish to me, but it appears the target is just parsing how many atoms are in a molecule string of some representation. That's cool, but to do just about anything useful in chemistry we need the bond graph (and often more - bond orders stereochemistry, plus much more for biopolymers).

the__alchemist 11mo ago
That was my initial reaction too, but I suspect this is has utility in applications other than what you and I are looking for. From context, I gather this may be for thermodynamic arithmetic, or reaction product arithmetic.
- mwt 11mo ago
  I'd be really interested to know of anybody making money with those topics (and doesn't already have their own domain-specific practice for the problem)
  - fred_tandemai 11mo ago
    Cheminformatics is such an example. Heavily used in computational drug discovery.
    - chermi 11mo ago
      Computational biology/cheminformatics has probably been on the most frustrating investments pharma companies have made in the past 20 years. There's been waves of optimism with many hires, then a slump after reality doesn't match optimistic expections, and so on. This time it may actually be different, and I myself am in that camp. I'm particularly excited by the discoveries in sampling methods that aren't just molecular dynamics. And the cellular foundation models for pre-screening drug interactions - they aren't quite there yet, but give it time.
    - mwt 10mo ago
      The cheminformatics I do (mostly drug discovery/biophysics) definitely requires bonds!

fred_tandemai 11mo ago

[dead]