8 ms·

TeX justification really needs to be ported to browsers. Are those hyphenation algorithms really that hard to implement?

fortes 16y ago
No, the challenge here is typically with having a high-quality hyphenation dictionary.
- Someone 16y ago
  A dictionary is insufficient for languages that use compound words such as German and Dutch. You also need some algorithm for taking apart such compound words.
  - fortes 16y ago
    Definitely -- but the algorithms are well-known here, right?
    - eru 16y ago
      Just rip it out of LaTeX.
    - Someone 16y ago
      Read a Dutch newspaper for a while, and you will not make that claim anymore. I do not know how well LateX works here, but I would guess that it only inserts 'safe' hyphens, using a word list. Without a word list, there simply is no way to know whether e.g. 'verstoren' is 'ver-storen' (a verb meaning 'to disturb') or 'vers-toren (could be a tower to store fresh stuff in or to sing from, but AFAIK is not an existing word) 'verst-oren' (AFAIK a meaningless word) without understanding the semantics of a text.
- ugh 16y ago
  I don’t think implementation is the problem. Standards are. This really should be done by the browser but it’s nowhere to be seen in the standards process. It wouldn’t be strange to have a CSS property that controls hyphenation, ‘text-overlow:ellipsis’ (which adds a ellipsis before the text overflows [+]) also changes the text at that level. [+] http://dev.w3.org/csswg/css3-text/#text-overflow http://dev.w3.org/csswg/css3-text/#text-overflow
  - fortes 16y ago
    Lack of standards haven't stopped browsers from implementing features in the past. True hyphenation requires: 0. A property to enable or disable hyphenation 1. Hyphenation algorithm (mostly a solved problem) 2. Hyphenation dictionaries: One per language, and need to be high quality -- OpenOffice has the best open source one, but generally these are not very common I'm not convinced that 0 or 1 are the blockers here. There could be another issue: speed. Doing proper hyphenation and justification slows layout.
    - ugh 16y ago
      Look what I found: http://www.w3.org/TR/css3-text/#hyphenate http://www.w3.org/TR/css3-text/#hyphenate (That’s the current CSS 3 Working Draft.) It seems I was a bit too brisk with my assertion. Hyphenation is coming (at some point in the indefinite future), only the implementation details seem to be not yet agreed upon – not even in a rudimentary way – which is probably why no browser dares to implement it. The Working Draft links to another Draft where a possible implementation is detailed [1]. Looking around the public mailing list it seems that some have been barking up that tree for a long time. Standards really don’t seem to be the holdup. At least some people have campaigned for the inclusion of hyphenation in the past. I still think that implementation isn’t the problem, either. This seems to be one of those eccentric typographic details that is ignored and can safely be ignored because the web doesn’t depend on it (kind of like kerning and ligatures which are now only slowly finding their way into browsers). It’s also not exactly flashy like rounded borders, drop shadows or gradients. That might be the reason why browser vendors are dragging their feet. They could implement it but it’s not super-trivial so they rather wait and see. [1] http://www.w3.org/TR/2007/WD-css3-gcpm-20070205/#hyphenation http://www.w3.org/TR/2007/WD-css3-gcpm-20070205/#hyphenation
sz 16y ago
> TeX really needs to be ported to browsers FTFY Seriously though, browsers need better typography in general. http://nitens.org/taraborelli/latex http://nitens.org/taraborelli/latex
celoyd 16y ago
Are those hyphenation algorithms really that hard to implement? Yes: see (for example) http://en.wikipedia.org/wiki/Hz-program http://en.wikipedia.org/wiki/Hz-program . A really good H&J algorithm is doing a significant amount of work. It has to think about things like successive hyphens (which get distracting quickly), rivers (the unpleasant vertical lines of whitespace that appear especially in over-spaced text), and so on. This involves lots of backtracking and image processing, and it fires every time the layout changes. It’s all possible to do in a browser someday, but it’s not trivial.
- deleted 16y ago
  [deleted]
- DennisP 16y ago
  TeX does a pretty good job with that stuff. Knuth gives a clear and concise explanation of the algorithm in a chapter of Digital Typography. I don't think you need to get too fancy with kerning and scaling, the way Hz does. It's a web page, not a printed book. Put the hyphens where they need to be to optimize each paragraph, and you'll probably get decent results. Knuth's algorithm will do that effectively and efficiently.

fortes 16y ago

No, the challenge here is typically with having a high-quality hyphenation dictionary.

Someone 16y ago
A dictionary is insufficient for languages that use compound words such as German and Dutch. You also need some algorithm for taking apart such compound words.
- fortes 16y ago
  Definitely -- but the algorithms are well-known here, right?
  - eru 16y ago
    Just rip it out of LaTeX.
  - Someone 16y ago
    Read a Dutch newspaper for a while, and you will not make that claim anymore. I do not know how well LateX works here, but I would guess that it only inserts 'safe' hyphens, using a word list. Without a word list, there simply is no way to know whether e.g. 'verstoren' is 'ver-storen' (a verb meaning 'to disturb') or 'vers-toren (could be a tower to store fresh stuff in or to sing from, but AFAIK is not an existing word) 'verst-oren' (AFAIK a meaningless word) without understanding the semantics of a text.
ugh 16y ago
I don’t think implementation is the problem. Standards are. This really should be done by the browser but it’s nowhere to be seen in the standards process. It wouldn’t be strange to have a CSS property that controls hyphenation, ‘text-overlow:ellipsis’ (which adds a ellipsis before the text overflows [+]) also changes the text at that level. [+] http://dev.w3.org/csswg/css3-text/#text-overflow http://dev.w3.org/csswg/css3-text/#text-overflow
- fortes 16y ago
  Lack of standards haven't stopped browsers from implementing features in the past. True hyphenation requires: 0. A property to enable or disable hyphenation 1. Hyphenation algorithm (mostly a solved problem) 2. Hyphenation dictionaries: One per language, and need to be high quality -- OpenOffice has the best open source one, but generally these are not very common I'm not convinced that 0 or 1 are the blockers here. There could be another issue: speed. Doing proper hyphenation and justification slows layout.
  - ugh 16y ago
    Look what I found: http://www.w3.org/TR/css3-text/#hyphenate http://www.w3.org/TR/css3-text/#hyphenate (That’s the current CSS 3 Working Draft.) It seems I was a bit too brisk with my assertion. Hyphenation is coming (at some point in the indefinite future), only the implementation details seem to be not yet agreed upon – not even in a rudimentary way – which is probably why no browser dares to implement it. The Working Draft links to another Draft where a possible implementation is detailed [1]. Looking around the public mailing list it seems that some have been barking up that tree for a long time. Standards really don’t seem to be the holdup. At least some people have campaigned for the inclusion of hyphenation in the past. I still think that implementation isn’t the problem, either. This seems to be one of those eccentric typographic details that is ignored and can safely be ignored because the web doesn’t depend on it (kind of like kerning and ligatures which are now only slowly finding their way into browsers). It’s also not exactly flashy like rounded borders, drop shadows or gradients. That might be the reason why browser vendors are dragging their feet. They could implement it but it’s not super-trivial so they rather wait and see. [1] http://www.w3.org/TR/2007/WD-css3-gcpm-20070205/#hyphenation http://www.w3.org/TR/2007/WD-css3-gcpm-20070205/#hyphenation

sz 16y ago

> TeX really needs to be ported to browsers FTFY Seriously though, browsers need better typography in general. http://nitens.org/taraborelli/latex http://nitens.org/taraborelli/latex

celoyd 16y ago

Are those hyphenation algorithms really that hard to implement? Yes: see (for example) http://en.wikipedia.org/wiki/Hz-program http://en.wikipedia.org/wiki/Hz-program . A really good H&J algorithm is doing a significant amount of work. It has to think about things like successive hyphens (which get distracting quickly), rivers (the unpleasant vertical lines of whitespace that appear especially in over-spaced text), and so on. This involves lots of backtracking and image processing, and it fires every time the layout changes. It’s all possible to do in a browser someday, but it’s not trivial.

deleted 16y ago
[deleted]
DennisP 16y ago
TeX does a pretty good job with that stuff. Knuth gives a clear and concise explanation of the algorithm in a chapter of Digital Typography. I don't think you need to get too fancy with kerning and scaling, the way Hz does. It's a web page, not a printed book. Put the hyphens where they need to be to optimize each paragraph, and you'll probably get decent results. Knuth's algorithm will do that effectively and efficiently.