8 ms·

Is OOXML Artifically Complex?

162 points by firexcy 1y ago

3cats-in-a-coat 1y ago
Microsoft just took what they had and directly translated it to XML. It's not intentionally messy, it's just a big corporation with old product acting like it.
- gitonup 1y ago
  This is the God's honest. I worked on the MS Word core team for a little over three years from 2010-2014, and de-facto owned a significant part of implementing ODF / OOXML Strict support. The binary format was a liability for Microsoft to begin with, because of decades of cruft lining up with actual memory alignment. During my tenure there I ran into code my GM had written as an intern and was still intact -- he had 20+ years of tenure (mostly on Word) when I joined the team. The translation of the file format to XML involved a significant amount of performance degradation if you weren't careful. Hundreds of millions of people use the app monthly, and MS still tries to maintain backwards compatibility. Given that open APIs were a relatively late development for the app, I really don't think in the current reality of what's expected by boards of directors for the companies they oversee that _anyone_ would take years to: a) define a spec that maintained that backwards compatibility b) reach whatever nebulous simplicity metric today's HN article wants c) not get whoever greenlit the project fired for taking that many engineering hours for a and b
piker 1y ago
Dead on. Microsoft is just dominant and exporting its 40 year old legacy codebase as a spec. LibreOffice team is frustrated that the for-profit model is beating the OSS model and crying foul over mostly necessary complexity. If LibreOffice started from scratch they’d probably appreciate how much Microsoft serializes because a sufficiently complicated document saved to .docx basically provides a reference implementation. We do need for-profit alternatives to Word, and I’m working on one in legal. [edit: I hope to put some real thoughts on this down soon, but most of the wonkiness emanates from evolving functionality and varying trends in best practices over the decades. I’ve implemented a fair bit of the spec here: https://tritium.legal https://tritium.legal, but most of the hard part is providing for bidi language support, fonts, real-time editing and re-rendering, UI and annotations like spellchecking and grammar, not conforming to the markup spec. Spec conformance is just polish and testing. A performant modern word processor of any spec, however, is a technological achievement on the order of a web browser.]
- trelane 1y ago
  LibreOffice has versions that you pay for, with support. The most prominent is Collabora, which is a (if not the) biggest contributor to LibreOffice.
- croes 1y ago
  Where does the article say it’s a necessary complexity? > Thus, the primary goal for this new format wasn’t to be elegant, universal, or easy to implement; it was to placate regulators while preserving Microsoft’s technological and commercial advantages. That sounds quite anti-competitive to me
- Gigachad 1y ago
  I feel like Libreoffice became largely irrelevant the day Google Docs came out. People put up with LO wonkyness because it was free and office was expensive. Google completely flipped the game and then cloud collaboration became everything.
  - toast0 1y ago
    I mean, multiplayer features are useful, but Google Docs is wonkier than LO. At least when LO loads a document, it's fully loaded.
- taftster 1y ago
  > We do need for-profit alternatives to Word, and I’m working on one in legal. Wow, big undertaking! What we really need, though, is a for-profit alternative to Excel, that's not Google. I think Excel is more of the Killer App than Word has ever been.
  - qcnguy 1y ago
    That's Apple Numbers.
    - mschuster91 1y ago
      ... which comes tied to macOS and with it Apple hardware. Neither play well in a shop that uses x86 Windows-only software, and Apple's switch to ARM hasn't made that easier.
etothepii 1y ago
I spent a lot of time last year replicating every valid Excel number format. I've really struggled to find good documentation on the excel format when you really get into the weeds. The use of namespaces is also incredibly annoying in so far as I can tell in every xml library I can find they really aren't well supported for that "human" readable component. When you crack open the file it feels like you are going to be able to find everything you need with an xpath like //w:t but none of the xml parsers I've found cope well with the namespaces.
- rhdunn 1y ago
  What language? In Python, the `find`, `findall`, etc. methods take a namespace dictionary. E.g. result = doc.findall("//w:t", namespaces={"w": "..."}) In C# you can do: var navigator = doc.Root!.CreateNavigator(); nsManager = new XmlNamespaceManager(navigator.NameTable); nsManager.AddNamespace("w", "..."); var results = doc.Root?.XPathSelectElements("//w:t", nsManager); In Java you need to enable a namespace-aware flag in the settings to get namespaces to work. I can't recall off-hand how to do that.
  - etothepii 1y ago
    Yes I'm talking Python. The namespaces are long, dare I say ugly, urls. Even though the xml file itself uses `<w:t` you can't unless you provide a dictionary as an argument that contains `{"w":"https://...."}` which means that you need to do a bunch of reading and understanding before you can start playing with the files.
Joker_vD 1y ago
sigh Just because it was not deliberately engineered to be prohitibively expensive to support does not mean that it can not be used to deliberately obstruct interoperability. It's really not that difficult a concept: if you want others to suffer, you can take a sad artifact of well-meant historical accidents, and say "welp, now it's a standard, you gotta support it!" There is nothing contradictory or conspirational.
- piker 1y ago
  I think we take issue with requiring the leap to Microsoft “deliberately” obstructing interoperability. Microsoft just isn’t incentivized to make it simple to implement, but it’s probably less complicated than the various web standards.
  - Joker_vD 1y ago
    An engineering team in Microsoft decides to switch from binary format to XML to save effort in the long run; even though it'll take some effort now, they have the competency, and can afford it. They are absolutely correct! But then their manager needs to sell this project to the higher-ups, who have read BillG's memo about how "One thing we have got to change in our strategy – allowing Office documents to be rendered very well by other people's browsers is one of the most destructive things we could do to the company. We have to stop putting any effort into this and make sure that Office documents very well depend on proprietary IE capabilities. Anything else is suicide for our platform. This is a case where Office has to avoid doing something to destroy Windows." and took it to heart. So what does he do? Why, he spins a tale that since it's XML, they'll be able to standardize it, and everyone else will still be forced to interoperate with MS Office anyhow, because it will be the de-facto reference implementation (by the virtue of being there first, and widely deployed), and the spec is going to be an absolute PITA to implement decently — and that manager too will be absolutely correct!
    - piker 1y ago
      It’s not actually that bad.
cyberax 1y ago
The answer: no. OOXML is an extremely detailed spec that lists minute details of the Office documents, with uncountable features. While it could have used some "standard" features, there weren't that many usable standards when OOXML was being developed. In comparison, OASIS OpenDocument spec is horribly ambiguous and has all the same issues (like units not being used consistently). It got better over the years, but it's still not at all great. And its size is now comparable to OOXML, when all the referenced specs are incorporated.
- rhdunn 1y ago
  There are places where it says the equivalent of "Works the same as Word 95" [3], but does not specify in the specification what that means. It's essentially a serialization of the binary format to XML. ODF 1.4 is around 1,100 pages across all 4 parts whereas OOXML is over 6,000. [1] https://stephesblog.blogs.com/my_weblog/2007/08/microsofts-fail.html https://stephesblog.blogs.com/my_weblog/2007/08/microsofts-f... [2] https://ooxmlisdefectivebydesign.blogspot.com/2007/08/microsoft-office-xml-formats-defective.html https://ooxmlisdefectivebydesign.blogspot.com/2007/08/micros... [3] https://www.robweir.com/blog/2007/01/how-to-hire-guillaume-portes.html https://www.robweir.com/blog/2007/01/how-to-hire-guillaume-p...
  - cyberax 1y ago
    > There are places where it says the equivalent of "Works the same as Word 95" [3], but does not specify in the specification what that means. Yeah, sure, whatever. You'll never see these kinds of documents in real life. And the specified quirks were minor. If you don't implement them, you'll get subtle formatting issues in documents imported directly from Word97. MS could have just put them into a "vendor-specific" extension and not documented them at all. > ODF 1.4 is around 1,100 pages across all 4 parts whereas OOXML is over 6,000. LOL, no. SVG spec alone is 800 pages. ODF formula spec is 200 pages alone, and is still underspecified.
  - xeeeeeeeeeeenu 1y ago
    They improved this in later revisions of the standard. The behaviour of autoSpaceLikeWord95 is now actually described and there's an example. You can see it for yourself here (in Part 4): https://ecma-international.org/publications-and-standards/standards/ecma-376/ https://ecma-international.org/publications-and-standards/st...
theanonymousone 1y ago
Duplicate: https://news.ycombinator.com/item?id=45147639 https://news.ycombinator.com/item?id=45147639
- fsflover 1y ago
  It's not a duplicate if there's no discussion.
  - theanonymousone 1y ago
    How come the same link was accepted twice,in the first place?
    - fsflover 1y ago
      Resubmissions are acceptable, if the discussion hasn't started.
fsflover 1y ago
See also: https://en.wikipedia.org/wiki/Standardization_of_Office_Open_XML https://en.wikipedia.org/wiki/Standardization_of_Office_Open...
charlieyu1 1y ago
I once digged through the 5000 page specification. There was a lot of useless stuff that only old Microsoft Word supported like WordArt items.
- bawolff 1y ago
  Does office no longer support word art? When i was a kid,making cool wordart headers for school projects was like 50% of what we used office for.
  - lblume 1y ago
    Office does still support word art. [0] [0]: https://support.microsoft.com/en-us/office/insert-wordart-c5070583-1ebe-4dc4-a41f-5e3729adce54 https://support.microsoft.com/en-us/office/insert-wordart-c5...
    - bjoli 1y ago
      How else would terminally uncool church youth groups advertise in their local church? It might be a Swedish thing, but I always laugh when I see them. Not nearly as common today as ten years ago, but I see them a couple of times a year.
RcouF1uZ4gsC 1y ago
> Faced with demands for openness, Microsoft could have produced a clean, modern spec and keep the mass pile of legacy inside the application. Very, very few people care about openness. Maybe a few hundred. Tens of millions care about docx capturing exactly what their doc files had. Microsoft made the correct choice.
stuzenz 1y ago
My theory (from anecdotal use) is that the OOXML complexity also explains why M365 office implementation is lacking in so many features and is just not very good at all when compared to the Google office suite. I do have strong memories of OOXML and the scandals that were with it when it became a standard through MS allegedly buying/stacking/influencing votes: https://chatgpt.com/share/68bf5e11-4e10-8003-ac9d-d4d10f795176 https://chatgpt.com/share/68bf5e11-4e10-8003-ac9d-d4d10f7951...
tracker1 1y ago
I think the last part is probably the biggest thing holding them back IMO... I tend not to install MS Office products on my personal devices, I haven't run Windows on a personal device in a few years. I've mostly maintained just my resume in word or libre-office format for well over a decade. I can't tell you how many times the LO format lost formatting, or just messed up between version upgrades. Same goes for opening a word version in LO. That doesn't count the various times where it behaved weird, inconsistently had fields/tables that were impossible to edit, etc. I've had to completely recreate everything a couple times over the years. That's just one document, for one guy that I don't really touch that often. Say what you will about Firefox vs Chrome in terms of usability, compared to MS Word using LibreOffice is worse than early betas of Netscape Navigator 4.0. It's both impressive and upsetting. OnlyOffice at least looks nicer, even if it doesn't really function any better. MS's online version of Word in the browser operates more consistently than either.
- abhinavk 1y ago
  Have you tried setting MSWord's default save format to Strict OOXML for inter-operation with LibreOffice?
  - tracker1 1y ago
    I've generated the docs in LibreOffice and tried to only use that. My intent wasn't too use MS Word at all. But I have used Word for work.
tannhaeuser 1y ago
Worth keeping in mind that the native MSO formats were using "structured storage", a horrible binary chunked serialization and metadata format from an era where binary embedding of document streams in other application documents via "Object linking and embedding" (OLE, see also Apple's OpenDoc format) was deemed desirable, with zero consideration given to third-party apps and segment formats tied to C++ data structures. Compared to that, OOXML is still a huge progress, and while it's complex I wouldn't say it's maliciously so. The Shakespeare example is a good one where the sentence is split into multiple spans to apply style rules yet the bare text content could be extracted by just removing all XML tags. Whereas the ODF variant is actually less recommendable as it relies on an unneccesarily complex formatting and text addressing language on top of XML. The article says > Even at a glance [ODF's markup] is more intelligible. Strip the text: namespaces and it’s nearly valid HTML. The only thing that needs explaining is that ODF doesn’t wrap To be with a dedicated “bold” tag. Instead, it applies an auto-style named T1 to a <text:span>, an act of separating content and presentation that mirrors established web practices. but this definitely makes things more complex for data exchange compared to OOXML.
- quotemstr 1y ago
  Can you explain what's wrong with the concept of a container format that allows embedding subdocuments of different types? > zero consideration given to third-party apps and segment formats The reality is the opposite. COM serialization was specifically built to allow for composing components (and serializations thereof) that didn't know about each other into a single document. That's why it leans so heavily on GUIDs for names: they avoid collisions without needing coordination. That's a laudable goal, not pointless bloat. And the COM people implemented it pretty efficiently too! > C++ data structures What gives you that idea? Yes, the OLE stream thing was a binary format, but so is DER for ASN.1. Every webpage you load goes over a binary tagged object format not too different from OLE/COM's. But due to a persistence of myths from the 90s, people still think of the Office binary format as "horrible" when it's actually quite elegant, especially considering the problems the authors had to solve and their constraints in doing so. In many ways, we've regressed. > Markup The author of the article nails it when he says ODF is meant to be a markup language and OOXML is the serialization of an object graph. So what? Do people write ODF by hand? There are countless JSON formats just as inscrutable as MSO's legacy streams. Anyway, the idea that the MSO binary format was crap because it was binary, lazy, and represented a "memory dump" is an old myth that just won't die. It wasn't a memory dump, it wasn't lazy, and it wasn't crap. Yes, there are real problems with some of the things people put inside the OLE container, but it's facile and wrong to blame the container or the OLE stream composition model for the problem.
pessimizer 1y ago
I have no idea what this article is intending to express. It is artificially complex to dump the exact implementations of your legacy products into a giant data structure and call it a standard. Nobody can implement that. Which is why they had to bribe, stuff committees and bully people to get it done. I don't think anyone cares about debating the word "artificial," I don't think that was anyone's point. It's just not a standard. It was, as is made clear here, a way to head off a standard that would be possible to competitors to implement with a fake standard that Microsoft couldn't even implement. I also don't think that it is "a counterproductive reflex that’s common in open-source circles: scolding users for accepting proprietary tech." I don't even know wtf that's supposed to mean. People are stuck with it because of corruption, they're not being scolded for using it. edit: "LibreOffice itself, as ODF’s flagship, still suffers from rough edges in design, interaction, and performance. As a result, even as Office hobble itself with bloat, most people still find it easier." Yeah, it'd be a lot easier if they didn't every have to deal with OOXML and could just work on their own product.
lorenzohess 1y ago
> In my view, OOXML is indeed complex, convoluted, and obscure. But that’s likely less about a plot to block third-party compatibility and more about a self-interested negligence: Microsoft prioritized the convenience of its own implementation and neglected the qualities of clarity, simplicity, and universality that a general-purpose standard should have. The author only provides arguments for "self-interested negligence". He provides no counterarguments to the claim that OOXML complexity was "a plot to block third-party compatibility". Therefore, he cannot compare "negligence" and "a plot". Therefore, his claim that "negligence" is a better explanation for OOXML complexity than "a plot" cannot follow. To restate: > If we dig into the context of OOXML’s creation, it can be argued that harming competitors was not Microsoft’s primary aim. The author provides no evidence to support this claim. At most, the evidence provided in this section at most supports the claim that "negligence" played a role in OOXML complexity. From this evidence alone, no conclusions can be drawn about the "primariness" of "negligence" vs "harming competitors".
- to11mtm 1y ago
  I mean sometimes you gotta ship a product (and remember back then, that meant masters for CDs,) and it's perfectly possible that whatever team was in charge of handling 'conversion' stuff for old format (remember that old excel formats have OLE type cruft going on, the sorts of things that led to VBA viruses, imagine what other functionality needs to be implemented) just plain had to take shortcuts in uglifying the spec to support all the jank.
- unscaled 1y ago
  Unless we ever get the full archive of Microsoft emails, meeting minutes and recordings from all the secret microphones they didn't have in their meeting rooms, I don't think you can ever disprove this claim. It's generally impossible to conclusively disprove conspiracy theories, because you could always claim you're only showing there are no documents proving the conspiracy, but there are no documents disproving it. The author is just implicitly appealing to Occam's razor here, as people often in face of accusations of a plot. They can show that Microsoft has backed the ANSI accreditation of ODF[1] and eventually implemented support for ODF import and export in Office, but that's not enough to prove there was no conspiracy. Instead, the article just provides a very plausible explanation for the complexity in OOXML. Does this explanation thoroughly disprove the accusations of a plot? Clear not. Is it more plausible than a great plot to crush a bunch of competitors that had no market share and kill a better standard document format that Microsoft did end up implementing in Office? Yes. This is probably as far as we can get. [1] https://news.microsoft.com/source/2007/05/16/microsoft-votes-for-choice/ https://news.microsoft.com/source/2007/05/16/microsoft-votes...
deleted 1y ago
[deleted]
- deleted 1y ago
  [deleted]
deleted 1y ago
[deleted]
PaulHoule 1y ago
People who were developing "office" programs in the early 1990s were thinking about the problem of serializing arbitrary object graphs into documents to support technologies like https://en.wikipedia.org/wiki/Object_Linking_and_Embedding https://en.wikipedia.org/wiki/Object_Linking_and_Embedding where you could embed an Excel spreadsheet inside a Word document or actually embedded any of a large range of COM objects into a Word document which on one hand is a really appealing vision but on the other hand means you have to have and be able to run all the binaries for all the objects that live in a document which ties the whole thing to Windows. PDF is a different sort of document format which privileges viewing over editing but it is also really about serializing an object graph when it comes down to it and then having various sorts of filters and transformations and a range of objects defined in the spec as opposed to open ended access to an object library. This kind of system has a lot of overlap with the serdes problem you get with RPC frameworks that used to be under the files "Sun RPC sucks", "DCOM Sucks", "CORBA Sucks" and "WS-* Sucks" Those things are mostly forgotten these days because well... they sucked, and now the usual complaint is "protobuf sucks" but you rarely hear "JSON sucks" because it gave up on graphs for trees, if you don't have a type system people can't say the type system sucks, and the only thing that really sucks about it is that people won't just use ISO 8601 dates but you can always rise above that by just using ISO 8601 dates without asking for permission. But we all agree YAML sucks. That points to any flexible document format sucking but also sucks because it has lots of poorly specified and obscure features that amount to "format this the same way Word 95 formatted it if you used a certain obscure option". From a glass is half empty perspective it sucks because it's close to impossible to make a Microsoft Office replacement that renders 100% of documents 100% correctly. From a glass is half empty perspective it rules because if you want to make a Python script that writes an Excel script with formulas it is easy. If you want to extract the images out of a Word document it is easy because a Word document is just a ZIP file. If you want to do anything with an OOXML document short of writing an Office replacement it's actually a pretty good situation.
- com2kid 1y ago
  > but you rarely hear "JSON sucks" because it gave up on graphs for trees Except it also spawned a thousand custom formats that include $ref support of some type, so we are right back to having graphs. :-D
Lammy 1y ago
I love this screen that shows you exactly why they named it “Office Open” XML: https://i.imgur.com/hnj3sdv.png https://i.imgur.com/hnj3sdv.png It was a pretty big deal when OpenOffice.org's 2.0 release came with OpenDocument as the default file format. Very easy for someone to misread this MSOffice screen and click on OOXML expecting it to mean OO.o.
- zamadatix 1y ago
  Oh wow. I must have clicked through that page dozens of times, selecting "Keep Current" after a quick scan and thinking the 2nd option was talking about Open Office.
  - Lammy 1y ago
    > after a quick scan I have to wonder what sort of psychologists they employ who come up with ideas like aligning the “Word, Excel, PowerPoint” word column in the first selection with “Open” in the second selection so you read that word first and backtrack left to “Office”. Or maybe it's just a happy accident lol
croes 1y ago
> Thus, the primary goal for this new format wasn’t to be elegant, universal, or easy to implement; it was to placate regulators while preserving Microsoft’s technological and commercial advantages. That sound exactly like it is an anti-competitive format. Keeping the own advantage sums pretty all anti-competitive behavior.
eirikbakke 1y ago
Microsoft Office has many features. Each feature must be reflected in the file format somehow. (I wonder what the specification-pages-to-man-years ratio is...)
freeopinion 1y ago
This is talking about OOXML the proprietary MS format, right? Not ISO/IEC 29500? ISO/IEC 29500 should be open to evolution, no? Just like all the open collaboration on it before it was confirmed as a standard.
mxmilkiib 1y ago
https://m.slashdot.org/story/78708 https://m.slashdot.org/story/78708 (2007)
themerone 1y ago
It's as complex as it needs to be to losslessly convert old binary office files. A better format would have made us geeks a lot happier, but the average user just wants things to work the way they always have.
- Gigachad 1y ago
  My possibly incomplete understanding was that the original office file format was basically just raw dumps of the internal C data structures. Not designed or specified in any way. The XML version likely carries a lot of baggage having to be compatible with that.
  - lmkg 1y ago
    They weren't "just" raw dumps of internal C structures. It takes careful design work to dump raw memory in a usable fashion. Consider: You can't just write a pointer to disk and then read it back next week. Binary MS Office format is a phenomenal piece of engineering to achieve a goal that's no longer relevant: fast save/load on late-80's hard drives. Other programs took minutes to save a spreadsheet, Excel took seconds. It did this by making sure it's in-memory data structures for a document could be dumped straight to disk without transformation. But yes, this approach carries a shitton of baggage. And that achievement is no longer relevant in a world where consumer hardware can parse XML documents on the fly. I have heard it argued, though, that the "baggage" isn't the file format. It's actually the full historical featureset of Excel. Being backwards-compatible means being able to faithfully represent the features of old Excel, and the essential complexity of that far outweighs the incidental complexity of how those features were encoded.
taspeotis 1y ago
Off topic sorry but with all the comments discussing Office's size and age and technical baggage ... does anyone know how they pivoted from X million lines of code for a desktop application to running it on the web with all those collaboration features?
nashashmi 1y ago
OOXML carries bloat from a full legacy doc file into a docx file. Readability was not the mission of the developers of the open format. Openness was the mission of the developers of the format. And they made it open enough.
s20n 1y ago
> Why Microsoft’s Motive Wasn’t Deliberate Sabotage I absolutely do not agree. Not only is the standard overly complex, Microsoft also indulged in all sorts of unscrupulous activities to corrupt various National Standards Organisations to get it approved through the ISO <https://en.wikipedia.org/wiki/Standardization_of_Office_Open_XML#Complaints_about_the_national_bodies_process https://en.wikipedia.org/wiki/Standardization_of_Office_Open...>, which is clear evidence of malicious intent. This is a quote from Richard Stallman: > The specifications document was so long that it would be difficult for anyone else to implement it properly. When the proposed standard was submitted through the usual track, experienced evaluators rejected it for many good reasons. Microsoft responded using a special override procedure in which its money buy the support of many of the voting countries, thus bypassing proper evaluation and demonstrating that ISO can be bought.
- monocasa 1y ago
  Specifically what I heard on the grapevine was that Microsoft sponsored a collection of small island nations into the ISO process, in exchange for their vote on OOXML.
  - miohtama 1y ago
    Not only small islands nations. For example in Finland Microsoft partners invaded the local working group to get the standard passed in the voting process.
  - Yizahi 1y ago
    That is not on MS though. That is a fault of those in change of ISO, that they assign same vote weight to the enormous empires and to the microstates. Votes should be proportional to the population, full stop. Then no one would be able to abuse the system by simply playing by the rules.
    - MereInterest 1y ago
      So, if I'm understanding your argument correctly, failure to stop a bad actor from taking a hostile action absolves the bad actor of all responsibility for that hostile action? Because that seems to be what you're saying.
nneonneo 1y ago
Microsoft seems to have known that they could ram basically anything through a standards body, so they presumably didn't bother to actually try and simplify the standard. Instead, it's basically an XML serialization of their older binary formats, complete with all of the quirks and bugs that have to be emulated for 100% compatibility. To be fair, we're talking about a product line with over 35 years of history here. Cruft in the format builds up but can never be removed, so long as you commit to strong backwards compatibility - which Microsoft has always done. Fun trivia: many of the old binary formats use a meta-format called OLE2 (Object Linking and Embedding). The file format is a FAT12 filesystem packed into a single file, with a FAT filesystem chain, file blocks aligned to a specific power-of-two size, etc. This made saving files very fast, but raised the possibility of internal fragmentation (where individual sub-files are scattered over many non-contiguous blocks); hence, users were recommended to "Save As..." periodically for large/complex files to optimize the internal storage.
- flomo 1y ago
  Officially now MS-CFB (i think). OLE2 generally refers to a predecessor to COM, and not just the file format. https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-cfb/53989ce4-7b05-4f8d-829b-d08d6148375b https://learn.microsoft.com/en-us/openspecs/windows_protocol...
  - masfuerte 1y ago
    Being pedantic, OLE1 was the predecessor. OLE2 used COM for its plumbing. Wikipedia has an article on the file format [1]. It was quite nice. It works like an uncompressed zip file with transactional updates. Earlier Word document formats were much worse. They were a dump of Word's memory contents. Saving and loading was very quick though! [1]: https://en.wikipedia.org/wiki/Compound_File_Binary_Format https://en.wikipedia.org/wiki/Compound_File_Binary_Format
- rtpg 1y ago
  "You have to standardize the format" "OK we will standardize our serialization format" It's... I guess malicious compliance, though also if you don't care about interop you're not going to try to abstract away your internal application structures, are you! I appreciate the standard existing rather than it not existing. Trying to have the standard exist in this way has always felt like an uphill battle, and at least now there's _something_. Just you will have a better time if you emulate how Office does things. But you have a bit more documentation to go along with it.
Mikhail_Edoshin 1y ago
I remember Spreadsheet ML, an older format compatible with Excel. It had a subset of features, I think, but it was a rather powerful subset: formatting, formulae, multiple sheets. And it was rather simple. (Had a silly design mistake: for some reason MS gave namespace to attributes, which is not necessary, only for rather specific purposes). Another XML standard from MS that also seems relatively simple is XPS, a PDF alternative. But it uses Open Packaging and that is somewhat hard to read.
CobrastanJorji 1y ago
The OOXML fight is near and dear to my heart because, when it happened, I was a baby developer, and I cared about the issue for some reason I can barely recall, and I found an expert on the issue on Twitter. That guy would regularly tweet about everything that was going on and the problems with the spec and the shenanigans, and I was one of the, like, 20 people who was hanging on his every word. And sometimes he'd talk about bee keeping instead. It was my first introduction to Twitter at its best. You got these unfiltered whole views of the lives and concerns of real people who were, in part, experts at what you cared about. So sometimes you had to listen to them talk about other random stuff they thought was neat. And that's great!
- fsflover 1y ago
  > and I cared [...] and I found an expert So did you somehow contribute to it in the end?
  - CobrastanJorji 1y ago
    Nope! I consumed content, felt informed, and did nothing to help anything. It's a tradition that I and many like me keep to this day.
Upoices 1y ago
[dead]
rjsw 1y ago
Another fake standard is ISO 14306.
Freak_NL 1y ago
If you ever write some HTTP endpoint where tabulated data is returned, you could quite reasonably return RFC 4180 style CSV. However, if your API ever interfaces with users in a corporate environment, parsing simple comma-separated UTF-8 CSV is suddenly quite beyond the reach of however is nibbling at your endpoint, so why not code up a simple little reusable bit of code where you can write any simple tabular data (string, numbers, and dates, in one or more sheets of data made up of rows and columns) that lets you choose the output format? A zip-archive of CSV-files (one per sheet), JSON, ODS, or XLSX; pick your poison. I did just that, and while it is perfectly doable, any low-level, low-resources, low-dependency approach will mean actually touching the XML in LibreOffice's ODS (fine), and Microsoft's OOXML (…). This is how you write a date in a cell in both. ODS: <table:table-row table:style-name="ro1"> <table:table-cell office:value-type="date" office:date-value="2021-04-10T12:34:56" calcext:value-type="date"> <text:p>10/4/2021, 12:34</text:p> </table:table-cell> </table:table-row> OK, a bit verbose, but trivial to implement. Format the date however you like — you'll probably use two different formatters on the same datetime instant. XLSX (OOXML): <row r="1" ht="12.8"> <c r="A1" s="1" t="n"> <v>39448.5</v> </c> </row> Obviously, as you can all plainly see, the date here is 2008-01-01T12:00:00… And of course it makes perfect sense to hardcode the cell coordinate there. It's not like you would dynamically generate a bunch of cells (…).
- boricj 1y ago
  > However, if your API ever interfaces with users in a corporate environment, parsing simple comma-separated UTF-8 CSV is suddenly quite beyond the reach of however is nibbling at your endpoint Excel can directly ingest a CSV file served over an URL as data source, with the Accept header manually set to text/csv. I wrote a backend once that supported this feature so that management could pull whatever data they wanted off an internal application without pestering me. They could literally take the URL of a page and pull it as a CSV file as-is. Anybody who knows a bit of Excel can pull that data themselves by following a set of simple instructions.
  - Freak_NL 1y ago
donatj 1y ago
You want to see something "fun" check out Apple's Numbers XML format from before they switched to protobuf. A simple 1,2,3 resulted in over 1 megabyte of XML. At a previous job I'd been tasked with developing support for importing Numbers files along side our existing Excel and CSV support. After a couple days we rightly gave up as the tiny fraction of people who actually wanted to import Numbers files was outweighed by it's massive complexity. We ended up just adding instructions for Numbers users to export to CSV
jonathaneunice 1y ago
Yes, artificially complex. No one genuinely interested in document openness—be it for document workflows, publishing automation, content archiving, or future-proofed documents—would have done it that way. Maybe it was as simple as dumping a simplistic re-encoding of its legacy binary format into XML and ramming it through standards organizations. But yes, there was malice aforethought and a classic Microsoft playbook in motion: embrace, extend, extinguish.
noAnswer 1y ago
I guess OP wasn't around back then when Microsoft openly bribed African countries to join ISO just to vote on this topic. This says it all: https://cdn.imgpile.com/p/RppGj1l https://cdn.imgpile.com/p/RppGj1l
- fantyoon 1y ago
  > The voting that followed was among ISO’s most contentious: several national bodies abruptly swelled with new members, many Microsoft partners, who then voted in favor. Sweden’s initial approval was voided after incentives linked to support came to light. Direct quote from the article.
EdwardCoffin 1y ago
I remember reading an early criticism of the spreadsheet side of OOXML, where a simple spreadsheet with three cells was created: A1 containing '1', B1 containing '2', and C1 containing the formula A1+B1. That spreadsheet was saved, the file opened in an editor which showed the values of the cells, and A1 changed to something else, say 3. This broke the spreadsheet, as there were all sorts of knock-on effects contained in the virtually opaque mess that followed the cell contents. I've probably got the details wrong, but that was the gist of it. I'd love to rediscover the analysis, but my searches have not yielded it.
- itsthecourier 1y ago
  exactly, the author ignored the specs and try to come with conclusions of a system, just by doing a Hello World
  - EdwardCoffin 1y ago
    I think the point of the criticism I read was that the edit should have worked. There is no reason why the opaque mess following what was obviously a definition of the contents of the spreadsheet should even be there let alone be dependent on the original contents of the cells.
    - jeroenhd 1y ago
      The obvious reason to have other stuff depend on the value of the cell would be to store a cache alongside the formulae. In a 300MiB XSLX, you don't want to evaluate every formula every time the spreadsheet is opened.
      - EdwardCoffin 1y ago
        If caches have a place in a file storage format, they should at least be optional and separate from mandatory content, and I got the impression from the critique that they were neither.
itsthecourier 1y ago
they should have used some examples with tables, tackle of contents, macros, styling, columns, page size, etc. the blog post is artificially shallow
bayindirh 1y ago
If a standard which allows embedding closed-spec binary blobs even Microsoft can't implement perfectly from version to version is not a deliberate sabotage attempt, then I don't know what is. This is Microsoft. Don't get distracted.

162 points by firexcy 1y ago

Microsoft just took what they had and directly translated it to XML. It's not intentionally messy, it's just a big corporation with old product acting like it.

gitonup 1y ago
This is the God's honest. I worked on the MS Word core team for a little over three years from 2010-2014, and de-facto owned a significant part of implementing ODF / OOXML Strict support. The binary format was a liability for Microsoft to begin with, because of decades of cruft lining up with actual memory alignment. During my tenure there I ran into code my GM had written as an intern and was still intact -- he had 20+ years of tenure (mostly on Word) when I joined the team. The translation of the file format to XML involved a significant amount of performance degradation if you weren't careful. Hundreds of millions of people use the app monthly, and MS still tries to maintain backwards compatibility. Given that open APIs were a relatively late development for the app, I really don't think in the current reality of what's expected by boards of directors for the companies they oversee that _anyone_ would take years to: a) define a spec that maintained that backwards compatibility b) reach whatever nebulous simplicity metric today's HN article wants c) not get whoever greenlit the project fired for taking that many engineering hours for a and b

piker 1y ago

Dead on. Microsoft is just dominant and exporting its 40 year old legacy codebase as a spec. LibreOffice team is frustrated that the for-profit model is beating the OSS model and crying foul over mostly necessary complexity. If LibreOffice started from scratch they’d probably appreciate how much Microsoft serializes because a sufficiently complicated document saved to .docx basically provides a reference implementation. We do need for-profit alternatives to Word, and I’m working on one in legal. [edit: I hope to put some real thoughts on this down soon, but most of the wonkiness emanates from evolving functionality and varying trends in best practices over the decades. I’ve implemented a fair bit of the spec here: https://tritium.legal https://tritium.legal, but most of the hard part is providing for bidi language support, fonts, real-time editing and re-rendering, UI and annotations like spellchecking and grammar, not conforming to the markup spec. Spec conformance is just polish and testing. A performant modern word processor of any spec, however, is a technological achievement on the order of a web browser.]

trelane 1y ago
LibreOffice has versions that you pay for, with support. The most prominent is Collabora, which is a (if not the) biggest contributor to LibreOffice.
croes 1y ago
Where does the article say it’s a necessary complexity? > Thus, the primary goal for this new format wasn’t to be elegant, universal, or easy to implement; it was to placate regulators while preserving Microsoft’s technological and commercial advantages. That sounds quite anti-competitive to me
Gigachad 1y ago
I feel like Libreoffice became largely irrelevant the day Google Docs came out. People put up with LO wonkyness because it was free and office was expensive. Google completely flipped the game and then cloud collaboration became everything.
- toast0 1y ago
  I mean, multiplayer features are useful, but Google Docs is wonkier than LO. At least when LO loads a document, it's fully loaded.
taftster 1y ago
> We do need for-profit alternatives to Word, and I’m working on one in legal. Wow, big undertaking! What we really need, though, is a for-profit alternative to Excel, that's not Google. I think Excel is more of the Killer App than Word has ever been.
- qcnguy 1y ago
  That's Apple Numbers.
  - mschuster91 1y ago
    ... which comes tied to macOS and with it Apple hardware. Neither play well in a shop that uses x86 Windows-only software, and Apple's switch to ARM hasn't made that easier.

etothepii 1y ago

I spent a lot of time last year replicating every valid Excel number format. I've really struggled to find good documentation on the excel format when you really get into the weeds. The use of namespaces is also incredibly annoying in so far as I can tell in every xml library I can find they really aren't well supported for that "human" readable component. When you crack open the file it feels like you are going to be able to find everything you need with an xpath like //w:t but none of the xml parsers I've found cope well with the namespaces.

rhdunn 1y ago
What language? In Python, the `find`, `findall`, etc. methods take a namespace dictionary. E.g. result = doc.findall("//w:t", namespaces={"w": "..."}) In C# you can do: var navigator = doc.Root!.CreateNavigator(); nsManager = new XmlNamespaceManager(navigator.NameTable); nsManager.AddNamespace("w", "..."); var results = doc.Root?.XPathSelectElements("//w:t", nsManager); In Java you need to enable a namespace-aware flag in the settings to get namespaces to work. I can't recall off-hand how to do that.
- etothepii 1y ago
  Yes I'm talking Python. The namespaces are long, dare I say ugly, urls. Even though the xml file itself uses `<w:t` you can't unless you provide a dictionary as an argument that contains `{"w":"https://...."}` which means that you need to do a bunch of reading and understanding before you can start playing with the files.

Joker_vD 1y ago

sigh Just because it was not deliberately engineered to be prohitibively expensive to support does not mean that it can not be used to deliberately obstruct interoperability. It's really not that difficult a concept: if you want others to suffer, you can take a sad artifact of well-meant historical accidents, and say "welp, now it's a standard, you gotta support it!" There is nothing contradictory or conspirational.

piker 1y ago
I think we take issue with requiring the leap to Microsoft “deliberately” obstructing interoperability. Microsoft just isn’t incentivized to make it simple to implement, but it’s probably less complicated than the various web standards.
- Joker_vD 1y ago
  An engineering team in Microsoft decides to switch from binary format to XML to save effort in the long run; even though it'll take some effort now, they have the competency, and can afford it. They are absolutely correct! But then their manager needs to sell this project to the higher-ups, who have read BillG's memo about how "One thing we have got to change in our strategy – allowing Office documents to be rendered very well by other people's browsers is one of the most destructive things we could do to the company. We have to stop putting any effort into this and make sure that Office documents very well depend on proprietary IE capabilities. Anything else is suicide for our platform. This is a case where Office has to avoid doing something to destroy Windows." and took it to heart. So what does he do? Why, he spins a tale that since it's XML, they'll be able to standardize it, and everyone else will still be forced to interoperate with MS Office anyhow, because it will be the de-facto reference implementation (by the virtue of being there first, and widely deployed), and the spec is going to be an absolute PITA to implement decently — and that manager too will be absolutely correct!
  - piker 1y ago
    It’s not actually that bad.

cyberax 1y ago

The answer: no. OOXML is an extremely detailed spec that lists minute details of the Office documents, with uncountable features. While it could have used some "standard" features, there weren't that many usable standards when OOXML was being developed. In comparison, OASIS OpenDocument spec is horribly ambiguous and has all the same issues (like units not being used consistently). It got better over the years, but it's still not at all great. And its size is now comparable to OOXML, when all the referenced specs are incorporated.

rhdunn 1y ago
There are places where it says the equivalent of "Works the same as Word 95" [3], but does not specify in the specification what that means. It's essentially a serialization of the binary format to XML. ODF 1.4 is around 1,100 pages across all 4 parts whereas OOXML is over 6,000. [1] https://stephesblog.blogs.com/my_weblog/2007/08/microsofts-fail.html https://stephesblog.blogs.com/my_weblog/2007/08/microsofts-f... [2] https://ooxmlisdefectivebydesign.blogspot.com/2007/08/microsoft-office-xml-formats-defective.html https://ooxmlisdefectivebydesign.blogspot.com/2007/08/micros... [3] https://www.robweir.com/blog/2007/01/how-to-hire-guillaume-portes.html https://www.robweir.com/blog/2007/01/how-to-hire-guillaume-p...
- cyberax 1y ago
  > There are places where it says the equivalent of "Works the same as Word 95" [3], but does not specify in the specification what that means. Yeah, sure, whatever. You'll never see these kinds of documents in real life. And the specified quirks were minor. If you don't implement them, you'll get subtle formatting issues in documents imported directly from Word97. MS could have just put them into a "vendor-specific" extension and not documented them at all. > ODF 1.4 is around 1,100 pages across all 4 parts whereas OOXML is over 6,000. LOL, no. SVG spec alone is 800 pages. ODF formula spec is 200 pages alone, and is still underspecified.
- xeeeeeeeeeeenu 1y ago
  They improved this in later revisions of the standard. The behaviour of autoSpaceLikeWord95 is now actually described and there's an example. You can see it for yourself here (in Part 4): https://ecma-international.org/publications-and-standards/standards/ecma-376/ https://ecma-international.org/publications-and-standards/st...

theanonymousone 1y ago

Duplicate: https://news.ycombinator.com/item?id=45147639 https://news.ycombinator.com/item?id=45147639

fsflover 1y ago
It's not a duplicate if there's no discussion.
- theanonymousone 1y ago
  How come the same link was accepted twice,in the first place?
  - fsflover 1y ago
    Resubmissions are acceptable, if the discussion hasn't started.

fsflover 1y ago

See also: https://en.wikipedia.org/wiki/Standardization_of_Office_Open_XML https://en.wikipedia.org/wiki/Standardization_of_Office_Open...

charlieyu1 1y ago

I once digged through the 5000 page specification. There was a lot of useless stuff that only old Microsoft Word supported like WordArt items.

bawolff 1y ago
Does office no longer support word art? When i was a kid,making cool wordart headers for school projects was like 50% of what we used office for.
- lblume 1y ago
  Office does still support word art. [0] [0]: https://support.microsoft.com/en-us/office/insert-wordart-c5070583-1ebe-4dc4-a41f-5e3729adce54 https://support.microsoft.com/en-us/office/insert-wordart-c5...
  - bjoli 1y ago
    How else would terminally uncool church youth groups advertise in their local church? It might be a Swedish thing, but I always laugh when I see them. Not nearly as common today as ten years ago, but I see them a couple of times a year.

RcouF1uZ4gsC 1y ago

> Faced with demands for openness, Microsoft could have produced a clean, modern spec and keep the mass pile of legacy inside the application. Very, very few people care about openness. Maybe a few hundred. Tens of millions care about docx capturing exactly what their doc files had. Microsoft made the correct choice.

stuzenz 1y ago

My theory (from anecdotal use) is that the OOXML complexity also explains why M365 office implementation is lacking in so many features and is just not very good at all when compared to the Google office suite. I do have strong memories of OOXML and the scandals that were with it when it became a standard through MS allegedly buying/stacking/influencing votes: https://chatgpt.com/share/68bf5e11-4e10-8003-ac9d-d4d10f795176 https://chatgpt.com/share/68bf5e11-4e10-8003-ac9d-d4d10f7951...

tracker1 1y ago

I think the last part is probably the biggest thing holding them back IMO... I tend not to install MS Office products on my personal devices, I haven't run Windows on a personal device in a few years. I've mostly maintained just my resume in word or libre-office format for well over a decade. I can't tell you how many times the LO format lost formatting, or just messed up between version upgrades. Same goes for opening a word version in LO. That doesn't count the various times where it behaved weird, inconsistently had fields/tables that were impossible to edit, etc. I've had to completely recreate everything a couple times over the years. That's just one document, for one guy that I don't really touch that often. Say what you will about Firefox vs Chrome in terms of usability, compared to MS Word using LibreOffice is worse than early betas of Netscape Navigator 4.0. It's both impressive and upsetting. OnlyOffice at least looks nicer, even if it doesn't really function any better. MS's online version of Word in the browser operates more consistently than either.

abhinavk 1y ago
Have you tried setting MSWord's default save format to Strict OOXML for inter-operation with LibreOffice?
- tracker1 1y ago
  I've generated the docs in LibreOffice and tried to only use that. My intent wasn't too use MS Word at all. But I have used Word for work.

tannhaeuser 1y ago

Worth keeping in mind that the native MSO formats were using "structured storage", a horrible binary chunked serialization and metadata format from an era where binary embedding of document streams in other application documents via "Object linking and embedding" (OLE, see also Apple's OpenDoc format) was deemed desirable, with zero consideration given to third-party apps and segment formats tied to C++ data structures. Compared to that, OOXML is still a huge progress, and while it's complex I wouldn't say it's maliciously so. The Shakespeare example is a good one where the sentence is split into multiple spans to apply style rules yet the bare text content could be extracted by just removing all XML tags. Whereas the ODF variant is actually less recommendable as it relies on an unneccesarily complex formatting and text addressing language on top of XML. The article says > Even at a glance [ODF's markup] is more intelligible. Strip the text: namespaces and it’s nearly valid HTML. The only thing that needs explaining is that ODF doesn’t wrap To be with a dedicated “bold” tag. Instead, it applies an auto-style named T1 to a <text:span>, an act of separating content and presentation that mirrors established web practices. but this definitely makes things more complex for data exchange compared to OOXML.

quotemstr 1y ago
Can you explain what's wrong with the concept of a container format that allows embedding subdocuments of different types? > zero consideration given to third-party apps and segment formats The reality is the opposite. COM serialization was specifically built to allow for composing components (and serializations thereof) that didn't know about each other into a single document. That's why it leans so heavily on GUIDs for names: they avoid collisions without needing coordination. That's a laudable goal, not pointless bloat. And the COM people implemented it pretty efficiently too! > C++ data structures What gives you that idea? Yes, the OLE stream thing was a binary format, but so is DER for ASN.1. Every webpage you load goes over a binary tagged object format not too different from OLE/COM's. But due to a persistence of myths from the 90s, people still think of the Office binary format as "horrible" when it's actually quite elegant, especially considering the problems the authors had to solve and their constraints in doing so. In many ways, we've regressed. > Markup The author of the article nails it when he says ODF is meant to be a markup language and OOXML is the serialization of an object graph. So what? Do people write ODF by hand? There are countless JSON formats just as inscrutable as MSO's legacy streams. Anyway, the idea that the MSO binary format was crap because it was binary, lazy, and represented a "memory dump" is an old myth that just won't die. It wasn't a memory dump, it wasn't lazy, and it wasn't crap. Yes, there are real problems with some of the things people put inside the OLE container, but it's facile and wrong to blame the container or the OLE stream composition model for the problem.

pessimizer 1y ago

I have no idea what this article is intending to express. It is artificially complex to dump the exact implementations of your legacy products into a giant data structure and call it a standard. Nobody can implement that. Which is why they had to bribe, stuff committees and bully people to get it done. I don't think anyone cares about debating the word "artificial," I don't think that was anyone's point. It's just not a standard. It was, as is made clear here, a way to head off a standard that would be possible to competitors to implement with a fake standard that Microsoft couldn't even implement. I also don't think that it is "a counterproductive reflex that’s common in open-source circles: scolding users for accepting proprietary tech." I don't even know wtf that's supposed to mean. People are stuck with it because of corruption, they're not being scolded for using it. edit: "LibreOffice itself, as ODF’s flagship, still suffers from rough edges in design, interaction, and performance. As a result, even as Office hobble itself with bloat, most people still find it easier." Yeah, it'd be a lot easier if they didn't every have to deal with OOXML and could just work on their own product.

lorenzohess 1y ago

> In my view, OOXML is indeed complex, convoluted, and obscure. But that’s likely less about a plot to block third-party compatibility and more about a self-interested negligence: Microsoft prioritized the convenience of its own implementation and neglected the qualities of clarity, simplicity, and universality that a general-purpose standard should have. The author only provides arguments for "self-interested negligence". He provides no counterarguments to the claim that OOXML complexity was "a plot to block third-party compatibility". Therefore, he cannot compare "negligence" and "a plot". Therefore, his claim that "negligence" is a better explanation for OOXML complexity than "a plot" cannot follow. To restate: > If we dig into the context of OOXML’s creation, it can be argued that harming competitors was not Microsoft’s primary aim. The author provides no evidence to support this claim. At most, the evidence provided in this section at most supports the claim that "negligence" played a role in OOXML complexity. From this evidence alone, no conclusions can be drawn about the "primariness" of "negligence" vs "harming competitors".

to11mtm 1y ago
I mean sometimes you gotta ship a product (and remember back then, that meant masters for CDs,) and it's perfectly possible that whatever team was in charge of handling 'conversion' stuff for old format (remember that old excel formats have OLE type cruft going on, the sorts of things that led to VBA viruses, imagine what other functionality needs to be implemented) just plain had to take shortcuts in uglifying the spec to support all the jank.
unscaled 1y ago
Unless we ever get the full archive of Microsoft emails, meeting minutes and recordings from all the secret microphones they didn't have in their meeting rooms, I don't think you can ever disprove this claim. It's generally impossible to conclusively disprove conspiracy theories, because you could always claim you're only showing there are no documents proving the conspiracy, but there are no documents disproving it. The author is just implicitly appealing to Occam's razor here, as people often in face of accusations of a plot. They can show that Microsoft has backed the ANSI accreditation of ODF[1] and eventually implemented support for ODF import and export in Office, but that's not enough to prove there was no conspiracy. Instead, the article just provides a very plausible explanation for the complexity in OOXML. Does this explanation thoroughly disprove the accusations of a plot? Clear not. Is it more plausible than a great plot to crush a bunch of competitors that had no market share and kill a better standard document format that Microsoft did end up implementing in Office? Yes. This is probably as far as we can get. [1] https://news.microsoft.com/source/2007/05/16/microsoft-votes-for-choice/ https://news.microsoft.com/source/2007/05/16/microsoft-votes...

deleted 1y ago

[deleted]

deleted 1y ago
[deleted]

deleted 1y ago

[deleted]

PaulHoule 1y ago

People who were developing "office" programs in the early 1990s were thinking about the problem of serializing arbitrary object graphs into documents to support technologies like https://en.wikipedia.org/wiki/Object_Linking_and_Embedding https://en.wikipedia.org/wiki/Object_Linking_and_Embedding where you could embed an Excel spreadsheet inside a Word document or actually embedded any of a large range of COM objects into a Word document which on one hand is a really appealing vision but on the other hand means you have to have and be able to run all the binaries for all the objects that live in a document which ties the whole thing to Windows. PDF is a different sort of document format which privileges viewing over editing but it is also really about serializing an object graph when it comes down to it and then having various sorts of filters and transformations and a range of objects defined in the spec as opposed to open ended access to an object library. This kind of system has a lot of overlap with the serdes problem you get with RPC frameworks that used to be under the files "Sun RPC sucks", "DCOM Sucks", "CORBA Sucks" and "WS-* Sucks" Those things are mostly forgotten these days because well... they sucked, and now the usual complaint is "protobuf sucks" but you rarely hear "JSON sucks" because it gave up on graphs for trees, if you don't have a type system people can't say the type system sucks, and the only thing that really sucks about it is that people won't just use ISO 8601 dates but you can always rise above that by just using ISO 8601 dates without asking for permission. But we all agree YAML sucks. That points to any flexible document format sucking but also sucks because it has lots of poorly specified and obscure features that amount to "format this the same way Word 95 formatted it if you used a certain obscure option". From a glass is half empty perspective it sucks because it's close to impossible to make a Microsoft Office replacement that renders 100% of documents 100% correctly. From a glass is half empty perspective it rules because if you want to make a Python script that writes an Excel script with formulas it is easy. If you want to extract the images out of a Word document it is easy because a Word document is just a ZIP file. If you want to do anything with an OOXML document short of writing an Office replacement it's actually a pretty good situation.

com2kid 1y ago
> but you rarely hear "JSON sucks" because it gave up on graphs for trees Except it also spawned a thousand custom formats that include $ref support of some type, so we are right back to having graphs. :-D

Lammy 1y ago

I love this screen that shows you exactly why they named it “Office Open” XML: https://i.imgur.com/hnj3sdv.png https://i.imgur.com/hnj3sdv.png It was a pretty big deal when OpenOffice.org's 2.0 release came with OpenDocument as the default file format. Very easy for someone to misread this MSOffice screen and click on OOXML expecting it to mean OO.o.

zamadatix 1y ago
Oh wow. I must have clicked through that page dozens of times, selecting "Keep Current" after a quick scan and thinking the 2nd option was talking about Open Office.
- Lammy 1y ago
  > after a quick scan I have to wonder what sort of psychologists they employ who come up with ideas like aligning the “Word, Excel, PowerPoint” word column in the first selection with “Open” in the second selection so you read that word first and backtrack left to “Office”. Or maybe it's just a happy accident lol

croes 1y ago

> Thus, the primary goal for this new format wasn’t to be elegant, universal, or easy to implement; it was to placate regulators while preserving Microsoft’s technological and commercial advantages. That sound exactly like it is an anti-competitive format. Keeping the own advantage sums pretty all anti-competitive behavior.

eirikbakke 1y ago

Microsoft Office has many features. Each feature must be reflected in the file format somehow. (I wonder what the specification-pages-to-man-years ratio is...)

freeopinion 1y ago

This is talking about OOXML the proprietary MS format, right? Not ISO/IEC 29500? ISO/IEC 29500 should be open to evolution, no? Just like all the open collaboration on it before it was confirmed as a standard.

mxmilkiib 1y ago

https://m.slashdot.org/story/78708 https://m.slashdot.org/story/78708 (2007)

themerone 1y ago

It's as complex as it needs to be to losslessly convert old binary office files. A better format would have made us geeks a lot happier, but the average user just wants things to work the way they always have.

Gigachad 1y ago
My possibly incomplete understanding was that the original office file format was basically just raw dumps of the internal C data structures. Not designed or specified in any way. The XML version likely carries a lot of baggage having to be compatible with that.
- lmkg 1y ago
  They weren't "just" raw dumps of internal C structures. It takes careful design work to dump raw memory in a usable fashion. Consider: You can't just write a pointer to disk and then read it back next week. Binary MS Office format is a phenomenal piece of engineering to achieve a goal that's no longer relevant: fast save/load on late-80's hard drives. Other programs took minutes to save a spreadsheet, Excel took seconds. It did this by making sure it's in-memory data structures for a document could be dumped straight to disk without transformation. But yes, this approach carries a shitton of baggage. And that achievement is no longer relevant in a world where consumer hardware can parse XML documents on the fly. I have heard it argued, though, that the "baggage" isn't the file format. It's actually the full historical featureset of Excel. Being backwards-compatible means being able to faithfully represent the features of old Excel, and the essential complexity of that far outweighs the incidental complexity of how those features were encoded.

taspeotis 1y ago

Off topic sorry but with all the comments discussing Office's size and age and technical baggage ... does anyone know how they pivoted from X million lines of code for a desktop application to running it on the web with all those collaboration features?

nashashmi 1y ago

OOXML carries bloat from a full legacy doc file into a docx file. Readability was not the mission of the developers of the open format. Openness was the mission of the developers of the format. And they made it open enough.

s20n 1y ago

> Why Microsoft’s Motive Wasn’t Deliberate Sabotage I absolutely do not agree. Not only is the standard overly complex, Microsoft also indulged in all sorts of unscrupulous activities to corrupt various National Standards Organisations to get it approved through the ISO <https://en.wikipedia.org/wiki/Standardization_of_Office_Open_XML#Complaints_about_the_national_bodies_process https://en.wikipedia.org/wiki/Standardization_of_Office_Open...>, which is clear evidence of malicious intent. This is a quote from Richard Stallman: > The specifications document was so long that it would be difficult for anyone else to implement it properly. When the proposed standard was submitted through the usual track, experienced evaluators rejected it for many good reasons. Microsoft responded using a special override procedure in which its money buy the support of many of the voting countries, thus bypassing proper evaluation and demonstrating that ISO can be bought.

monocasa 1y ago
Specifically what I heard on the grapevine was that Microsoft sponsored a collection of small island nations into the ISO process, in exchange for their vote on OOXML.
- miohtama 1y ago
  Not only small islands nations. For example in Finland Microsoft partners invaded the local working group to get the standard passed in the voting process.
- Yizahi 1y ago
  That is not on MS though. That is a fault of those in change of ISO, that they assign same vote weight to the enormous empires and to the microstates. Votes should be proportional to the population, full stop. Then no one would be able to abuse the system by simply playing by the rules.
  - MereInterest 1y ago
    So, if I'm understanding your argument correctly, failure to stop a bad actor from taking a hostile action absolves the bad actor of all responsibility for that hostile action? Because that seems to be what you're saying.

nneonneo 1y ago

Microsoft seems to have known that they could ram basically anything through a standards body, so they presumably didn't bother to actually try and simplify the standard. Instead, it's basically an XML serialization of their older binary formats, complete with all of the quirks and bugs that have to be emulated for 100% compatibility. To be fair, we're talking about a product line with over 35 years of history here. Cruft in the format builds up but can never be removed, so long as you commit to strong backwards compatibility - which Microsoft has always done. Fun trivia: many of the old binary formats use a meta-format called OLE2 (Object Linking and Embedding). The file format is a FAT12 filesystem packed into a single file, with a FAT filesystem chain, file blocks aligned to a specific power-of-two size, etc. This made saving files very fast, but raised the possibility of internal fragmentation (where individual sub-files are scattered over many non-contiguous blocks); hence, users were recommended to "Save As..." periodically for large/complex files to optimize the internal storage.

flomo 1y ago
Officially now MS-CFB (i think). OLE2 generally refers to a predecessor to COM, and not just the file format. https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-cfb/53989ce4-7b05-4f8d-829b-d08d6148375b https://learn.microsoft.com/en-us/openspecs/windows_protocol...
- masfuerte 1y ago
  Being pedantic, OLE1 was the predecessor. OLE2 used COM for its plumbing. Wikipedia has an article on the file format [1]. It was quite nice. It works like an uncompressed zip file with transactional updates. Earlier Word document formats were much worse. They were a dump of Word's memory contents. Saving and loading was very quick though! [1]: https://en.wikipedia.org/wiki/Compound_File_Binary_Format https://en.wikipedia.org/wiki/Compound_File_Binary_Format
rtpg 1y ago
"You have to standardize the format" "OK we will standardize our serialization format" It's... I guess malicious compliance, though also if you don't care about interop you're not going to try to abstract away your internal application structures, are you! I appreciate the standard existing rather than it not existing. Trying to have the standard exist in this way has always felt like an uphill battle, and at least now there's _something_. Just you will have a better time if you emulate how Office does things. But you have a bit more documentation to go along with it.

Mikhail_Edoshin 1y ago

I remember Spreadsheet ML, an older format compatible with Excel. It had a subset of features, I think, but it was a rather powerful subset: formatting, formulae, multiple sheets. And it was rather simple. (Had a silly design mistake: for some reason MS gave namespace to attributes, which is not necessary, only for rather specific purposes). Another XML standard from MS that also seems relatively simple is XPS, a PDF alternative. But it uses Open Packaging and that is somewhat hard to read.

CobrastanJorji 1y ago

The OOXML fight is near and dear to my heart because, when it happened, I was a baby developer, and I cared about the issue for some reason I can barely recall, and I found an expert on the issue on Twitter. That guy would regularly tweet about everything that was going on and the problems with the spec and the shenanigans, and I was one of the, like, 20 people who was hanging on his every word. And sometimes he'd talk about bee keeping instead. It was my first introduction to Twitter at its best. You got these unfiltered whole views of the lives and concerns of real people who were, in part, experts at what you cared about. So sometimes you had to listen to them talk about other random stuff they thought was neat. And that's great!

fsflover 1y ago
> and I cared [...] and I found an expert So did you somehow contribute to it in the end?
- CobrastanJorji 1y ago
  Nope! I consumed content, felt informed, and did nothing to help anything. It's a tradition that I and many like me keep to this day.

Upoices 1y ago

[dead]

rjsw 1y ago

Another fake standard is ISO 14306.

Freak_NL 1y ago

If you ever write some HTTP endpoint where tabulated data is returned, you could quite reasonably return RFC 4180 style CSV. However, if your API ever interfaces with users in a corporate environment, parsing simple comma-separated UTF-8 CSV is suddenly quite beyond the reach of however is nibbling at your endpoint, so why not code up a simple little reusable bit of code where you can write any simple tabular data (string, numbers, and dates, in one or more sheets of data made up of rows and columns) that lets you choose the output format? A zip-archive of CSV-files (one per sheet), JSON, ODS, or XLSX; pick your poison. I did just that, and while it is perfectly doable, any low-level, low-resources, low-dependency approach will mean actually touching the XML in LibreOffice's ODS (fine), and Microsoft's OOXML (…). This is how you write a date in a cell in both. ODS: <table:table-row table:style-name="ro1"> <table:table-cell office:value-type="date" office:date-value="2021-04-10T12:34:56" calcext:value-type="date"> <text:p>10/4/2021, 12:34</text:p> </table:table-cell> </table:table-row> OK, a bit verbose, but trivial to implement. Format the date however you like — you'll probably use two different formatters on the same datetime instant. XLSX (OOXML): <row r="1" ht="12.8"> <c r="A1" s="1" t="n"> <v>39448.5</v> </c> </row> Obviously, as you can all plainly see, the date here is 2008-01-01T12:00:00… And of course it makes perfect sense to hardcode the cell coordinate there. It's not like you would dynamically generate a bunch of cells (…).

boricj 1y ago
> However, if your API ever interfaces with users in a corporate environment, parsing simple comma-separated UTF-8 CSV is suddenly quite beyond the reach of however is nibbling at your endpoint Excel can directly ingest a CSV file served over an URL as data source, with the Accept header manually set to text/csv. I wrote a backend once that supported this feature so that management could pull whatever data they wanted off an internal application without pestering me. They could literally take the URL of a page and pull it as a CSV file as-is. Anybody who knows a bit of Excel can pull that data themselves by following a set of simple instructions.
- Freak_NL 1y ago

donatj 1y ago

You want to see something "fun" check out Apple's Numbers XML format from before they switched to protobuf. A simple 1,2,3 resulted in over 1 megabyte of XML. At a previous job I'd been tasked with developing support for importing Numbers files along side our existing Excel and CSV support. After a couple days we rightly gave up as the tiny fraction of people who actually wanted to import Numbers files was outweighed by it's massive complexity. We ended up just adding instructions for Numbers users to export to CSV

jonathaneunice 1y ago

Yes, artificially complex. No one genuinely interested in document openness—be it for document workflows, publishing automation, content archiving, or future-proofed documents—would have done it that way. Maybe it was as simple as dumping a simplistic re-encoding of its legacy binary format into XML and ramming it through standards organizations. But yes, there was malice aforethought and a classic Microsoft playbook in motion: embrace, extend, extinguish.

noAnswer 1y ago

I guess OP wasn't around back then when Microsoft openly bribed African countries to join ISO just to vote on this topic. This says it all: https://cdn.imgpile.com/p/RppGj1l https://cdn.imgpile.com/p/RppGj1l

fantyoon 1y ago
> The voting that followed was among ISO’s most contentious: several national bodies abruptly swelled with new members, many Microsoft partners, who then voted in favor. Sweden’s initial approval was voided after incentives linked to support came to light. Direct quote from the article.

EdwardCoffin 1y ago

I remember reading an early criticism of the spreadsheet side of OOXML, where a simple spreadsheet with three cells was created: A1 containing '1', B1 containing '2', and C1 containing the formula A1+B1. That spreadsheet was saved, the file opened in an editor which showed the values of the cells, and A1 changed to something else, say 3. This broke the spreadsheet, as there were all sorts of knock-on effects contained in the virtually opaque mess that followed the cell contents. I've probably got the details wrong, but that was the gist of it. I'd love to rediscover the analysis, but my searches have not yielded it.

itsthecourier 1y ago
exactly, the author ignored the specs and try to come with conclusions of a system, just by doing a Hello World
- EdwardCoffin 1y ago
  I think the point of the criticism I read was that the edit should have worked. There is no reason why the opaque mess following what was obviously a definition of the contents of the spreadsheet should even be there let alone be dependent on the original contents of the cells.
  - jeroenhd 1y ago
    The obvious reason to have other stuff depend on the value of the cell would be to store a cache alongside the formulae. In a 300MiB XSLX, you don't want to evaluate every formula every time the spreadsheet is opened.
    - EdwardCoffin 1y ago
      If caches have a place in a file storage format, they should at least be optional and separate from mandatory content, and I got the impression from the critique that they were neither.

itsthecourier 1y ago

they should have used some examples with tables, tackle of contents, macros, styling, columns, page size, etc. the blog post is artificially shallow

bayindirh 1y ago

If a standard which allows embedding closed-spec binary blobs even Microsoft can't implement perfectly from version to version is not a deliberate sabotage attempt, then I don't know what is. This is Microsoft. Don't get distracted.