8 ms·

A jump into the middle of an instruction from nowhere (2023)

131 points by smitop 3y ago

mbork_pl 3y ago
Seeing the title and the domain I instantly knew whose blog post this is...
- deusum 3y ago
  Is it just me? His posts always begin with an interesting technical anecdote and then veer off into boring propaganda with sullies the reputability. It's an intellectual let-down.
  - bigstrat2003 3y ago
    That's just you. Raymond Chen is a great programmer and writer, we are lucky to have him.
- layer8 3y ago
  I wonder who will tackle those problems when he finally retires. ;)
deleted 3y ago
[deleted]
TravHatesMe 3y ago
Maybe large swaths of users wouldn't download enhancement suites if you actually listened to them? It took like 3 years to support the basic feature to "never group tabs" in the Taskbar -- that's been supported by previous versions of windows for decades. That's why I downloaded patches.
- deleted 3y ago
  [deleted]
- TremendousJudge 3y ago
  Agree. I didn't download patches because I knew from experience (Win8 start menu patching) that it was asking for trouble down the road.
saurik 3y ago
Frankly, it is really annoying when developers working at one of the large closed-source platform oligopolists blame their users for trying to work around the deficiencies (or even outright hostile "business model" optimizations) in their software by whatever scant means are left to them: maybe if Microsoft didn't keep removing good functionality and stopped forcing bad "features" on everyone, fewer people would see a need to install and use these extensions that are being scapegoated here for the lack of satisfaction in Windows updates. Just look through the comments on this article to see people pointing out how Microsoft keeps removing options for how people can do this in less risky ways, or even ideas for how Microsoft can improve the safety of these extensions (which aren't all exactly right but I will note that if Microsoft embraced these more and provided an official loader--not one that limited the access but merely coordinated the installation--this is easily solved with the same kind of "safe mode" I designed for shell extensions on jailbroken iOS devices). To quote one of the comments: > Not just this article, but far too often when articles such as this are written, about widely used 3rd party tweaks, and the commenting strongly supports the tweak in question, but MS charges forward that “all these enthusiasts are wrong, we will continue our way” is telling of how far out of touch the Win dev team is away from its user base.
- rob74 3y ago
  Well yeah, I mean no one forces you to use Explorer for file management under Windows. I'm an old-time Norton Commander user, and when Windows came around I switched to Total Commander. There are open-source alternatives too, even cross-platform ones, like this one: https://doublecmd.sourceforge.io/ https://doublecmd.sourceforge.io/. That being said, no one forces you to use Windows either - except maybe your employer or the software you are using, but this is getting less and less of a problem fortunately (web apps, Proton etc.).
  - abbeyj 3y ago
    This isn't really about using Explorer for file management. It is about using Explorer as the shell. That's the default setup for everybody. Changing the default steel is difficult and some programs may not be compatible with whatever alternative shell you switch to. Most people don't want to learn a whole new shell. They just want the Explorer shell to have the features that it has had since Windows 7. Features that Microsoft inexplicably chose to remove in Windows 11 and that they are only now belatedly adding back.
Ciantic 3y ago
I know it must be frustrating from Microsoft's standpoint, but those are needed because they keep changing things. Start All Back is one very popular explorer.exe patch that makes the Windows 11 taskbar function a lot like Windows 10. For one thing, it allows you to move the taskbar again. I've also made an explorer.exe in-memory patch to disable flashing taskbar buttons in Windows 10, it injected itself to WndProc and set the specific message to zero. In Windows 11 it's now a setting, so I don't need that anymore. Now I'm investigating an explorer.exe patch for Windows 11 to speed up wallpaper animations. Which cannot be configured at the moment, there is only one setting to turn off all animations but no setting to speed up the wallpaper fade in animation.
- vesinisa 3y ago
  > flashing taskbar buttons Oh geez these are annoying! I've spent hours trying to find a setting to disable the flashing taskbar on the Teams client on my corporate issued Win10 laptop. The best I've found is to keep the Teams client minimized in tray just to avoid this behavior. It boggles my mind there's no way to disable this feature that is so distracting it almost makes working impossible whenever I forget to kill the Teams client window. And really goes to show why people need to resort to these hacks.
  - n6h6 3y ago
    I am glad someone else agrees. It's absolutely horrible!
- kristianp 3y ago
  Do you have to turn off all system file modification detection to do these hacks? Sounds risky.
  - arsome 3y ago
    These are not applied on disk, they're injected at runtime. Just open the process and write to its memory directly.
jackewiehose 3y ago
There is an interview with Raymond Chen where he is trying to defend the reason why they crippled the taskbar with Windows 11 [1]. Watching that I was just thinking NO NO NO - it worked fine for the last 25 years and there can be absolutely no reason why they had to destroy the taskbar now. He deserves every headache he gets with these support requests. Or to quote this comment of the linked article: > There wouldn’t be as much of a need if the windows shell team was smarter [...] In fact I actively encourage, support, and celebrate efforts like this. Make their life hell and delay them. If they spend more time on investigating issues like this, they’ll have less time to f*k something else up. [1]: https://www.youtube.com/watch?v=HDicLHBvQQM https://www.youtube.com/watch?v=HDicLHBvQQM
- daemin 3y ago
  Unfortunately it's the designers that are screwing up the UI/UX of Windows, not the programmers. If you give more time to the designers then you're just going to get more of a clone of OSX rather than something useful.
  - Drakim 3y ago
    All the while they use OSX themselves and don't feel any of the pain of their half-baked copy.
  - jonhohle 3y ago
    I don’t use windows often, but I was using it to check some hardware the other day and had to dig through nests of settings to get back to back to what was more or less the display settings screen from the Windows 9x/NT era, albeit with a less efficient layout and lots of superfluous white space. Unfortunately, macOS has gone this direction as well and it’s settings have become inscrutable. Simple things like using drag and drop to reorder network interface priorities are luxuries of the past. It seems like those designing have never used them and never plan to.
  - WorldMaker 3y ago
    The OSX dock has almost always supported vertical orientations, nearly as long as the Windows taskbar used to. If they are trying to clone OSX they are doing a miserable job at it in their choices of features to cut.
protoman3000 3y ago
[flagged]
revskill 3y ago
[flagged]
adrian_b 3y ago
While in this case the jump into the middle of an instruction was not intentional, several decades ago many Microsoft programs for MS-DOS, like their BASIC interpreter, contained frequent intentional jumps into the middle of instructions. This technique had already been used in many Microsoft programs for CP/M, i.e. for Intel 8080 or Zilog Z80 CPUs. There were two reasons for the use of such jumps in the middle of instructions. One was to obfuscate the program for disassemblers, which were desynchronized with the instruction stream by such jumps. The other reason is that each such jump usually saved one byte of instruction code in comparison with conventional conditional program structures. When the program size was still measured in kilobytes, saving even only one hundred bytes by this method was important. The instruction into the middle of which a jump was performed was usually an instruction that loaded a 16-bit register with an immediate value that happened to be the encoding for some other instruction. The value loaded into the register was not used. Therefore, when the instruction was entered through its start address, no operation was done, but when it was entered through a jump, some extra instruction was executed. A typical use case would be at the beginning of a loop, which was entered after initializing a control variable, while whenever the loop was repeated later an extra instruction was executed (corresponding to the immediate value of the register load), to update the control variable. This saved one byte upon the standard loop implementation that uses an unconditional jump for entering the body of the loop in its middle, after the loop initialization.
- praptak 3y ago
  Biology sometimes "jumps into the middle of instructions" when decoding DNA/RNA. The "instruction" length is always 3 nucleotides and the sequence usually has only one valid "entry point". There are exceptions though. Some sequences do encode biologically useful information with sequences which not only overlap but do so with the starting points differing by a number not divisible by 3: https://en.wikipedia.org/wiki/Reading_frame#Multiple_reading_frames https://en.wikipedia.org/wiki/Reading_frame#Multiple_reading...
  - PaulDavisThe1st 3y ago
    You'd almost hope there was some younger more code-centric version of Douglas Hofstadter, figuring out how to write palindromic or multi-framed assembler that was semantically equivalent to, for example, his "Crab Canon".
snapcaster 3y ago
[flagged]
bombcar 3y ago
Kind of sad. Even from this blog, he admits that the Microsoft of old would test software and see if it worked; the Microsoft of now obviously knows of the existence of these shell enhancements, but clearly doesn’t test patches against them at all.
- throwawaaarrgh 3y ago
  That's a lot of money and time to spend on supporting someone else's bug-riddled software. You can't test literally all software that exists before you release a security patch. Just imagine testing every single solitary Windows application that exists, or has every existed, just to see if one of them crashes due to intentionally doing the wrong thing. What are they supposed to do, fix the 3rd party software? Delay fixing the security hole? Testing is the responsibility of the 3rd party (in addition to using supported methods). If the 3rd party was part of a preview release program, they could test changes before they are officially released.
  - userbinator 3y ago
    That's a lot of money and time to spend on supporting someone else's bug-riddled software. Consider that it's money and time that MS spent to get into their position today. They built their empire on "back-compat is king", and betraying that principle gives far less reason for their customers to continue using their software instead of moving to alternatives.
    - Kwpolska 3y ago
      Microsoft cares about backwards compatibility and does a good job at it IMO, especially compared to their competitors. The thing that broke was not a public API, but an internal, undocumented, unexported function in Explorer. Microsoft did patch API abuse in the past for prominent software, but they can't be expected to do it forever and for all software.
      - JohnFen 3y ago
        > Microsoft cares about backwards compatibility and does a good job at it Indeed. This is one of the few points that I give high marks to Microsoft for, and when it comes to Windows, is the only thing that makes me feel sympathy for Microsoft devs. Pulling off the level of backward compatibility that Windows has maintained for so long is an incredible accomplishment.
mst 3y ago
One of the comments says: > There’s at least one such utility out there that properly enough validates if their patches would work, or blocklist specific Windows builds known to break from it completely. I wish the other very-obvious-from-your-article software would do so too. and I can't help but agree. If you're going to reach into the guts of another process and start flipping bits, it's only Correct to take responsibility for doing so carefully enough you don't blow the entire thing up. (I have written code that replaced/wrapped other people's function pointers for various reasons (occasionally even -good- ones) and considered any problem caused by an upgrade of the code hosting my cuckoo chick to be my bug - and while it was occasionally bloody annoying, I did this to myself ;)
EdSharkey 3y ago
So these patchers are monkey patching a system executable needed to have a bootable/usable system? Why is this allowed? This sounds like a small hell. I left Windows the moment we shifted from "developers, developers, developers, developers" to "advertisers, advertisers, advertisers, advertisers" and never looked back. Ubuntu is very pleasant to work with, nowadays.
- deleted 3y ago
  [deleted]
omoikane 3y ago
I wonder if there is a special name for this patching pattern, where not only does a piece of code depend on undocumented behavior, it's depending on the exact binary bytes. See also: https://www.hyrumslaw.com/ https://www.hyrumslaw.com/
Jones4000 3y ago
[dead]

131 points by smitop 3y ago

mbork_pl 3y ago

Seeing the title and the domain I instantly knew whose blog post this is...

deusum 3y ago
Is it just me? His posts always begin with an interesting technical anecdote and then veer off into boring propaganda with sullies the reputability. It's an intellectual let-down.
- bigstrat2003 3y ago
  That's just you. Raymond Chen is a great programmer and writer, we are lucky to have him.
layer8 3y ago
I wonder who will tackle those problems when he finally retires. ;)

deleted 3y ago

[deleted]

TravHatesMe 3y ago

Maybe large swaths of users wouldn't download enhancement suites if you actually listened to them? It took like 3 years to support the basic feature to "never group tabs" in the Taskbar -- that's been supported by previous versions of windows for decades. That's why I downloaded patches.

deleted 3y ago
[deleted]
TremendousJudge 3y ago
Agree. I didn't download patches because I knew from experience (Win8 start menu patching) that it was asking for trouble down the road.

saurik 3y ago

Frankly, it is really annoying when developers working at one of the large closed-source platform oligopolists blame their users for trying to work around the deficiencies (or even outright hostile "business model" optimizations) in their software by whatever scant means are left to them: maybe if Microsoft didn't keep removing good functionality and stopped forcing bad "features" on everyone, fewer people would see a need to install and use these extensions that are being scapegoated here for the lack of satisfaction in Windows updates. Just look through the comments on this article to see people pointing out how Microsoft keeps removing options for how people can do this in less risky ways, or even ideas for how Microsoft can improve the safety of these extensions (which aren't all exactly right but I will note that if Microsoft embraced these more and provided an official loader--not one that limited the access but merely coordinated the installation--this is easily solved with the same kind of "safe mode" I designed for shell extensions on jailbroken iOS devices). To quote one of the comments: > Not just this article, but far too often when articles such as this are written, about widely used 3rd party tweaks, and the commenting strongly supports the tweak in question, but MS charges forward that “all these enthusiasts are wrong, we will continue our way” is telling of how far out of touch the Win dev team is away from its user base.

rob74 3y ago
Well yeah, I mean no one forces you to use Explorer for file management under Windows. I'm an old-time Norton Commander user, and when Windows came around I switched to Total Commander. There are open-source alternatives too, even cross-platform ones, like this one: https://doublecmd.sourceforge.io/ https://doublecmd.sourceforge.io/. That being said, no one forces you to use Windows either - except maybe your employer or the software you are using, but this is getting less and less of a problem fortunately (web apps, Proton etc.).
- abbeyj 3y ago
  This isn't really about using Explorer for file management. It is about using Explorer as the shell. That's the default setup for everybody. Changing the default steel is difficult and some programs may not be compatible with whatever alternative shell you switch to. Most people don't want to learn a whole new shell. They just want the Explorer shell to have the features that it has had since Windows 7. Features that Microsoft inexplicably chose to remove in Windows 11 and that they are only now belatedly adding back.

Ciantic 3y ago

I know it must be frustrating from Microsoft's standpoint, but those are needed because they keep changing things. Start All Back is one very popular explorer.exe patch that makes the Windows 11 taskbar function a lot like Windows 10. For one thing, it allows you to move the taskbar again. I've also made an explorer.exe in-memory patch to disable flashing taskbar buttons in Windows 10, it injected itself to WndProc and set the specific message to zero. In Windows 11 it's now a setting, so I don't need that anymore. Now I'm investigating an explorer.exe patch for Windows 11 to speed up wallpaper animations. Which cannot be configured at the moment, there is only one setting to turn off all animations but no setting to speed up the wallpaper fade in animation.

vesinisa 3y ago
> flashing taskbar buttons Oh geez these are annoying! I've spent hours trying to find a setting to disable the flashing taskbar on the Teams client on my corporate issued Win10 laptop. The best I've found is to keep the Teams client minimized in tray just to avoid this behavior. It boggles my mind there's no way to disable this feature that is so distracting it almost makes working impossible whenever I forget to kill the Teams client window. And really goes to show why people need to resort to these hacks.
- n6h6 3y ago
  I am glad someone else agrees. It's absolutely horrible!
kristianp 3y ago
Do you have to turn off all system file modification detection to do these hacks? Sounds risky.
- arsome 3y ago
  These are not applied on disk, they're injected at runtime. Just open the process and write to its memory directly.

jackewiehose 3y ago

There is an interview with Raymond Chen where he is trying to defend the reason why they crippled the taskbar with Windows 11 [1]. Watching that I was just thinking NO NO NO - it worked fine for the last 25 years and there can be absolutely no reason why they had to destroy the taskbar now. He deserves every headache he gets with these support requests. Or to quote this comment of the linked article: > There wouldn’t be as much of a need if the windows shell team was smarter [...] In fact I actively encourage, support, and celebrate efforts like this. Make their life hell and delay them. If they spend more time on investigating issues like this, they’ll have less time to f*k something else up. [1]: https://www.youtube.com/watch?v=HDicLHBvQQM https://www.youtube.com/watch?v=HDicLHBvQQM

daemin 3y ago
Unfortunately it's the designers that are screwing up the UI/UX of Windows, not the programmers. If you give more time to the designers then you're just going to get more of a clone of OSX rather than something useful.
- Drakim 3y ago
  All the while they use OSX themselves and don't feel any of the pain of their half-baked copy.
- jonhohle 3y ago
  I don’t use windows often, but I was using it to check some hardware the other day and had to dig through nests of settings to get back to back to what was more or less the display settings screen from the Windows 9x/NT era, albeit with a less efficient layout and lots of superfluous white space. Unfortunately, macOS has gone this direction as well and it’s settings have become inscrutable. Simple things like using drag and drop to reorder network interface priorities are luxuries of the past. It seems like those designing have never used them and never plan to.
- WorldMaker 3y ago
  The OSX dock has almost always supported vertical orientations, nearly as long as the Windows taskbar used to. If they are trying to clone OSX they are doing a miserable job at it in their choices of features to cut.

protoman3000 3y ago

[flagged]

revskill 3y ago

[flagged]

adrian_b 3y ago

While in this case the jump into the middle of an instruction was not intentional, several decades ago many Microsoft programs for MS-DOS, like their BASIC interpreter, contained frequent intentional jumps into the middle of instructions. This technique had already been used in many Microsoft programs for CP/M, i.e. for Intel 8080 or Zilog Z80 CPUs. There were two reasons for the use of such jumps in the middle of instructions. One was to obfuscate the program for disassemblers, which were desynchronized with the instruction stream by such jumps. The other reason is that each such jump usually saved one byte of instruction code in comparison with conventional conditional program structures. When the program size was still measured in kilobytes, saving even only one hundred bytes by this method was important. The instruction into the middle of which a jump was performed was usually an instruction that loaded a 16-bit register with an immediate value that happened to be the encoding for some other instruction. The value loaded into the register was not used. Therefore, when the instruction was entered through its start address, no operation was done, but when it was entered through a jump, some extra instruction was executed. A typical use case would be at the beginning of a loop, which was entered after initializing a control variable, while whenever the loop was repeated later an extra instruction was executed (corresponding to the immediate value of the register load), to update the control variable. This saved one byte upon the standard loop implementation that uses an unconditional jump for entering the body of the loop in its middle, after the loop initialization.

praptak 3y ago
Biology sometimes "jumps into the middle of instructions" when decoding DNA/RNA. The "instruction" length is always 3 nucleotides and the sequence usually has only one valid "entry point". There are exceptions though. Some sequences do encode biologically useful information with sequences which not only overlap but do so with the starting points differing by a number not divisible by 3: https://en.wikipedia.org/wiki/Reading_frame#Multiple_reading_frames https://en.wikipedia.org/wiki/Reading_frame#Multiple_reading...
- PaulDavisThe1st 3y ago
  You'd almost hope there was some younger more code-centric version of Douglas Hofstadter, figuring out how to write palindromic or multi-framed assembler that was semantically equivalent to, for example, his "Crab Canon".

snapcaster 3y ago

[flagged]

bombcar 3y ago

Kind of sad. Even from this blog, he admits that the Microsoft of old would test software and see if it worked; the Microsoft of now obviously knows of the existence of these shell enhancements, but clearly doesn’t test patches against them at all.

throwawaaarrgh 3y ago
That's a lot of money and time to spend on supporting someone else's bug-riddled software. You can't test literally all software that exists before you release a security patch. Just imagine testing every single solitary Windows application that exists, or has every existed, just to see if one of them crashes due to intentionally doing the wrong thing. What are they supposed to do, fix the 3rd party software? Delay fixing the security hole? Testing is the responsibility of the 3rd party (in addition to using supported methods). If the 3rd party was part of a preview release program, they could test changes before they are officially released.
- userbinator 3y ago
  That's a lot of money and time to spend on supporting someone else's bug-riddled software. Consider that it's money and time that MS spent to get into their position today. They built their empire on "back-compat is king", and betraying that principle gives far less reason for their customers to continue using their software instead of moving to alternatives.
  - Kwpolska 3y ago
    Microsoft cares about backwards compatibility and does a good job at it IMO, especially compared to their competitors. The thing that broke was not a public API, but an internal, undocumented, unexported function in Explorer. Microsoft did patch API abuse in the past for prominent software, but they can't be expected to do it forever and for all software.
    - JohnFen 3y ago
      > Microsoft cares about backwards compatibility and does a good job at it Indeed. This is one of the few points that I give high marks to Microsoft for, and when it comes to Windows, is the only thing that makes me feel sympathy for Microsoft devs. Pulling off the level of backward compatibility that Windows has maintained for so long is an incredible accomplishment.

mst 3y ago

One of the comments says: > There’s at least one such utility out there that properly enough validates if their patches would work, or blocklist specific Windows builds known to break from it completely. I wish the other very-obvious-from-your-article software would do so too. and I can't help but agree. If you're going to reach into the guts of another process and start flipping bits, it's only Correct to take responsibility for doing so carefully enough you don't blow the entire thing up. (I have written code that replaced/wrapped other people's function pointers for various reasons (occasionally even -good- ones) and considered any problem caused by an upgrade of the code hosting my cuckoo chick to be my bug - and while it was occasionally bloody annoying, I did this to myself ;)

EdSharkey 3y ago

So these patchers are monkey patching a system executable needed to have a bootable/usable system? Why is this allowed? This sounds like a small hell. I left Windows the moment we shifted from "developers, developers, developers, developers" to "advertisers, advertisers, advertisers, advertisers" and never looked back. Ubuntu is very pleasant to work with, nowadays.

deleted 3y ago
[deleted]

omoikane 3y ago

I wonder if there is a special name for this patching pattern, where not only does a piece of code depend on undocumented behavior, it's depending on the exact binary bytes. See also: https://www.hyrumslaw.com/ https://www.hyrumslaw.com/

Jones4000 3y ago

[dead]