5 ms·

Exploring the Halo 1 System Link Protocol

220 points by nhellman 3y ago

kmeisthax 3y ago
Tangential point: the weirdest part about the Xbox was the system name. As far as I'm aware that was never configurable by users, but certain games would set your system name as an easter egg. Any reason why it was built that way?
reidjs 3y ago
This is a well written, in depth analysis of the system. Why do you think the packets are encrypted?
- PawBer 3y ago
  The LAN code is probably the same code as the online multiplayer.
  - ClassyJacket 3y ago
    Halo 1 on original Xbox does not have online multiplayer. It launched before Xbox Live did.
    - msk-lywenn 3y ago
      They were probably already working on it
      - rafark 3y ago
        The game was actually close to ship without multiplayer, it was an afterthought. Some devs have talked about this (iirc, Marty O’Donnell was one of them).
        notnaut 3y ago
        I’ve heard this was also the case for Goldeneye on the N64
        ravenstine 3y ago
        Goldeneye is one of the best case studies for why companies should give their engineers room to experiment. If I remember correctly, the multiplayer mode was originally a side project of one of the engineers and was added near the end of development. This isn't to say the campaign mode isn't good, but the ability for boys to play a shooter game against each other on the same console is what made that game.
        thomastjeffery 3y ago
        The game being an FPS was an afterthought, too. Originally, they were working on an RTS.
        monocasa 3y ago
        I wouldn't call it an afterthought per se, more a pivot with a lot of work. It was already an FPS when it was shown off at MacWorld'99.
    - monocasa 3y ago
      As someone who's written replacement backend Xbox Live server infra, I can confirm that system link is very nearly the same base protocol set as is used by Live. They must have had that nailed down pretty close to launch, and were mainly working on support services that live needs but system link doesn't up until Live actually launched.
- zootboy 3y ago
  Probably for the reason the author speculates about in the article: > For example, would it not be cool if there was a game with a stack buffer overflow bug that allowed us to run arbitrary code remotely? That could potentially enable us to softmod an Xbox over the network, without the need of any special hardware.
- nhellman 3y ago
  Thanks, I was initially surprised that they were encrypted but I can speculate on a few potential reasons: - The Xbox was designed to be able to play online via Xbox Live (e.g. with Halo 2 that came out later in 2004), and they might have simply reused the network stack for System Link over LAN. I looked a little bit at Halo 2 system link, it uses the same system calls from the kernel but the protocol seems to be more complicated (e.g. the IV is never sent in plaintext, the two consoles derive it somehow). I haven't looked at Xbox Live, but Halo 2 could potentially use the same for System Link and Xbox Live. - The Xbox also really tried to lock down its security in order to prevent game piracy and homebrew games. This might have been another attempt to reduce the attack surface. - They also might have just wanted to keep their game protocols secret for trade secrets or simply avoiding scrutiny. In this case, we were able to create a kind of cheat, which they might have also wanted to avoid. Even though it is rarely an issue at LAN parties, it might have simply looked bad for the brand.
  - monocasa 3y ago
    From what I've heard from microsofties over beers, it was really the second one. They didn't expect developers to write network code in ~2001 that didn't have memory safety vulnerabilities and wanted a barrier there between simple overflows and system security.
    - mywittyname 3y ago
      Makes total sense, especially recalling the zeitgeist of the era. IIRC, this was during the height of worms which propagated via vulnerabilities in code that handles network traffic.
    - willis936 3y ago
      The child in me from 20 years ago dreams of a software tool downloaded from a questionable website that jailbroke an xbox just by putting in the IP address.
      - monocasa 3y ago
        I hear that; I wish I was cool enough back in the day to release a tool on xbins. I'll have to settle for retro computing.
- rasz 3y ago
  There are cheats for certain online games that just listen to traffic. Sometimes games use bad encryption (static key) and cheats decrypt on the fly.
loeg 3y ago
"age in a bot" might be a truncated form of "message in a bottle." The PRNG exponentiation scheme is essentially Diffie-Hellman. > Modifying the fire duration does not seem to have any effect Including, e.g., plasma pistol? Very cool investigation and writeup.
- nhellman 3y ago
  > "age in a bot" might be a truncated form of "message in a bottle." Yeah, that makes sense, it is a very odd sentence otherwise. Truncating on "bot" might be a play on words for game AI/NPC. > The PRNG exponentiation scheme is essentially Diffie-Hellman. Ah, of course, yes, I was thinking it reminded of public-key cryptography. > Including, e.g., plasma pistol? Yeah, it seems so, that was the first weapon I was thinking it to be used for. I tried e.g. setting the duration to zero and charging the plasma pistol. It still shoots like normal and does not seem to affect visual effects, audio or damage for neither host nor guest. It is still possible that it is used for something else that I have not noticed. Modifying the host's fire duration does not cause a desync so it might not affect the game state. > Very cool investigation and writeup. Thanks!
  - coldpie 3y ago
    > Yeah, that makes sense, it is a very odd sentence otherwise. Truncating on "bot" might be a play on words for game AI/NPC. I wonder if you could dig into the game code and see where the string is coming from.
- monocasa 3y ago
  Yeah, it's straight up diffe-hellman, even using the First Oakley Default Group as it's prime. https://github.com/XombieOnline/xombie/blob/7a1ef08045271437efc5938c996cb0c209a31408/libs/xblive/src/crypto/primitives.rs#L66 https://github.com/XombieOnline/xombie/blob/7a1ef08045271437...
- CYR1X 3y ago
  People have determined what the player structs are on the xbox-memory-state side of things, so my guess is "age_in_a_bot" might be the time elapsed for a given player aka how long they've been alive: https://github.com/CYRiXplaysHalo/XboxHaloGameLogger/blob/master/memory_mappings_and_offsets.py#L45 https://github.com/CYRiXplaysHalo/XboxHaloGameLogger/blob/ma...
cglong 3y ago
It's a shame it's proprietary, but there's a neat service[1] that's been around for over a decade that reverse engineered System Link and similar protocols to make them Internet-enabled. [1]: https://www.teamxlink.co.uk/ https://www.teamxlink.co.uk/
- nhellman 3y ago
  I haven't tried this out, I wonder how well the Halo 1 net protocol works over a less reliable connection. I encounter issues as soon as I momentarily minimize the emulator and it does not recover without starting a new game. I wonder if it is enough for XLink to simply send the packets through a networked tunnel or if it actually needs to modify the packet payload somehow. The consoles might be able to handle everything as long as they are able to communicate with each other?
  - paulryanrogers 3y ago
    My friends tried it back in the day and it seemed pretty laggy, like 100ms, even with broadband cable. Hosts definitely had the advantage.
  - bradfox2 3y ago
    I played competitive HaloCE through 3 at a professional level. XBConnect is what people playing competitive CE used prior to halo 2 and xbox live. Game was playable up to about 60ms, with <40 greatly preferred. Hosts had a large advantage, but off host was very predictable, and required leading the targets. The same advantage existed on LAN, with competitive tournaments setup to trade host advantage for team color advantage. The last in person CE lan I attended, we were playing on modded Xboxes with a patched version of the game that allowed a third box as a neutral host, and added an on screen timer. Happy to expand on the details if anyone is interested.
    - k8svet 3y ago
      Maybe you're aware but... https://halo1nhe.com/ https://halo1nhe.com/ Shout out the {Halo 1 Hub, xemu, Halo Modding Reclaimers, Invaer} Discord members. I'm sure some of you in this thread are already there. :D
monocasa 3y ago
So that's basically most of the lowest layers of the Xbox Live protocol as well. Simply game discovery happens over local broadcast instead of through the Live servers. There's some other specifics that change, such as logging into a host xbox doesn't diffie-hellman anymore but instead key sets are distributed by the server backends with the session information. Additionally the auth side is basically non existent on system link. Great RE work! Disclaimer: I created a proof of concept implementation of the Xbox Live server infrastructure here: https://github.com/xombieonline/xombie https://github.com/xombieonline/xombie
- nhellman 3y ago
  > So that's basically most of the lowest layers of the Xbox Live protocol as well. Simply game discovery happens over local broadcast instead of through the Live servers. There's some other specifics that change, such as logging into a host xbox doesn't diffie-hellman anymore but instead key sets are distributed by the server backends with the session information. Additionally the auth side is basically non existent on system link. Ah, cool, there has been some similar work done before! Is there any documented resources or write-ups of the Xbox Live protocol somewhere? > Great RE work! Thanks! > Disclaimer: I created a proof of concept implementation of the Xbox Live server infrastructure here: https://github.com/xombieonline https://github.com/xombieonline That is a really cool project! How does one make sure the console connects to the server instead of the (now shutdown) Xbox Live servers? Does it e.g. have a hardcoded domain so one can simply add a DNS entry?
  - monocasa 3y ago
    > Ah, cool, there has been some similar work done before! Is there any documented resources or write-ups of the Xbox Live protocol somewhere? I talked at defcon a bit last year. https://www.youtube.com/watch?v=HLyZfZMu-5E https://www.youtube.com/watch?v=HLyZfZMu-5E Otherwise I've been a "source is documentation" kind of guy so far. Could definitely use a high level walkthrough as good as you've done here; I should probably get on that. > That is a really cool project! How does one make sure the console connects to the server instead of the (now shutdown) Xbox Live servers? Does it e.g. have a hardcoded domain so one can simply add a DNS entry? Yeah, they bootstrap off of a set of hardcoded domains. The config screens on the boxes let you specify a DNS server, so the project runs it's own DNS server that's basically when obi-wan says "of course I know him, he's me". Then the relative lack of any public/private crypto lets us take over as long as we know the preshared key in the individual xbox's eeprom.
Sxubas 3y ago
Loved it, thanks for being so thorough and document the train of thought. I want to ask about one thing I could not understand completely on the final section: If there was a client that sent arbitrary values for selected weapon, forward, left, etc; would the host count them as valid? (I understood this is essentially what the MITM allowed to do) Also, a little feedback, my immersion broke when the video did not show Howard and Ghost anymore. Something like Howard1 and Ghost1 would've helped understand a little bit more.
- nhellman 3y ago
  > Loved it, thanks for being so thorough and document the train of thought. Glad to hear! I was hoping to make the whole process easy to follow without any large skips or prior knowledge. It is more or less my process but with a lot of dead ends removed. > I want to ask about one thing I could not understand completely on the final section: If there was a client that sent arbitrary values for selected weapon, forward, left, etc; would the host count them as valid? (I understood this is essentially what the MITM allowed to do) Yes, it appears so. The host seems to accept more or less arbitrary values (at least for forward, left, actions) and also just re-broadcasts them to everyone. The guest client then also ignores what it sent and just uses what it receives from the host. This allows us to modify the inputs/speed of any guest client players without desyncing the clients. > Also, a little feedback, my immersion broke when the video did not show Howard and Ghost anymore. Something like Howard1 and Ghost1 would've helped understand a little bit more. Good idea, I could have made it clearer which perspective the video was taken from (the guest client). Might be able to re-record or simply mention it in the text.
dinartem 3y ago
Nice work! Always fun to see something I wrote long ago reverse engineered. The packet format was indeed inspired by ESP over UDP, and I named it XSP. After system link shipped with the original launch of the console, I also worked on Xbox Live networking, including the client/server interactions and the design and implementation of the front-end Security Gateways that all Xboxes would talk to, first to authenticate themselves to the service, and then to maintain a heartbeat connection to the service (to keep NAT ports open during idle time), and to facilitate NAT traversal.
- monocasa 3y ago
  Nice! You did a great job on the protocol. Probably my only complaint on the XSP side of things is the fact that you have to do relatively complex parsing of the XSP packets before you can get to the point of verifying the signature of the packet. Seems like all of the corner cases were handled well in the implementation on the boxes, but as someone who does auth/cryptography in my day job, it kind of gives me the heebie-jeebies. Do you know if the auth side was carried into deeper parts of the backend? So like, did the SG decorate incoming connections with the auth information as they made their way to the different services? There seemed to be more auth information than I expected in headers on some of those HTTP calls into services like matchmaking.
  - dinartem 3y ago
    That's a valid point about complex parsing. I remember being very concerned about adding unnecessary overhead to each packet during encapsulation. As for the SG, it primarily authenticated the Xbox machine account using Kerberos and then maintained a security association, accepted heartbeats, authenticated and decrypted incoming ESP-UDP packets into IP packets that it forwarded to the backend servers. Responses from the backend would be encrypted, authenticated, and encapsulated before sending back to the Xbox. I don't think the SG had any knowledge of higher level connections running through it, such as TCP or HTTP, so it would not have manipulated HTTP headers as they passed through.
    - monocasa 3y ago
nmstoker 3y ago
I recall the game recording feature feeling pretty cutting edge at the time, as you could pause it and rotate/fly round the scene, it felt really fluid and responsive. I guess now it would be common place but back then it definitely seemed a little magical.
- nhellman 3y ago
  Are you thinking of Theater mode that first appeared in Halo 3? Unfortunately it was slimmed down in later Halo games and it is rarely seen in other games at all, especially not console games.
- CYR1X 3y ago
  This was common in PC games at the time, but not on console games. Essentially you're just dumping game state memory somewhere and then replaying it. Half-life games had it.
k8svet 3y ago
Halo 1 + system link + XBConnect is responsible for a huge swath of my career, life, friendships. I remember sitting in my parents basement as a [very young person] marveling at XBC and wondering how it worked. I can still remember the layout of the webpage I found diving into the protocol, how discovery worked, broadcasting discovery packets to 0.0.0.1. I remember opening Ethereal and seeing the packets. I remember, a bit later, hacking together a Java app, failing at a C++ app, many years later trying one in Go and then... pre-drop-@-gc Rust. Wonder if instead of encapsulating packets with src/dst mac I could send a map back and forth and index into it to save the whole... 6 bytes? So many hours of my life spent playing that game. Even recently I re-discovered an old Halo (x)ISO I mastered in high school containing a multitude of map packs that the community had made for it (NMP, NMPv2, CXE, +??). I even hacked them to change their internal map IDs to prevent cache conflicts when switching packs. My friend dug it out of his collection, copied the ISO and I fired it up in XEMU. Wild to see some of those maps, that some random people made with hacked together tools, and wild to read this now, and the comment from /u/dinartem. Even wilder that it's playable emulatable now. Especially given the way MCC massacred Halo1 with the horrendous Halo 1 PC port back to Xbox, and then later to PC again. I'm horrified to see someone comment that multiplayer almost didn't launch with Halo 1. My life would be so unimaginably different. So many memories, this comment doesn't mean much, but what a thing to see on HN. lol, I'm almost tempted to drop my XBConnect Forum name here. I remember when I thought Todd was an absolute god among humans. Oh man, thank you HN for the dose of nostalgia. If anyone remembers a huge block-letter forum signature that was briefly animated ;). The era of sprawling PHP file upload sites. Wow. The internet before it became truly cursed. edit: shout out if anyone knows what I mean by "clear walls". Oh man, what a world. editN: oh wow, "cross over cables" is a phrase I haven't thought of in a long time. editLast: there was a glitch that was supposedly reproducible that caused a tertiary console's player to override the inputs of another console player. Afaik it was never widely discussed, despite repeated claims that it was reproducible on demand. If anyone has any details, you'd make this a truly magical thread for me. <3.
- Mintograde 3y ago
  For your last edit, I've only seen that glitch twice: once in a youtube video [1], and once in one of my own games on XBConnect. I heard similar unsubstantiated claims of intentionally reproducing the glitch, but didn't dig too far into it myself. [1]: https://www.youtube.com/watch?v=TEkmqKYhxnU https://www.youtube.com/watch?v=TEkmqKYhxnU
burnsaga 3y ago
Huge fan of Halo 1, and for me, the original (via LAN, XLAN or Xlink Kai) is still the preferred way to play compared to Halo MCC (RIP XBconnect). I dabble in map mods so this is mostly over my head, but interesting read. Don't know if would help your research: but are you aware of a mod/mappack called Halo 1: NHE (No Host Edition)? It's a hacky way of using a third box to host the system link game. http://halo1nhe.com/ http://halo1nhe.com/
- k8svet 3y ago
  4 year old account, and this is your first comment? :D You might be interested in super custom xISO with NMP, NMPv2, CXE, etc. I mentioned it in another comment. I'm going to upload to Internet Archive soon. This is officially my favorite thread on HN, ever. Just waiting for rothgar to show up, sorry @rothgar, I should've hit you up for some Halo while I was in Seattle (and the k8s space).
- nhellman 3y ago
  NHE looks interesting, they seem to have modified all clients to make the host player teleport above the map. With the current protocol, it seems like it is impossible to achieve this without modifying the guest clients as well. And by modifying all clients they are also able to add extra maps and make gameplay changes.

220 points by nhellman 3y ago

Tangential point: the weirdest part about the Xbox was the system name. As far as I'm aware that was never configurable by users, but certain games would set your system name as an easter egg. Any reason why it was built that way?

reidjs 3y ago

This is a well written, in depth analysis of the system. Why do you think the packets are encrypted?

PawBer 3y ago
The LAN code is probably the same code as the online multiplayer.
- ClassyJacket 3y ago
  Halo 1 on original Xbox does not have online multiplayer. It launched before Xbox Live did.
  - msk-lywenn 3y ago
    They were probably already working on it
    - rafark 3y ago
      The game was actually close to ship without multiplayer, it was an afterthought. Some devs have talked about this (iirc, Marty O’Donnell was one of them).
      - notnaut 3y ago
        I’ve heard this was also the case for Goldeneye on the N64
        ravenstine 3y ago
        Goldeneye is one of the best case studies for why companies should give their engineers room to experiment. If I remember correctly, the multiplayer mode was originally a side project of one of the engineers and was added near the end of development. This isn't to say the campaign mode isn't good, but the ability for boys to play a shooter game against each other on the same console is what made that game.
      - thomastjeffery 3y ago
        The game being an FPS was an afterthought, too. Originally, they were working on an RTS.
        monocasa 3y ago
        I wouldn't call it an afterthought per se, more a pivot with a lot of work. It was already an FPS when it was shown off at MacWorld'99.
  - monocasa 3y ago
    As someone who's written replacement backend Xbox Live server infra, I can confirm that system link is very nearly the same base protocol set as is used by Live. They must have had that nailed down pretty close to launch, and were mainly working on support services that live needs but system link doesn't up until Live actually launched.
zootboy 3y ago
Probably for the reason the author speculates about in the article: > For example, would it not be cool if there was a game with a stack buffer overflow bug that allowed us to run arbitrary code remotely? That could potentially enable us to softmod an Xbox over the network, without the need of any special hardware.
nhellman 3y ago
Thanks, I was initially surprised that they were encrypted but I can speculate on a few potential reasons: - The Xbox was designed to be able to play online via Xbox Live (e.g. with Halo 2 that came out later in 2004), and they might have simply reused the network stack for System Link over LAN. I looked a little bit at Halo 2 system link, it uses the same system calls from the kernel but the protocol seems to be more complicated (e.g. the IV is never sent in plaintext, the two consoles derive it somehow). I haven't looked at Xbox Live, but Halo 2 could potentially use the same for System Link and Xbox Live. - The Xbox also really tried to lock down its security in order to prevent game piracy and homebrew games. This might have been another attempt to reduce the attack surface. - They also might have just wanted to keep their game protocols secret for trade secrets or simply avoiding scrutiny. In this case, we were able to create a kind of cheat, which they might have also wanted to avoid. Even though it is rarely an issue at LAN parties, it might have simply looked bad for the brand.
- monocasa 3y ago
  From what I've heard from microsofties over beers, it was really the second one. They didn't expect developers to write network code in ~2001 that didn't have memory safety vulnerabilities and wanted a barrier there between simple overflows and system security.
  - mywittyname 3y ago
    Makes total sense, especially recalling the zeitgeist of the era. IIRC, this was during the height of worms which propagated via vulnerabilities in code that handles network traffic.
  - willis936 3y ago
    The child in me from 20 years ago dreams of a software tool downloaded from a questionable website that jailbroke an xbox just by putting in the IP address.
    - monocasa 3y ago
      I hear that; I wish I was cool enough back in the day to release a tool on xbins. I'll have to settle for retro computing.
rasz 3y ago
There are cheats for certain online games that just listen to traffic. Sometimes games use bad encryption (static key) and cheats decrypt on the fly.

loeg 3y ago

"age in a bot" might be a truncated form of "message in a bottle." The PRNG exponentiation scheme is essentially Diffie-Hellman. > Modifying the fire duration does not seem to have any effect Including, e.g., plasma pistol? Very cool investigation and writeup.

nhellman 3y ago
> "age in a bot" might be a truncated form of "message in a bottle." Yeah, that makes sense, it is a very odd sentence otherwise. Truncating on "bot" might be a play on words for game AI/NPC. > The PRNG exponentiation scheme is essentially Diffie-Hellman. Ah, of course, yes, I was thinking it reminded of public-key cryptography. > Including, e.g., plasma pistol? Yeah, it seems so, that was the first weapon I was thinking it to be used for. I tried e.g. setting the duration to zero and charging the plasma pistol. It still shoots like normal and does not seem to affect visual effects, audio or damage for neither host nor guest. It is still possible that it is used for something else that I have not noticed. Modifying the host's fire duration does not cause a desync so it might not affect the game state. > Very cool investigation and writeup. Thanks!
- coldpie 3y ago
  > Yeah, that makes sense, it is a very odd sentence otherwise. Truncating on "bot" might be a play on words for game AI/NPC. I wonder if you could dig into the game code and see where the string is coming from.
monocasa 3y ago
Yeah, it's straight up diffe-hellman, even using the First Oakley Default Group as it's prime. https://github.com/XombieOnline/xombie/blob/7a1ef08045271437efc5938c996cb0c209a31408/libs/xblive/src/crypto/primitives.rs#L66 https://github.com/XombieOnline/xombie/blob/7a1ef08045271437...
CYR1X 3y ago
People have determined what the player structs are on the xbox-memory-state side of things, so my guess is "age_in_a_bot" might be the time elapsed for a given player aka how long they've been alive: https://github.com/CYRiXplaysHalo/XboxHaloGameLogger/blob/master/memory_mappings_and_offsets.py#L45 https://github.com/CYRiXplaysHalo/XboxHaloGameLogger/blob/ma...

cglong 3y ago

It's a shame it's proprietary, but there's a neat service[1] that's been around for over a decade that reverse engineered System Link and similar protocols to make them Internet-enabled. [1]: https://www.teamxlink.co.uk/ https://www.teamxlink.co.uk/

nhellman 3y ago
I haven't tried this out, I wonder how well the Halo 1 net protocol works over a less reliable connection. I encounter issues as soon as I momentarily minimize the emulator and it does not recover without starting a new game. I wonder if it is enough for XLink to simply send the packets through a networked tunnel or if it actually needs to modify the packet payload somehow. The consoles might be able to handle everything as long as they are able to communicate with each other?
- paulryanrogers 3y ago
  My friends tried it back in the day and it seemed pretty laggy, like 100ms, even with broadband cable. Hosts definitely had the advantage.
- bradfox2 3y ago
  I played competitive HaloCE through 3 at a professional level. XBConnect is what people playing competitive CE used prior to halo 2 and xbox live. Game was playable up to about 60ms, with <40 greatly preferred. Hosts had a large advantage, but off host was very predictable, and required leading the targets. The same advantage existed on LAN, with competitive tournaments setup to trade host advantage for team color advantage. The last in person CE lan I attended, we were playing on modded Xboxes with a patched version of the game that allowed a third box as a neutral host, and added an on screen timer. Happy to expand on the details if anyone is interested.
  - k8svet 3y ago
    Maybe you're aware but... https://halo1nhe.com/ https://halo1nhe.com/ Shout out the {Halo 1 Hub, xemu, Halo Modding Reclaimers, Invaer} Discord members. I'm sure some of you in this thread are already there. :D

monocasa 3y ago

So that's basically most of the lowest layers of the Xbox Live protocol as well. Simply game discovery happens over local broadcast instead of through the Live servers. There's some other specifics that change, such as logging into a host xbox doesn't diffie-hellman anymore but instead key sets are distributed by the server backends with the session information. Additionally the auth side is basically non existent on system link. Great RE work! Disclaimer: I created a proof of concept implementation of the Xbox Live server infrastructure here: https://github.com/xombieonline/xombie https://github.com/xombieonline/xombie

nhellman 3y ago
> So that's basically most of the lowest layers of the Xbox Live protocol as well. Simply game discovery happens over local broadcast instead of through the Live servers. There's some other specifics that change, such as logging into a host xbox doesn't diffie-hellman anymore but instead key sets are distributed by the server backends with the session information. Additionally the auth side is basically non existent on system link. Ah, cool, there has been some similar work done before! Is there any documented resources or write-ups of the Xbox Live protocol somewhere? > Great RE work! Thanks! > Disclaimer: I created a proof of concept implementation of the Xbox Live server infrastructure here: https://github.com/xombieonline https://github.com/xombieonline That is a really cool project! How does one make sure the console connects to the server instead of the (now shutdown) Xbox Live servers? Does it e.g. have a hardcoded domain so one can simply add a DNS entry?
- monocasa 3y ago
  > Ah, cool, there has been some similar work done before! Is there any documented resources or write-ups of the Xbox Live protocol somewhere? I talked at defcon a bit last year. https://www.youtube.com/watch?v=HLyZfZMu-5E https://www.youtube.com/watch?v=HLyZfZMu-5E Otherwise I've been a "source is documentation" kind of guy so far. Could definitely use a high level walkthrough as good as you've done here; I should probably get on that. > That is a really cool project! How does one make sure the console connects to the server instead of the (now shutdown) Xbox Live servers? Does it e.g. have a hardcoded domain so one can simply add a DNS entry? Yeah, they bootstrap off of a set of hardcoded domains. The config screens on the boxes let you specify a DNS server, so the project runs it's own DNS server that's basically when obi-wan says "of course I know him, he's me". Then the relative lack of any public/private crypto lets us take over as long as we know the preshared key in the individual xbox's eeprom.

Sxubas 3y ago

Loved it, thanks for being so thorough and document the train of thought. I want to ask about one thing I could not understand completely on the final section: If there was a client that sent arbitrary values for selected weapon, forward, left, etc; would the host count them as valid? (I understood this is essentially what the MITM allowed to do) Also, a little feedback, my immersion broke when the video did not show Howard and Ghost anymore. Something like Howard1 and Ghost1 would've helped understand a little bit more.

nhellman 3y ago
> Loved it, thanks for being so thorough and document the train of thought. Glad to hear! I was hoping to make the whole process easy to follow without any large skips or prior knowledge. It is more or less my process but with a lot of dead ends removed. > I want to ask about one thing I could not understand completely on the final section: If there was a client that sent arbitrary values for selected weapon, forward, left, etc; would the host count them as valid? (I understood this is essentially what the MITM allowed to do) Yes, it appears so. The host seems to accept more or less arbitrary values (at least for forward, left, actions) and also just re-broadcasts them to everyone. The guest client then also ignores what it sent and just uses what it receives from the host. This allows us to modify the inputs/speed of any guest client players without desyncing the clients. > Also, a little feedback, my immersion broke when the video did not show Howard and Ghost anymore. Something like Howard1 and Ghost1 would've helped understand a little bit more. Good idea, I could have made it clearer which perspective the video was taken from (the guest client). Might be able to re-record or simply mention it in the text.

dinartem 3y ago

Nice work! Always fun to see something I wrote long ago reverse engineered. The packet format was indeed inspired by ESP over UDP, and I named it XSP. After system link shipped with the original launch of the console, I also worked on Xbox Live networking, including the client/server interactions and the design and implementation of the front-end Security Gateways that all Xboxes would talk to, first to authenticate themselves to the service, and then to maintain a heartbeat connection to the service (to keep NAT ports open during idle time), and to facilitate NAT traversal.

monocasa 3y ago
Nice! You did a great job on the protocol. Probably my only complaint on the XSP side of things is the fact that you have to do relatively complex parsing of the XSP packets before you can get to the point of verifying the signature of the packet. Seems like all of the corner cases were handled well in the implementation on the boxes, but as someone who does auth/cryptography in my day job, it kind of gives me the heebie-jeebies. Do you know if the auth side was carried into deeper parts of the backend? So like, did the SG decorate incoming connections with the auth information as they made their way to the different services? There seemed to be more auth information than I expected in headers on some of those HTTP calls into services like matchmaking.
- dinartem 3y ago
  That's a valid point about complex parsing. I remember being very concerned about adding unnecessary overhead to each packet during encapsulation. As for the SG, it primarily authenticated the Xbox machine account using Kerberos and then maintained a security association, accepted heartbeats, authenticated and decrypted incoming ESP-UDP packets into IP packets that it forwarded to the backend servers. Responses from the backend would be encrypted, authenticated, and encapsulated before sending back to the Xbox. I don't think the SG had any knowledge of higher level connections running through it, such as TCP or HTTP, so it would not have manipulated HTTP headers as they passed through.
  - monocasa 3y ago

nmstoker 3y ago

I recall the game recording feature feeling pretty cutting edge at the time, as you could pause it and rotate/fly round the scene, it felt really fluid and responsive. I guess now it would be common place but back then it definitely seemed a little magical.

nhellman 3y ago
Are you thinking of Theater mode that first appeared in Halo 3? Unfortunately it was slimmed down in later Halo games and it is rarely seen in other games at all, especially not console games.
CYR1X 3y ago
This was common in PC games at the time, but not on console games. Essentially you're just dumping game state memory somewhere and then replaying it. Half-life games had it.

k8svet 3y ago

Halo 1 + system link + XBConnect is responsible for a huge swath of my career, life, friendships. I remember sitting in my parents basement as a [very young person] marveling at XBC and wondering how it worked. I can still remember the layout of the webpage I found diving into the protocol, how discovery worked, broadcasting discovery packets to 0.0.0.1. I remember opening Ethereal and seeing the packets. I remember, a bit later, hacking together a Java app, failing at a C++ app, many years later trying one in Go and then... pre-drop-@-gc Rust. Wonder if instead of encapsulating packets with src/dst mac I could send a map back and forth and index into it to save the whole... 6 bytes? So many hours of my life spent playing that game. Even recently I re-discovered an old Halo (x)ISO I mastered in high school containing a multitude of map packs that the community had made for it (NMP, NMPv2, CXE, +??). I even hacked them to change their internal map IDs to prevent cache conflicts when switching packs. My friend dug it out of his collection, copied the ISO and I fired it up in XEMU. Wild to see some of those maps, that some random people made with hacked together tools, and wild to read this now, and the comment from /u/dinartem. Even wilder that it's playable emulatable now. Especially given the way MCC massacred Halo1 with the horrendous Halo 1 PC port back to Xbox, and then later to PC again. I'm horrified to see someone comment that multiplayer almost didn't launch with Halo 1. My life would be so unimaginably different. So many memories, this comment doesn't mean much, but what a thing to see on HN. lol, I'm almost tempted to drop my XBConnect Forum name here. I remember when I thought Todd was an absolute god among humans. Oh man, thank you HN for the dose of nostalgia. If anyone remembers a huge block-letter forum signature that was briefly animated ;). The era of sprawling PHP file upload sites. Wow. The internet before it became truly cursed. edit: shout out if anyone knows what I mean by "clear walls". Oh man, what a world. editN: oh wow, "cross over cables" is a phrase I haven't thought of in a long time. editLast: there was a glitch that was supposedly reproducible that caused a tertiary console's player to override the inputs of another console player. Afaik it was never widely discussed, despite repeated claims that it was reproducible on demand. If anyone has any details, you'd make this a truly magical thread for me. <3.

Mintograde 3y ago
For your last edit, I've only seen that glitch twice: once in a youtube video [1], and once in one of my own games on XBConnect. I heard similar unsubstantiated claims of intentionally reproducing the glitch, but didn't dig too far into it myself. [1]: https://www.youtube.com/watch?v=TEkmqKYhxnU https://www.youtube.com/watch?v=TEkmqKYhxnU

burnsaga 3y ago

Huge fan of Halo 1, and for me, the original (via LAN, XLAN or Xlink Kai) is still the preferred way to play compared to Halo MCC (RIP XBconnect). I dabble in map mods so this is mostly over my head, but interesting read. Don't know if would help your research: but are you aware of a mod/mappack called Halo 1: NHE (No Host Edition)? It's a hacky way of using a third box to host the system link game. http://halo1nhe.com/ http://halo1nhe.com/

k8svet 3y ago
4 year old account, and this is your first comment? :D You might be interested in super custom xISO with NMP, NMPv2, CXE, etc. I mentioned it in another comment. I'm going to upload to Internet Archive soon. This is officially my favorite thread on HN, ever. Just waiting for rothgar to show up, sorry @rothgar, I should've hit you up for some Halo while I was in Seattle (and the k8s space).
nhellman 3y ago
NHE looks interesting, they seem to have modified all clients to make the host player teleport above the map. With the current protocol, it seems like it is impossible to achieve this without modifying the guest clients as well. And by modifying all clients they are also able to add extra maps and make gameplay changes.