14 ms·

“ Cut-through vs Store-and-Forward” I had a Google interview collapse because the last interviewer was a Principle Engineer who just couldn’t accept that a swi

by noncoml 2mo ago

“ Cut-through vs Store-and-Forward” I had a Google interview collapse because the last interviewer was a Principle Engineer who just couldn’t accept that a switch will ever forward a frame before reading in full and checking the FCS. I was too much of an idiot to go along with it and instead ended up having an argument with him.

random__duck 2mo ago
Sounds like they could have used you on that team. Bad luck, but it at least you can walk away proud knowing it was not an issue with your technical skills. :)
inigyou 2mo ago
Cut-through is applicable in a minority of scenarios which are sometimes common. The majority of switches, especially the ones that support any advanced features at all, can't use it. Even when you can use it, you still can't use it all the time because of port conflicts. As far as advanced features go, even adding and removing VLAN tags is a headache in a cut-through switch.
- userbinator 2mo ago
  As far as advanced features go, even adding and removing VLAN tags is a headache in a cut-through switch. Not really; it just needs to inject the tag right after the source address, or not pass it through at all; and of course recompute throughout and replace the FCS at the end. Other more advanced packet editing can be done in a streaming fashion too, with the latency only limited by the length of any "forward references" needed.
  - crote 2mo ago
    But "just recompute the FCS" isn't quite the right approach, is it? With a traditional cut-through switch the entire packet is passed as-is, so you're not modifying a corrupt package. You don't drop it, but the corruption is trivially detectable on the other side. If you naively recompute the FCS on transmit you essentially mark a corrupted package as valid, so you have to compute the FCS on transmit and receive, and intentionally send out the wrong FCS on transmit if you determine the receive FCS is invalid.
    - bcrl 2mo ago
      It's easy enough to write a bit of Verilog to perform partial updates on the CRC to only compensate for what has changed. I did so for an FPGA project I implemented a number of years ago when processing VLAN tags and doing IP forwarding. That said, there are lazy hardware designers that just slap on a block to recompute the CRC and replace the old one with the new. An experience I had years ago at Red Hat taught me that the hard way as a shiny new Cisco ethernet switch IT deployed ended up costing us weeks of frustration and delayed a release when it caused bit flips in NFS packets during stress testing. The switch was so awesomely advanced that it recomputed the CRC on the bit flipped packets causing kernel builds to fail. <sigh>
      - foobiekr 2mo ago
        Wow that rings a bell. The lack of end to end ECC protection in some packet paths on very high end ASICs is something I just can’t get over.
        bcrl 2mo ago
        ECC tends to be too expensive to be used widely in hardware data paths. Most of the time hardware will use a parity bit as checking it is far less costly (both in terms of gates and power). Parity bits are usually sufficient, as the purpose is really limited to identifying hardware failure. The CRC should prevent delivery of a corrupted packet provided the hardware doesn't cheat and blindly recompute it.
        foobiekr 1mo ago
        ECC is _not_ too expensive to use in hardware data paths. The existence proof of this is almost every medium to high end NPU has end to end ECC.
    - userbinator 2mo ago
      so you have to compute the FCS on transmit and receive, and intentionally send out the wrong FCS on transmit if you determine the receive FCS is invalid. Yes, exactly; but that isn't particularly difficult either, compared to all the other packet-editing capabilities that switch ASICs have.
sophacles 2mo ago
I'm struggling to think of any real-world networking scenario where cut-through switching for ethernet can provide any sort of benefit. Most data-center networks are involve aggregation/spine/core switches at higher speed than TOR/leaf switches - so the packet has to be buffered. Most every workload is going to involve port collisions (2 packets at the same time needing the same output port) - again a full buffering required (or drops and throughput reduction). Bigger networks start using vlans, mpls, ip routing at switches, and vxlan - requiring more lookups per-packet, larger "minimum chunks of packet seen before port is determined" sizes, etc reducing the value of cut-through. And so on. But i also struggle to find a scenario where a principle engineer at google is refusing to accept that cut-through switches exist, but rather, was refusing to accept that the scenario they're talking through with you would ever involve an effectively deployed cut-through switch (since that's basically never).
- noncoml 2mo ago
  > but rather, was refusing to accept that the scenario they're talking through with you would ever involve an effectively deployed cut-through switch (since that's basically never). Nope, the argument was exactly if a cut-through switch exists or ever existed Edit: Actually the conversation came back to me: He asked me about the frame format. Once I drew that on the board, he asked why is the DST before the SRC. My answer was in order for the switch to start doing the lookups ASAP and that's when I mentioned that a cut-through switch can switch the packet before it even finishes receiving it. After that we spend the rest of the interview time arguing if cut-through switches actually exist or not. Both he and I having too big of an ego to let it go..
  - icedchai 2mo ago
    So he didn't want to google it?
    - stinkbeetle 2mo ago
      I see you've never had a run-in with a blowhard in a position of power. Lucky.
gjvc 2mo ago
"principal"
Aurornis 2mo ago
I remember studying cut-through forwarding in a class and thinking it was obviously going to be used everywhere. Then I was disappointed to enter the real world of networking and learn that it wasn’t actually used as often as I had been led to believe due to conflicting with higher priority features. So the interview may have been a conflict between textbook learning and the Google engineer’s practical experience. It’s tough to encounter that conflict in interviews because you get two people talking past each other.

by noncoml 2mo ago

random__duck 2mo ago

Sounds like they could have used you on that team. Bad luck, but it at least you can walk away proud knowing it was not an issue with your technical skills. :)

inigyou 2mo ago

Cut-through is applicable in a minority of scenarios which are sometimes common. The majority of switches, especially the ones that support any advanced features at all, can't use it. Even when you can use it, you still can't use it all the time because of port conflicts. As far as advanced features go, even adding and removing VLAN tags is a headache in a cut-through switch.

userbinator 2mo ago
As far as advanced features go, even adding and removing VLAN tags is a headache in a cut-through switch. Not really; it just needs to inject the tag right after the source address, or not pass it through at all; and of course recompute throughout and replace the FCS at the end. Other more advanced packet editing can be done in a streaming fashion too, with the latency only limited by the length of any "forward references" needed.
- crote 2mo ago
  But "just recompute the FCS" isn't quite the right approach, is it? With a traditional cut-through switch the entire packet is passed as-is, so you're not modifying a corrupt package. You don't drop it, but the corruption is trivially detectable on the other side. If you naively recompute the FCS on transmit you essentially mark a corrupted package as valid, so you have to compute the FCS on transmit and receive, and intentionally send out the wrong FCS on transmit if you determine the receive FCS is invalid.
  - bcrl 2mo ago
    It's easy enough to write a bit of Verilog to perform partial updates on the CRC to only compensate for what has changed. I did so for an FPGA project I implemented a number of years ago when processing VLAN tags and doing IP forwarding. That said, there are lazy hardware designers that just slap on a block to recompute the CRC and replace the old one with the new. An experience I had years ago at Red Hat taught me that the hard way as a shiny new Cisco ethernet switch IT deployed ended up costing us weeks of frustration and delayed a release when it caused bit flips in NFS packets during stress testing. The switch was so awesomely advanced that it recomputed the CRC on the bit flipped packets causing kernel builds to fail. <sigh>
    - foobiekr 2mo ago
      Wow that rings a bell. The lack of end to end ECC protection in some packet paths on very high end ASICs is something I just can’t get over.
      - bcrl 2mo ago
        ECC tends to be too expensive to be used widely in hardware data paths. Most of the time hardware will use a parity bit as checking it is far less costly (both in terms of gates and power). Parity bits are usually sufficient, as the purpose is really limited to identifying hardware failure. The CRC should prevent delivery of a corrupted packet provided the hardware doesn't cheat and blindly recompute it.
        foobiekr 1mo ago
        ECC is _not_ too expensive to use in hardware data paths. The existence proof of this is almost every medium to high end NPU has end to end ECC.
  - userbinator 2mo ago
    so you have to compute the FCS on transmit and receive, and intentionally send out the wrong FCS on transmit if you determine the receive FCS is invalid. Yes, exactly; but that isn't particularly difficult either, compared to all the other packet-editing capabilities that switch ASICs have.

sophacles 2mo ago

I'm struggling to think of any real-world networking scenario where cut-through switching for ethernet can provide any sort of benefit. Most data-center networks are involve aggregation/spine/core switches at higher speed than TOR/leaf switches - so the packet has to be buffered. Most every workload is going to involve port collisions (2 packets at the same time needing the same output port) - again a full buffering required (or drops and throughput reduction). Bigger networks start using vlans, mpls, ip routing at switches, and vxlan - requiring more lookups per-packet, larger "minimum chunks of packet seen before port is determined" sizes, etc reducing the value of cut-through. And so on. But i also struggle to find a scenario where a principle engineer at google is refusing to accept that cut-through switches exist, but rather, was refusing to accept that the scenario they're talking through with you would ever involve an effectively deployed cut-through switch (since that's basically never).

noncoml 2mo ago
> but rather, was refusing to accept that the scenario they're talking through with you would ever involve an effectively deployed cut-through switch (since that's basically never). Nope, the argument was exactly if a cut-through switch exists or ever existed Edit: Actually the conversation came back to me: He asked me about the frame format. Once I drew that on the board, he asked why is the DST before the SRC. My answer was in order for the switch to start doing the lookups ASAP and that's when I mentioned that a cut-through switch can switch the packet before it even finishes receiving it. After that we spend the rest of the interview time arguing if cut-through switches actually exist or not. Both he and I having too big of an ego to let it go..
- icedchai 2mo ago
  So he didn't want to google it?
  - stinkbeetle 2mo ago
    I see you've never had a run-in with a blowhard in a position of power. Lucky.

gjvc 2mo ago

"principal"

Aurornis 2mo ago

I remember studying cut-through forwarding in a class and thinking it was obviously going to be used everywhere. Then I was disappointed to enter the real world of networking and learn that it wasn’t actually used as often as I had been led to believe due to conflicting with higher priority features. So the interview may have been a conflict between textbook learning and the Google engineer’s practical experience. It’s tough to encounter that conflict in interviews because you get two people talking past each other.