5 ms·

Assuming both Intel and AMD implement performance monitors the same (i.e. same notion of instructions executed, which may be hard to measure with speculative ex

Assuming both Intel and AMD implement performance monitors the same (i.e. same notion of instructions executed, which may be hard to measure with speculative execution), the comparison is still flawed because it doesn’t matter if Intel can do more instruction per cycle if AMD can produce more cycles in a span of wall time. > However, it is not clear whether these reports are genuinely based on measures of instruction per cycle. Rather it appears that they are measures of the amount of work done per unit of time normalized by processor frequency. That’s precisely why nobody really uses IPC as a way to compare processors. “How much work done per unit of time” is a much better measurement and I guess for historical reasons, people conflate it with IPC. But real textbook IPC is useless for comparison.

BeeOnRope 7y ago
I'm this case the frequencies are similar and so wall clock time reflects the IPC difference (also, the two CPUs take the same code path, so the I is the same in this case, which isn't always true).
- mping 7y ago
  But on these processors, I believe the frequency is rarely sustained right? Due to thermal throttling and other factors.
  - ShinyRice 7y ago
    That only really happens on laptops, which can't dissipate as much heat as desktop systems due to size constraints. On a desktop, if you're using even AMD's stock cooler, you won't thermal throttle. That is, if you don't overclock.
    - AstralStorm 7y ago
      It's not about throttling. What will happen is that the CPU won't automatically clock up dynamically as much if you have worse cooling. They behave like GPUs more and more with regards to clocks.
      - BubRoss 7y ago
        That's the same thing. Intel calls their stuff a dynamic boost so that some of their measurements like TDW are for lower clocks. Both CPUs end up scaling their clocks to a wide range.
      - gruez 7y ago
        >They behave like GPUs more and more with regards to clocks. I think it's the other way around? CPUs had "boost" before GPUs.
    - vel0city 7y ago
      Modern processors with boost configurations are rather complicated about "thermally throttling". These days with AMD's stock coolers you will be able to at least get the sticker speed on the CPU even at 100% load for a sustained time. Chances are, you'll actually get some % more speed than the sticker as it will usually continue to boost as long as power delivery and temperatures are stable. So even with an entirely stock configuration, a better motherboard and cooling system will overall net you more performance. This is without doing any traditional "overclocking" and just going with the settings designated with the CPU and motherboard. This same idea also applies to most of Intel's parts as well.
- rocqua 7y ago
  Could be that one process spends much less time sleeping for IO thus still having the same wall clock time. In this case, there's probably only memory IO which (afaik) cannot put a process to sleep.
  - BeeOnRope 7y ago
    There is no IO, and it is not memory intensive.
shaklee3 7y ago
I think it would have been useful if the author benchmarked the actual time taken to parse a large json file, and did a sanity check to make sure the time difference made sense with ipc/clock factored in.
black_puppydog 7y ago
> But real textbook IPC is useless for comparison. It's useful for comparing architectures and the implementation thereof, to gauge the potential of one line of processors over the other. I agree that for the customer it's not the right thing to be looking for.
- AstralStorm 7y ago
  There's no "gauging potential". Would you suddenly go with OICC if it has extremely high IPC? How about old Core instead of new Skylake? Oh shoot, there is no potential in Core if it's not being made! Even different Zen 2 CPUs have varied performance properties not just due to cores, but due to CCX count. The exactly one use for such microbenchmark and that's optimizing the compilers. Even if there were multiple implementations? Also remember that x86-64 unlike x86 is not closed, and unlike POWER, RISC-V, ARM or MIPS is not actually well defined. If AMD suddenly adds a new but useful instruction set like they did with 3DNOW in ancient times, or accelerate something reasonably common that way, say add a special SIMD conditional, where do you even start in comparison? What if Intel actually does add a useful FPGA programmable computing capability as promised or enhanced DMA?
- Symmetry 7y ago
  It's really not useful for gauging potential. There are tradeoffs in how deeply you pipeline your architectures that'll tend to result in higher clock rates for shorter pipeline stages but higher IPC for longer pipeline stages, for instance. It's pretty easy to make a design with an IPC that'll blow everything else out of the water if it only needs to hit 100 MHz. For instance the slower a clock cycle is the larger you can make your caches and the less clock cycles it takes to read from them. Also, on real world benchmarks that don't fit neatly in cache, for a given chip IPC will tend to increase as you underclock it because that will cause memory latency to go down.
  - sitkack 7y ago
    IPC is _usually_ a good measure for the last phase of optimization. But it is only the local Δ that is meaningful, comparing IPC across different vendors is only useful as a gross measure.
deleted 7y ago
[deleted]
Someone 7y ago
I don’t see how it is flawed. The article doesn’t discuss whether the AMD CPU is faster than the Intel CPU, it discusses the claim "that the most recent AMD processors surpass Intel in terms of instructions per cycle” (https://www.guru3d.com/articles_pages/amd_ryzen_7_3800x_review,9.html https://www.guru3d.com/articles_pages/amd_ryzen_7_3800x_revi...) And IPC, IMO, is a better measurement for a chip’s design than pure speed, as it removes the “but how good a process do you have access to” from the equation.
- mda 7y ago
  The article gives 2 benchmarks, I am pretty sure it is easy to mash up another benchmark with totally opposite results (e.g subset of specint). I found author's inclusion of an obviously skewed example as proof a little bit disingenuous as well. Having said that in general Intel still holds a slight edge on pure Ipc. However, considering the terrible track record of security issues and abysmal price performance ratio, a slight edge on ipc can be ignored and I would not consider Intel for most workloads at the moment. Above all, actual application benchmark trumps any ipc microbencmark.
endorphone 7y ago
"the comparison is still flawed because it doesn’t matter if Intel can do more instruction per cycle if AMD can produce more cycles in a span of wall time." The reason Intel had the "per core" superiority crown for years is that it had a better IPC performance due to design efficiency. Both manufacturers are pushing against the same frequency ceiling, so if you went AMD you had to significantly increase the core count to catch up, and could never match the still important single-thread performance. We know from large scale, comprehensive benchmarks that AMD has massively picked up the pace and is neck and neck with Intel. At the same processor speed it matches the best Intel processors. But yeah, this article is just terrible. Not just tiny, minuscule, extremely myopic benchmarks, but then a gross over-reach with conclusions. And in the way that ignorance begets ignorance, the fact that it's trending on a couple of social news sites means that now Google is surfacing it as canonical information when it's just a junk, extremely lazy analysis.
- api 7y ago
  > surfacing it as canonical information when it's just a junk, extremely lazy analysis. Isn't that what the Internet is for?
- Tempest1981 7y ago
  He ran a few basic tests, and showed the results. Where was the "gross over-reach"? The article ends with a "your mileage may vary" disclaimer.
  - endorphone 7y ago
    "So AMD runs at 2/3 the IPC of an old Intel processor. That is quite poor!" That is most certainly an overreach. An extraordinary overreach. Worse, it's absurdly using an AVX2 codebase, optimized for Westmere, as the baseline for "IPC" testing? The premise itself borders of gross negligence. IPC as a generalized concept is a broad, general purpose set of instructions, not an absurdly narrow test. Saying "Intel is faster at AVX512" is going to surprise exactly no one, and also happens to be irrelevant for the overwhelming majority of users and uses. The microbenchmarking thing has gone on for years, and at this point anyone who has paid any attention is rightly cautious when stomping their feet and making declarations, because usually they're just pouring noise into the mix. Lazily running a couple of tiny tests is not the rigour to avoid deserved criticism.
MrSauna 7y ago
Some more realistic single core workloads at same frequencies: 3900x vs 9900k https://hothardware.com/reviews/amd-ryzen-9-3900x-vs-core-i9-9900k-ipc-shootout?page=1 https://hothardware.com/reviews/amd-ryzen-9-3900x-vs-core-i9...

BeeOnRope 7y ago

I'm this case the frequencies are similar and so wall clock time reflects the IPC difference (also, the two CPUs take the same code path, so the I is the same in this case, which isn't always true).

mping 7y ago
But on these processors, I believe the frequency is rarely sustained right? Due to thermal throttling and other factors.
- ShinyRice 7y ago
  That only really happens on laptops, which can't dissipate as much heat as desktop systems due to size constraints. On a desktop, if you're using even AMD's stock cooler, you won't thermal throttle. That is, if you don't overclock.
  - AstralStorm 7y ago
    It's not about throttling. What will happen is that the CPU won't automatically clock up dynamically as much if you have worse cooling. They behave like GPUs more and more with regards to clocks.
    - BubRoss 7y ago
      That's the same thing. Intel calls their stuff a dynamic boost so that some of their measurements like TDW are for lower clocks. Both CPUs end up scaling their clocks to a wide range.
    - gruez 7y ago
      >They behave like GPUs more and more with regards to clocks. I think it's the other way around? CPUs had "boost" before GPUs.
  - vel0city 7y ago
    Modern processors with boost configurations are rather complicated about "thermally throttling". These days with AMD's stock coolers you will be able to at least get the sticker speed on the CPU even at 100% load for a sustained time. Chances are, you'll actually get some % more speed than the sticker as it will usually continue to boost as long as power delivery and temperatures are stable. So even with an entirely stock configuration, a better motherboard and cooling system will overall net you more performance. This is without doing any traditional "overclocking" and just going with the settings designated with the CPU and motherboard. This same idea also applies to most of Intel's parts as well.
rocqua 7y ago
Could be that one process spends much less time sleeping for IO thus still having the same wall clock time. In this case, there's probably only memory IO which (afaik) cannot put a process to sleep.
- BeeOnRope 7y ago
  There is no IO, and it is not memory intensive.

shaklee3 7y ago

I think it would have been useful if the author benchmarked the actual time taken to parse a large json file, and did a sanity check to make sure the time difference made sense with ipc/clock factored in.

black_puppydog 7y ago

> But real textbook IPC is useless for comparison. It's useful for comparing architectures and the implementation thereof, to gauge the potential of one line of processors over the other. I agree that for the customer it's not the right thing to be looking for.

AstralStorm 7y ago
There's no "gauging potential". Would you suddenly go with OICC if it has extremely high IPC? How about old Core instead of new Skylake? Oh shoot, there is no potential in Core if it's not being made! Even different Zen 2 CPUs have varied performance properties not just due to cores, but due to CCX count. The exactly one use for such microbenchmark and that's optimizing the compilers. Even if there were multiple implementations? Also remember that x86-64 unlike x86 is not closed, and unlike POWER, RISC-V, ARM or MIPS is not actually well defined. If AMD suddenly adds a new but useful instruction set like they did with 3DNOW in ancient times, or accelerate something reasonably common that way, say add a special SIMD conditional, where do you even start in comparison? What if Intel actually does add a useful FPGA programmable computing capability as promised or enhanced DMA?
Symmetry 7y ago
It's really not useful for gauging potential. There are tradeoffs in how deeply you pipeline your architectures that'll tend to result in higher clock rates for shorter pipeline stages but higher IPC for longer pipeline stages, for instance. It's pretty easy to make a design with an IPC that'll blow everything else out of the water if it only needs to hit 100 MHz. For instance the slower a clock cycle is the larger you can make your caches and the less clock cycles it takes to read from them. Also, on real world benchmarks that don't fit neatly in cache, for a given chip IPC will tend to increase as you underclock it because that will cause memory latency to go down.
- sitkack 7y ago
  IPC is _usually_ a good measure for the last phase of optimization. But it is only the local Δ that is meaningful, comparing IPC across different vendors is only useful as a gross measure.

deleted 7y ago

[deleted]

Someone 7y ago

I don’t see how it is flawed. The article doesn’t discuss whether the AMD CPU is faster than the Intel CPU, it discusses the claim "that the most recent AMD processors surpass Intel in terms of instructions per cycle” (https://www.guru3d.com/articles_pages/amd_ryzen_7_3800x_review,9.html https://www.guru3d.com/articles_pages/amd_ryzen_7_3800x_revi...) And IPC, IMO, is a better measurement for a chip’s design than pure speed, as it removes the “but how good a process do you have access to” from the equation.

mda 7y ago
The article gives 2 benchmarks, I am pretty sure it is easy to mash up another benchmark with totally opposite results (e.g subset of specint). I found author's inclusion of an obviously skewed example as proof a little bit disingenuous as well. Having said that in general Intel still holds a slight edge on pure Ipc. However, considering the terrible track record of security issues and abysmal price performance ratio, a slight edge on ipc can be ignored and I would not consider Intel for most workloads at the moment. Above all, actual application benchmark trumps any ipc microbencmark.

endorphone 7y ago

"the comparison is still flawed because it doesn’t matter if Intel can do more instruction per cycle if AMD can produce more cycles in a span of wall time." The reason Intel had the "per core" superiority crown for years is that it had a better IPC performance due to design efficiency. Both manufacturers are pushing against the same frequency ceiling, so if you went AMD you had to significantly increase the core count to catch up, and could never match the still important single-thread performance. We know from large scale, comprehensive benchmarks that AMD has massively picked up the pace and is neck and neck with Intel. At the same processor speed it matches the best Intel processors. But yeah, this article is just terrible. Not just tiny, minuscule, extremely myopic benchmarks, but then a gross over-reach with conclusions. And in the way that ignorance begets ignorance, the fact that it's trending on a couple of social news sites means that now Google is surfacing it as canonical information when it's just a junk, extremely lazy analysis.

api 7y ago
> surfacing it as canonical information when it's just a junk, extremely lazy analysis. Isn't that what the Internet is for?
Tempest1981 7y ago
He ran a few basic tests, and showed the results. Where was the "gross over-reach"? The article ends with a "your mileage may vary" disclaimer.
- endorphone 7y ago
  "So AMD runs at 2/3 the IPC of an old Intel processor. That is quite poor!" That is most certainly an overreach. An extraordinary overreach. Worse, it's absurdly using an AVX2 codebase, optimized for Westmere, as the baseline for "IPC" testing? The premise itself borders of gross negligence. IPC as a generalized concept is a broad, general purpose set of instructions, not an absurdly narrow test. Saying "Intel is faster at AVX512" is going to surprise exactly no one, and also happens to be irrelevant for the overwhelming majority of users and uses. The microbenchmarking thing has gone on for years, and at this point anyone who has paid any attention is rightly cautious when stomping their feet and making declarations, because usually they're just pouring noise into the mix. Lazily running a couple of tiny tests is not the rigour to avoid deserved criticism.

MrSauna 7y ago

Some more realistic single core workloads at same frequencies: 3900x vs 9900k https://hothardware.com/reviews/amd-ryzen-9-3900x-vs-core-i9-9900k-ipc-shootout?page=1 https://hothardware.com/reviews/amd-ryzen-9-3900x-vs-core-i9...