9 ms·

Ask HN: Why call it "inference" instead of just "model hosting"?

6 points by bluebic 2mo ago

mrhottakes 2mo ago
Inference sounds cooler and more mystical. Any old company can host something, "running inference" requires 10x rock star engineers and a CEO that acts like a badass who wears a leather jacket and has Thoughts about demographics.
- rubylimetea 2mo ago
  This literally has to be it. The people that say it tend to say it a lot.
  - deleted 2mo ago
    [deleted]
bitpush 2mo ago
Same difference between S3 bucket and Google Photos (web). Both "host" images but one provides an experience on top of the underlying assets.
- rubylimetea 2mo ago
  That's interesting, the idea that hosting implies a front-end (whether UI or API). So it's similar to why we don't call Heroku or Vercel "hosting" or "compute" because it's more of a service (though "cloud" and "platform" are pretty vague too).
  - Jtsummers 2mo ago
    When you run a compiler, do you "host" it, or do you run it? Inference is the lingo in the ML world for running input through a model to get some output, where that execution occurs is irrelevant to the idea of inference itself. If you have a hosted compiler service and you compiled a program with it, you wouldn't say you "compiler hosted" the source code, no, you still "compiled the source code". In the same way, with ML, if it's hosted or local when you run input through the model you performed "inference", that is the act that has occurred (we can debate the term "inference", but "model hosting" is a worse term).
dtagames 2mo ago
Like so many things in computing, we like to make up words and also reuse old ones for completely different things. I'd say "hosting" means making something available to you in the cloud. Model makers don't offer that. They sell "inference," which is a compute task with real hardware costs for them. The task runs against their model, which is not hosted for you.
seydor 2mo ago
Are you really hosting the model? Are you making its bed, cooking its food and cleaning up for it?
- rubylimetea 2mo ago
  Insert joke about lack of garbage collection in Rust ultimately concluding that I'm not very hospitable
dlcarrier 2mo ago
Inference is what you do with the model. It's not an executable, so you don't run a model directly, instead a separate program runs and that program reads from the model. Also, hosting can be used to mean running inference on a model, but it could also be used to mean just storing the files and possibly making them available for download, so it's a bit more vague.
MostlyFragile 2mo ago
I wouldn’t say they’re interchangeable. You would have both apply to a model but they have different meanings. You’d host the model so it’s available to be used. You’d request it to infer when you wanted to use the model for its purpose.

6 points by bluebic 2mo ago

Inference sounds cooler and more mystical. Any old company can host something, "running inference" requires 10x rock star engineers and a CEO that acts like a badass who wears a leather jacket and has Thoughts about demographics.

rubylimetea 2mo ago
This literally has to be it. The people that say it tend to say it a lot.
- deleted 2mo ago
  [deleted]

bitpush 2mo ago

Same difference between S3 bucket and Google Photos (web). Both "host" images but one provides an experience on top of the underlying assets.

rubylimetea 2mo ago
That's interesting, the idea that hosting implies a front-end (whether UI or API). So it's similar to why we don't call Heroku or Vercel "hosting" or "compute" because it's more of a service (though "cloud" and "platform" are pretty vague too).
- Jtsummers 2mo ago
  When you run a compiler, do you "host" it, or do you run it? Inference is the lingo in the ML world for running input through a model to get some output, where that execution occurs is irrelevant to the idea of inference itself. If you have a hosted compiler service and you compiled a program with it, you wouldn't say you "compiler hosted" the source code, no, you still "compiled the source code". In the same way, with ML, if it's hosted or local when you run input through the model you performed "inference", that is the act that has occurred (we can debate the term "inference", but "model hosting" is a worse term).

dtagames 2mo ago

Like so many things in computing, we like to make up words and also reuse old ones for completely different things. I'd say "hosting" means making something available to you in the cloud. Model makers don't offer that. They sell "inference," which is a compute task with real hardware costs for them. The task runs against their model, which is not hosted for you.

seydor 2mo ago

Are you really hosting the model? Are you making its bed, cooking its food and cleaning up for it?

rubylimetea 2mo ago
Insert joke about lack of garbage collection in Rust ultimately concluding that I'm not very hospitable

dlcarrier 2mo ago

Inference is what you do with the model. It's not an executable, so you don't run a model directly, instead a separate program runs and that program reads from the model. Also, hosting can be used to mean running inference on a model, but it could also be used to mean just storing the files and possibly making them available for download, so it's a bit more vague.

MostlyFragile 2mo ago

I wouldn’t say they’re interchangeable. You would have both apply to a model but they have different meanings. You’d host the model so it’s available to be used. You’d request it to infer when you wanted to use the model for its purpose.