5 ms·
They’re very nice for some things. One challenge I run into is I run many agents at once, so local resource are a tiny fraction of the total inference we’re us
by trollbridge 24d ago
They’re very nice for some things.
One challenge I run into is I run many agents at once, so local resource are a tiny fraction of the total inference we’re using. Every developer with his own Pro 20x, Max, etc. accounts is letting us hit literally trillions of tokens a month.