5 ms·

That was my first thought, varnish (vinyl these days) in front of the website should dramatically reduce this as the git repo should be practically static for m

by Zariel 18d ago

That was my first thought, varnish (vinyl these days) in front of the website should dramatically reduce this as the git repo should be practically static for most of the content.

teo_zero 18d ago
But there are "cubic bazillions" of possible URLs that are being requested. Even if they boil down to "only" some millions actual commits, their rendered HTMLs are all different.
- kijin 18d ago
  Yes, this is very difficult to solve for sites with many URL variations, like git repos and heavily threaded forums. Your cache is always full, but the hit ratio is abysmal.
  - oowa 18d ago
    no its only difficult for you. and only at this moment... any minute now you will see the way. btw, the OP is just saying "its OK for now". And the OP is just telling us: this is what has been happening... maybe its difficult for OP also, but they didnt say that. they just said their current setup cant handle it. old tech...have a hackathon to solve for this. OP says it's not a problem for him right now but if we extrapolate what he's talking about it's definitely a problem aaaaaaand It's totally solvable, Even with all of the crazy combinations he's talking about it's still solvable. And it's already been solved using patterns we see in streaming services. This is completely hackathonable. but why do we even need to bother with this? The slurpers are the cause of this, and they can cause this problem because of Murphy's Law. well you can only account for Murphy's Law with good architecture or something like that or whatever. Ha ha hackathon.
  - oowa 18d ago
    remember, its only difficult for you. at the end of the day its just 7gb of data...
- oowa 18d ago
  have a hackathon to solve for this. OP says it's not a problem for him right now but if we extrapolate what he's talking about it's definitely a problem aaaaaaand It's totally solvable, Even with all of the crazy combinations he's talking about it's still solvable. And it's already been solved using patterns we see in streaming services. This is completely hackathonable. but why do we even need to bother with this? The slurpers are the cause of this, and they can cause this problem because of Murphy's Law. well you can only account for Murphy's Law with good architecture or something like that or whatever. Ha ha hackathon.

by Zariel 18d ago

That was my first thought, varnish (vinyl these days) in front of the website should dramatically reduce this as the git repo should be practically static for most of the content.

teo_zero 18d ago

But there are "cubic bazillions" of possible URLs that are being requested. Even if they boil down to "only" some millions actual commits, their rendered HTMLs are all different.

kijin 18d ago
Yes, this is very difficult to solve for sites with many URL variations, like git repos and heavily threaded forums. Your cache is always full, but the hit ratio is abysmal.
- oowa 18d ago
  no its only difficult for you. and only at this moment... any minute now you will see the way. btw, the OP is just saying "its OK for now". And the OP is just telling us: this is what has been happening... maybe its difficult for OP also, but they didnt say that. they just said their current setup cant handle it. old tech...have a hackathon to solve for this. OP says it's not a problem for him right now but if we extrapolate what he's talking about it's definitely a problem aaaaaaand It's totally solvable, Even with all of the crazy combinations he's talking about it's still solvable. And it's already been solved using patterns we see in streaming services. This is completely hackathonable. but why do we even need to bother with this? The slurpers are the cause of this, and they can cause this problem because of Murphy's Law. well you can only account for Murphy's Law with good architecture or something like that or whatever. Ha ha hackathon.
- oowa 18d ago
  remember, its only difficult for you. at the end of the day its just 7gb of data...
oowa 18d ago
have a hackathon to solve for this. OP says it's not a problem for him right now but if we extrapolate what he's talking about it's definitely a problem aaaaaaand It's totally solvable, Even with all of the crazy combinations he's talking about it's still solvable. And it's already been solved using patterns we see in streaming services. This is completely hackathonable. but why do we even need to bother with this? The slurpers are the cause of this, and they can cause this problem because of Murphy's Law. well you can only account for Murphy's Law with good architecture or something like that or whatever. Ha ha hackathon.