httparchive

7 ms·

1.
by httparchive 3y ago
Update: Google has been helping me out now, thankfully. Hopefully we can make sure this doesn't happen to others.
2.
by httparchive 3y ago
Yeah, 'LIKE' ops usually give you a full table scan, which is brutal. If it was my own data I'd chop the fields up and index them properly - which is the issue here, it's not your data, so you don't get a say in th
3.
by httparchive 3y ago
SELECT page, url, payload FROM `{table}` WHERE page like '%{site_domain}/%' AND url like '%[EXAMPLE.COM]%'
4.
by httparchive 3y ago
Here you go! SELECT page, url, payload FROM `{table}` WHERE page like '%{site_domain}/%' AND url like '%[EXAMPLE.COM]%' --- There's no LIMIT on it b/c I actually need all the results.
5.
by httparchive 3y ago
Exactly, even after seeing the issue I can't make heads or tails of what the hell a "TBs scanned" is relative to row counts, etc. Likewise, it seems to place a lot of assumptions on knowing what tables include - and on a dat
6.
by httparchive 3y ago
Yeah, I have spent much more than $14k to date and would have spent much more over time, losing my business isn't rational. I think it's just another "Google can't do customer support to literally save their life"
7.
by httparchive 3y ago
I've forgotten more Sql than most people ever learn. Time is also valuable and I make trade-offs. Should I spend hours (eg. $$$) to optimize or run a non-optimized query in the background for a different cost? Well, I didn't th
8.
by httparchive 3y ago
I was doing historical evaluation for a few sites, so I was running a query for each month going back to 2016 for each site. I've done this before with no real issues, and if I knew the charges were rapidly exploding I'd have hal
9.
by httparchive 3y ago
Yes, sure there's stuff I could have done better, and stayed up all night looking at the fine print. But that's not the point - this is *warning* to other people who see the Internet Archive logo, the words "public", an
10.
by httparchive 3y ago
It wasn't personal use, for business - but I'm bootstrapping a startup, so it's a very tough lesson to learn.
11.
by httparchive 3y ago
Yeah, I'm basically just having to write this off so it sucks for me (a lot - I'm bootstrapping a start up), but I'm more worried about other people (especially students) getting caught up in what feels like a scam given the
12.
by httparchive 3y ago
Yes and no, I ran the script before and the fee wasn't that high (they jacked it up last summer). Usually I have to jump through a ton of hoops just to add more CPU cores to my VMs so I "trusted" that GCP would warn me if I
13.
by httparchive 3y ago
yeah, but don't "eat up your quota" seems rather tame, whereas "you can get billed $14k with no warning" is the truth.
14.
by httparchive 3y ago
I learned of that billing limiting mechanism after the $14k was charged to my account. As designed.
15.
by httparchive 3y ago
I used this data when I was a grad student, back when there wasn't a fee for it, so I'm mostly concerned students will get hit with charges that will make it so they can't buy groceries. The website has the Internet Archive l