11 ms·
Slashing data transfer costs in AWS
- jakozaur 3y agoS3 is a nice trick. More tricks: 1. Ask for discounts if you are a big AWS customer (e.g., spend $1mln+/year). At some point, they were huge for inter-AZ transfers. 2. Put things in one AZ. Running DB in "b" zone and your only server in "a" is even worse than just standardizing on one zone. 3. When using multiple AZ do load aware AZ balancing.
- pibefision 3y ago4. Activate S3 Inteligent-Tier storage class?
- danielklnstein 3y agoThis is great for saving on S3 storage costs! But in the context of data transfer costs, this would actually increase the costs, because there's a small surcharge for Intelligent Tiering - and the only relevant storage class for sidestepping data transfer costs is standard storage (because it's the only one with free download), so Intelligent Tiering won't provide value.
- endgame 3y agoYou've got to be careful of the automation charge with Intelligent Tiering. https://discourse.nixos.org/t/the-nixos-foundations-call-to-action-s3-costs-require-community-support/28672/103?u=endgame https://discourse.nixos.org/t/the-nixos-foundations-call-to-...
- throwaway167 3y ago> Running DB in "b" zone and your only server in "a" There must be use cases for this, but I lack imagination. Cost? But not cost?
- jakozaur 3y agoDefaults. Either as a code or using click ops. Many companies run servers without considering AZ. Then you can get the "best" of the worlds: 1. Your service is down if either of AZs gets hiccups. 2. You pay network charges and latency cost.
- QuadmasterXLII 3y agoCost, unlimited cost- but no cost.
- sokoloff 3y agoI can't see a reason to do this intentionally within a single account, but use cases with multiple accounts should be aware that what AZ has ID us-east-1a in Account 1 is not necessarily the same AZ that has the ID us-east-1a in Account 2. https://docs.aws.amazon.com/ram/latest/userguide/working-with-az-ids.html https://docs.aws.amazon.com/ram/latest/userguide/working-wit...
- andrewstuart 3y agoAn alternative to sophisticated cloud cost minimization systems is…….. don’t use the cloud. Host it yourself. Or use Cloudflare which has 0 cents per gigabyte egress fees. Or just rent cloud servers from one of the many much cheaper VPS hosting services and don’t use all the expensive and complex cloud services all designed to lock you in and drain cash from your at 9 or 12 or 17 cents per gigabyte. Seriously, if you’re at the point that you’re doing sophisticated analysis of cloud costs, consider dropping the cloud.
- throwaway2990 3y ago[flagged]
- junon 3y agoThere are a slew of reasons why people say this. Generalizing HN folks as "dumb" doesn't help your point either.
- cdchn 3y agoWhich VPS services don't charge you at all for bandwidth?
- dijit 3y agoyou'd be harder pressed finding ones that do (before a reasonable limit). tilaa.com vultr.com hetzner.com linode.com
- JoshuaRogers 3y agoWhile they might not charge you directly as a line item, you still get charged: Linode in the above list is what I use. I get a fixed cap of bandwidth each month. Anything beyond that is charged. So, you don't get charged IF you stay below the initial cap.
- cdchn 3y ago
- Havoc 3y agoIt’s unfortunate that such shenanigans are even necessary
- ishitatsuyuki 3y agoGCP patched a similar loophole [1] in 2023 presumably because some of their customers were abusing it. I'd expect AWS to do the same if this becomes widespread enough. [1]: https://cloud.google.com/storage/pricing-announce#network https://cloud.google.com/storage/pricing-announce#network
- rfoo 3y agoUnlikely. The "loophole" GCP patched was that you can use GCS to transfer data between regions on the same continent for free. This is already non-free on AWS. What OP mentioned is that transferring data between availability zones *in the same region* also costs $0.02 per GB and can be worked around.
- cedws 3y agoI know of a way to get data out of GCP for free, although I haven't tried it in practice. Wonder if I could find a buyer for this info ;)
- greyface- 3y agoA guess: tunnel through 169.254.169.254 DNS server?
- cedws 3y agoIt's a good guess, but what I have in mind would have high bandwidth.
- BonoboIO 3y agoI got one for Azure, where it would be nearly free to egress data from Azure to any other cloud provider or the internet. It works, but i have no use case for it. 100TB egress to the internet costs about 7000$ ... i think, i could do it for 20$-50$.
- Cthulhu_ 3y agoThis doesn't feel like a loophole though, it feels like they have optimized S3 and intend your EC2 instances to use S3 as storage. But maybe not as transfer window, that is, they expect you to put and leave your data on there.
- jonatron 3y agoIf you're a heavy bandwidth user it's worth looking at Leaseweb, PhoenixNAP, Hetzner, OVH, and others who have ridiculously cheaper bandwidth pricing. I remember a bizarre situation where the AWS sales guys wouldn't budge on bandwidth pricing even though the company wouldn't be viable at the standard prices.
- declan_roberts 3y agoThat’s very unusual, I think. Transfer cost seem to be something most people can negotiate.
- jonatron 3y agoI hadn't really thought about it much, but googling it looks like there's a discount programme for a committed spend of around $1M/year. For a small company, that's a lot of money, and it was an unusually large amount of bandwidth for the size of company. I suppose it makes sense now I know they're interested in companies spending that sort of money.
- rescbr 3y agoThe trick is to use CloudFront if possible, even if not caching, just passing through requests. Standard discounts start at 10 TB, which is not that much. If not using HTTP, then it's a no starter.
- quickthrower2 3y agoThis is a loophole. Hitting some loss leader at AWS, but if everyone only buys the $1 hotdog and nothing else then the $1 hotdog gets removed.
- danielklnstein 3y agoI'm not sure how this could be removed - the fundamentals behind it are basic building blocks of S3. Maybe raising the cost of transient storage? e.g. If you have to pay for a minimum of a day's storage - but even if that was the case this would still be cost-effective, and at any rate it seems very unnatural for AWS to charge on such granularity. + I would guess that S3 is orders of magnitude more profitable for AWS than cross-AZ charges, so I'm not sure they'd consider it a loss-leader.
- kevincox 3y agoIt would be fairly easy to change the pricing policy. GCP did something similar for cross-region https://cloud.google.com/storage/pricing-announce#network https://cloud.google.com/storage/pricing-announce#network. This is pretty severe because it seems to affect all reads. However I can imagine an alternate implementation where the source AZ is tracked when data is written and egress fees are charged when the data is read (as if the data was always stored in the source AZ). This could even be done more complexly such as only charging the first time data is read in another AZ. Once you read once it is free as-if it is now cached in that new AZ forever. Another option would just be raising the minimum storage duration so that it basically costs all or most of what the data transfer would. It would definitely piss a lot of people off as it is adding to their bill, but it could likely be done in a way that makes exploiting this for just data transfer not worth it without adding huge costs to most "real" use cases.
- danielklnstein 3y agoYeah, I see what you mean - that'd indeed render this method ineffective. Like you said I'm sure this would bother a lot of customers, but it's not a completely unrealistic overhaul of S3 pricing. That being said, that'd be sort of "mean" of AWS to do - the data is already replicated across AZs whether you pay for it or not because of how S3 works.
- DeathArrow 3y agoCan you do the same on Azure?
- andersa 3y agoI don't understand how AWS can keep ripping people off with these absurd data transfer fees, when there is Cloudflare R2 just right over there offering a 100 times better deal.
- fabian2k 3y agoR2 is still pretty new. I don't know how well it works in practice in terms of performance and availability. And of course durability, which is difficult if not impossible to judge. S3 has a much longer history and track record, so it has the advantage here. And if all your stuff is inside AWS already there are advantages to keeping the data closer. Depending on how the data is used, egress might also not always be such a major cost. But yes, the moment you actually produce significant amounts of egress traffic it gets absurdly expensive. And I would expect competitors like R2 to gain ground if they can provide reasonably competitive reliability and performance.
- tnolet 3y agowe just built a new feature for our pretty bandwidth heavy SaaS on R2. Works pretty damn good with indeed massive savings. We just use the AWS-SDK (Node.js) and use the R2 endpoint.
- karlkatzke 3y agoData has "gravity" -- as in, it holds you down to where your data is, and you have to spend money to move it just like you have to spend money to escape gravity.
- perryizgr8 3y agoWhen all my VMs and containers are hosted in AWS, and S3 has rock solid support no matter what language, framework, setup I use, it becomes really tough to ask the team to use another vendor for object storage. If something goes wrong with R2 (data loss, slow transfer, etc.) I will get blamed (or at least asked for help). If S3 loses data or performs slowly in some case, people will just figure we're somehow using it wrong. And they will figure out how to make it better. Nobody gets blamed. And to be honest, data transfer fees is negligible if your business is creating any sort of value. You don't need to optimise it.
- develatio 3y agoI'll share my trick :) Lightsail instances can be used to "proxy" data from other AWS resources (eg EC2 instances or S3 buckets). Each Lightsail instance has a certain amount of data transfer included in it's price ($3.5 instance has 1TB, $5 instance has 2TB, $10 instance has 3TB, $20 instance has 4TB, $40 instance has 5TB). The best value (dollar per transferred data) is the $10 instance, which gives you 3TB of traffic. Using the data provided by the post: 3TB worth of traffic from an EC2 would cost $276.48 (us-east-1). 3TB worth of traffic from a S3 bucket would cost $69. Note: one downside of using Lightsail instances is that both ingress and egress traffic counts as "traffic".
- jonatron 3y agohttps://aws.amazon.com/service-terms/ https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspend your data services or suspend your account.
- develatio 3y agoA had a suspicion that this was against AWS's terms, but I never bothered to look if that was actually the case. Thank you for the heads up!
- slashdev 3y agoIt’s mostly in there to scare people into not doing it. AFAIK they’ve never taken action on that. Of course if you abuse it, you’re asking for trouble.
- mdasen 3y agoAs someone who has dealt with users who use a system in an unintended way, you don't go looking for those people and you don't build something to enforce a policy like this. When you're running services for lots of customers, you often don't know a lot of what's going on in the system and how people are using it. Then something seems weird or something is causing a problem and you want to deal with it - and you want the language out there so that you can deal with it. In Amazon's case, their bandwidth pricing isn't really defendable. It's just crap. However, sometimes you're trying to offer something reasonable, but need to make sure that a customer doesn't end up abusing something. For example, Chia is a cryptocurrency that will basically wear through SSDs (it's a proof-of-space system). There aren't explicit limits on how frequently you can write to a disk from most hosting providers, but Chia goes beyond what normal usage would do to a disk. Chia farmers would rather burn someone else's SSD that they're renting than their own. But no one at most hosting providers was probably looking at how frequently people were writing before noticing "hey, why are the disks failing faster than we'd expect?" They probably haven't taken action on it because they probably haven't noticed it being a problem. But if you're a whale of a customer and suddenly your data transfer charges drop off a cliff, someone might end up looking into that and seeing what's going on.
- explain 3y agoPaying for bandwidth is crazy.
- TruthWillHurt 3y agoTrue meaning of Trustless Environment.
- glenngillen 3y agoThis is clever. And as I understand it, one of the tricks WarpStream (https://www.warpstream.com https://www.warpstream.com) use to reduce the costs of operating a Kafka cluster.
- lucidguppy 3y agoThis feels like the tech equivalent of tax avoidance. If too many people do this - AWS will "just close the loophole". There's not one AWS - there are probably dozens if not hundreds of AWS - each with their own KPIs. One wants to reduce your spend - but not tell you how to really reduce your spend. If you make something complex enough (AWS) - it will be impossible for customers to optimize in any one factor - as everything is complected together.
- karlkatzke 3y agoThis isn't a loophole. This is by design. AWS wants you to use specific services in specific ways, so they make it really cheap to do so. Using an endpoint for S3 is one of the ways they want you to use the S3 service. Another example is using CloudFront. AWS wants you to use CloudFront, so they make CloudFront cheaper than other types of data egress.
- lucidguppy 3y agoIf they wanted you to behave in specific ways logically - wouldn't their documentation be less ambiguous? https://www.lastweekinaws.com/blog/aws-cross-az-data-transfer-costs-more-than-aws-says/ https://www.lastweekinaws.com/blog/aws-cross-az-data-transfe...
- dangoodmanUT 3y agoI've not seen any evidence that multi-AZ is more resilient. There's no history of an entire AZ going down that doesn't affect the entire region, at least that I can find on the internet within 15 minutes of googling.
- playingalong 3y agoDo you mean S3 or all services? If all services, then things like whole or most of a single AZ being borked happens fairly often.
- arpinum 3y agoAnother trick is to use ECR. You can transfer 5TB out to the internet each month for free. The container image must be public, but you can encrypt the contents. Useful when storing media archives in Glacier.
- declan_roberts 3y agoSneaky idea! I love it!
- deleted 3y ago[deleted]
- gumballindie 3y agoI reduced them to 0 by not using AWS. This simple trick lets you install and configure dedicated servers that work just fine. Most of your auto scaling needs can be solved using a CDN. But by the time you reach such needs you'd have hired competent engineers to properly architect things - it will be cheaper than using amazon anyway.
- TheNewsIsHere 3y agoThis may be arguably nitpicking, but the following statement from TFA isn’t exactly the case: > Moreover, uploading to S3 - in any storage class - is also free! Depending upon how much data you’re transferring in terms of storage class, number of API calls your software makes to do so, and the capacity used, you may incur charges. This is very easy to inadvertently do when uploading large volumes of archival data directly to the S3 Glacier tiers. You absolutely will pay if you end up using millions of API calls to upload tens of millions of objects comprising tens of terabytes or more.
- danielklnstein 3y agoThanks for the feedback! I don't think it's nitpicking, you're right that it's misleadingly phrased - in fact, the only S3 costs I observed weren't storage at all, but rather the API calls. I updated the phrasing.
- boiler_up800 3y agoS3 storage costs are charged per GB month so 1 TB * .023 per GB / 730 hrs per month… should be 3 cents if the data was left in the bucket for an hour.[1] However sounds like it was deleted almost right away. In that case the charge might be 0.03 / 60 if the data was around for a minute. Normally I would expect AWS to round this up to $0.01.. The TimedByteStorage value from the cost and usage report would be the ultimate determinant here. [1] https://handbook.vantage.sh/aws/services/s3-pricing/ https://handbook.vantage.sh/aws/services/s3-pricing/
- mlhpdx 3y agoI’ve been deploying 3xAZs in 3xRegions for a while now (years). The backing store being regional s3 buckets (keeping data in the local compliance region) and DDB with replication (opaque indexing and global data) and Lambda or Sfn for compute. So far data transfer hasn’t risen to the level of even tertiary concern (at scale). Perhaps because I don’t have video, docker or “AI” in play?
- hipadev23 3y agoI’m guessing either you don’t have much data or your infra is already so absurd that, yeah, the transfer costs are irrelevant by comparison.
- mlhpdx 3y agoNot using VPCs (no need without instances/containers/RDS) mean most of the “absurd” costs go away. It’s cheap by any standard.
- hipadev23 3y agoVPCs don’t cost anything.
- mlhpdx 3y agoSorry, that was indeed nonspecific, you’re right. The add-on features for VPCs are commingled with the concept for me since they almost always go hand in hand. Internet gateways, transit gateways, EIPs, service endpoints, etc., and their fixed costs. Yuck.
- icedchai 3y agoAll that stuff definitely adds up. I'm familiar with some low traffic projects with high "security" requirements that have so much overhead due to those sorts of adds-on. All the overhead winds up costing more than the actual compute + bandwidth running the site.
- playingalong 3y agoThis trades costs for latency. Which is not a big deal for some use cases, but may be a real breaker for some of the others.
- esafak 3y agoHow do people economically run multi-region databases for low latency?
- declan_roberts 3y agoData transfer costs are extremely easy to negotiate with Amazon.
- vlovich123 3y ago> it’s almost as if S3 doesn’t charge you anything for transient storage? This is very unlike AWS, and I’m not sure how to explain this. I suspected that maybe the S3 free tier was hiding away costs, but - again, shockingly - my S3 storage free tier was totally unaffected by the experiment, none of it was consumed (as opposed to the requests free tier, which was 100% consumed). It’s also possible their billing system can’t detect transient storage usage. Request billing would work differently from how billed storage is tracked. It depends on how billing is implemented but would be my guess. That may change in the future.
- adrianmonk 3y agoMaybe some sampling mechanism comes along and takes a snapshot once per hour. Suppose you store the data there for 6 minutes. Then there's an 90% probability that the sampler misses it entirely and you pay $0. But there's a 10% probability that the sampler does catch it. Then you pay for a whole hour even though you used a fraction of that. Over many events, it averages out close to actual usage[1]. In 9 out of 10 cases, you pay 0X actual usage. In 1 out of 10 cases, you pay 10X actual usage. (But you can't complain because you did agree to 1-hour increments.) --- [1] Assuming no correlation between your timing and the sampler's timing. If you can evade the sampler by guessing when it runs and carefully timing your access, then you can save a few pennies at the risk of a ban.
- lijok 3y agoThere are tons of these tricks you can use to cut costs and get resources for free. It's smart, but not reliable. It's the same type of hacking that leads to crypto mining on github actions via OSS repos. Treat this as an interesting hacking exercise, but do not deploy a solution like this to production (or at least get your account managers blessing first), lest you risk waking up to a terminated AWS account.
- huslage 3y agoI have used this and other techniques for years and never gotten shut down. Passing through S3 is also generally more efficient for distributing data to multiple sources than running some sync process.
- sebazzz 3y agoOfftopic but related: Has anyone noticed transient AWS routing issues as of late? I’ve on three or four occasions the last three months notices that I got served a completely different SSL certificate than the domain I was visiting, of a domain that often could not be reached on publicly - probably pointing to some organizations internal OTA environment. In all occasions the URL I wanted to visit and the DNS of the site I was then actually visiting were located in AWS. Then less than a minute elapsed the issue is resolved. I first thought it must be my side, my DNS server malfunctioning or something, but the served sites could not be accessed publicly anyway, and I had the issue on two separate networks with two separate clients and separate DNS servers. I’ve had it with polar.co internal environment, bank of ireland (iirc), multiple times with download.mozilla.org and a few other occassions. I contacted AWS on Twitter about it, but just got some generic pointless response I should make an incident - but I’m just some random user, I’m not an AWS client. Somehow I could not get it clear to the AWS support on the other side of Twitter.
- issafram 3y agoI've been looking for a place to store files for backup. Already keeping a local copy on NAS, but I want another one to be remote. Would you guys recommend S3? Wouldn't be using any other services.
- spieden 3y agoI use S3 with the DEEP_ARCHIVE storage class for disaster recovery. Costs go up if you have many thousands of files so careful there. Hopefully will never need to access the objects and it's the cheapest I could find.
- nodeshift 3y agoSomeone in the thread said that if you're 'at the point that you’re doing sophisticated analysis of cloud costs, consider dropping the cloud.' We've built https://nodeshift.com/ https://nodeshift.com/ with the idea that cloud is affordable by default without any additional optimization, you focus on your app with no concerns on costs or anything else.
- akira2501 3y agoCost analysis has helped me build great infrastructure on AWS. The costs are communicating to you what is and is not efficient for AWS to do on your behalf, by analyzing the costs and working to reduce them, you also incidentally increase efficiency and in some cases, such as this one, workload durability.
- nodeshift 3y agoCost analysis should of course play the foundation of everything you build, regardless if it's SaaS tooling or infrastructure. But surely it's easier to do a cost assessment and optimization exercise on something that is fundamentally more affordable than AWS and doesn't have as high of margin costs? That's why we have built a platform that creates all the value at a low cost.
- xbar 3y agoAfter my account started getting bills this month for pennies for which there was no obvious accounting, I slashed my AWS costs by 100%. I'm back to managing my own systems. So much cheaper and less chance of nonlinear bills.
- danielklnstein 3y agoIn case you ever decide to return to AWS, its Cost Explorer is far from perfect but it can show you where your expenses are coming from, especially if your costs are pennies. In the last re:invent they even released daily granularity when grouping by resources (https://aws.amazon.com/blogs/aws-cloud-financial-management/extended-history-and-more-granular-data-available-within-aws-cost-explorer/ https://aws.amazon.com/blogs/aws-cloud-financial-management/...).
- rospaya 3y agoProbably free tier expiration for some small change. With me it was AWS KMS.
- rco8786 3y agoThere's going to a be a huge market for consultants to unwind people's cloud stacks and go back to simpler on-prem/colo (or Heroku-like) deployments in the coming years.
- salawat 3y agoOr... Build your own cloud and transfer data to your hearts content for free (minus power).
- deleted 3y ago[deleted]
- emmanueloga_ 3y agoFor those suggesting VPSs instead of cloud based solutions, how do you deal with high availability? Even for a small business you may need it to stay up at all times. With a VPS this is harder to accomplish. Do you setup the same infrastructure in two or more VPS instances and then load balance? (say, [1]). Feels a bit of an ... artisanal solution, compared to using something like AWS ECS. 1: https://www.hetzner.com/cloud/load-balancer https://www.hetzner.com/cloud/load-balancer