The email arrived on March 3rd. "Your AWS estimated charges for February are $3,417.82."
I stared at it for a few seconds, convinced I was misreading. My expected spend was maybe $180. I opened the AWS Cost Explorer and felt my stomach drop.
Data Transfer: $2,891.44.
What I Had Done Wrong
I had set up a small data processing pipeline in February. EC2 instances in private subnets, talking to S3 and RDS, routing outbound traffic through a NAT Gateway. Standard architecture. I'd done it this way a dozen times before.
What I hadn't done was check the data volume. My processing job was reading and writing to S3 in a loop — about 2.3TB of data over the month. NAT Gateway charges data processing fees per GB of traffic, both in and out. At $0.045 per GB, that's roughly $0.09 per round-trip GB. Times 2,300GB. You do the math.
The fix was a one-line change: use VPC endpoints for S3. Traffic from EC2 to S3 through a VPC endpoint is free — it stays on Amazon's internal network and bypasses the NAT Gateway entirely. I didn't know this. I should have.
The Bigger Problem: AWS Billing Is Designed to Surprise You
I don't want to just tell a mistake story. The structural problem is that AWS pricing involves dozens of dimensions — data transfer, request counts, instance-hours, storage, API calls, AZ transfers — and the interactions between them are not obvious. You can architect what looks like a standard setup and produce a completely unexpected bill because of one dimension you didn't consider.
Three things I changed after this experience:
1. AWS Budgets with Email Alerts
Set a budget with alerts at 50%, 80%, and 100% of your expected spend. Five minutes to set up. This would have caught my overrun on day 3 instead of day 28.
2. Cost Explorer Anomaly Detection
AWS has a built-in anomaly detection feature that emails you when your spending pattern changes significantly. It's free and takes two minutes to enable in Cost Explorer.
3. Check NAT Gateway vs VPC Endpoints Before Any Data-Heavy Workload
VPC endpoints exist for S3, DynamoDB, and many other AWS services. If you have private subnets communicating with these services through a NAT Gateway, you're almost certainly paying for data transfer you don't need to pay for. Check the VPC Endpoints page in your console.
What AWS Reimbursed
I contacted support, explained the situation, and they refunded $1,800 as a one-time courtesy credit. They were clear it wouldn't happen again. The remaining $1,617 was my education cost. I still have the receipt as a reminder.
