Skip to content
Learni
View all tutorials
Cloud & DevOps

How to Implement AWS Cloud Cost Allocation in 2026

22 minEXPERT
AWSFinOpsboto3TerraformAthena
Lire en français

Introduction

Cloud cost allocation is crucial in 2026 for companies managing multi-team AWS environments: it enables precise attribution of expenses (EC2, S3, RDS) to projects, departments, or environments, promoting financial accountability (FinOps). Without it, AWS bills balloon without visibility, leading to 30-50% overages per Gartner.

This expert tutorial guides you step by step: configuring Cost and Usage Reports (CUR) for granular data, mandatory tag policies via Terraform, boto3 queries to extract costs by tag, and advanced Athena SQL analyses for allocations. Imagine a dashboard where the DevOps team sees its EC2 costs at €2,500/month vs. Prod at €15,000 – that's what we're building.

By the end, you'll have an automatable pipeline producing monthly allocated CSVs, integrable with QuickSight or Excel. Based on AWS Cost Explorer API and CUR (historical data up to 14 months), this setup scales for billions in spend. Ready to optimize your €100k+ annual costs?

Prerequisites

  • AWS account with Billing and Organizations permissions (admin).
  • AWS CLI v2 installed and configured (aws configure).
  • Python 3.12+ with pip install boto3 pandas openpyxl.
  • Terraform 1.9+ for policies.
  • S3/Athena access for CUR (enable in Billing console > Cost Reports).

Install and Configure AWS CLI + boto3

setup.sh
#!/bin/bash

# Install AWS CLI v2 (macOS/Linux)
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Configure with MFA if needed
aws configure

# Install Python deps
pip install boto3 pandas openpyxl 'boto3-stubs[essential]'

# Verify
aws sts get-caller-identity
python -c "import boto3; print('boto3 OK')"

This complete bash script initializes AWS CLI and boto3 in one run. It handles installation without excessive sudo, sets up credentials (Access Key + Secret), and tests access. Pitfall: Skip MFA for prod – use IAM roles for EC2/Lambda instead.

Configure Cost and Usage Report (CUR)

Enable CUR in the AWS Billing console > Cost & Usage Reports: select a dedicated S3 bucket, Parquet/ORC format, include tags (up to 50). Delay: 24h for first data. This generates Athena tables like cur_table with columns line_item_usage_account_id, line_item_product_code, resource_tags_user_key.

Analogy: CUR is your AWS 'general ledger' – without tags, it's noise; with tags, it's gold for allocating 95% of costs.

Terraform: Mandatory Tag Policy

tags_policy.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "eu-west-1"
}

resource "aws_organizations_policy" "tag_policy" {
  name        = "RequireCostTags"
  description = "Oblige tags: Environment, Team, Project"
  type        = "TAG_POLICY"

  document = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid       = "DenyUntaggedResources"
        Effect    = "Deny"
        Resources = ["*"]
        Actions   = [
          "ec2:RunInstances",
          "rds:CreateDBInstance",
          "s3:CreateBucket"
        ]
        Condition = {
          Null = {
            "aws:RequestTag/Environment" = "true"
            "aws:RequestTag/Team"        = "true"
            "aws:RequestTag/Project"     = "true"
          }
          ForAnyValue:StringEquals = {
            "aws:TagKeys" = ["Environment", "Team", "Project"]
          }
        }
      }
    ]
  })
}

# Appliquer à OU
resource "aws_organizations_policy_attachment" "attach" {
  policy_id = aws_organizations_policy.tag_policy.id
  target_id = data.aws_organizations_organization.this.id
}

data "aws_organizations_organization" "this" {}

This Terraform module deploys an AWS Organizations Tag Policy that blocks creations without Environment, Team, Project tags. Run terraform init/apply from the management account. Pitfall: Tags propagate in 1-24h; test with EC2 deny. Covers 80% of AWS services.

Terraform: Example Tagged EC2 Resource

ec2_tagged.tf
resource "aws_instance" "example" {
  ami           = "ami-0c02fb55956c7d316" # Amazon Linux 2023 eu-west-1
  instance_type = "t3.micro"

  tags = {
    Name        = "cost-allocation-demo"
    Environment = "dev"
    Team        = "devops"
    Project     = "finops-tutorial"
  }

  # Security group minimal
  vpc_security_group_ids = [aws_security_group.ssh.id]
}

resource "aws_security_group" "ssh" {
  name_prefix = "ssh-demo"
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

output "instance_id" {
  value = aws_instance.example.id
}

Deploy this tagged EC2 via terraform apply: generates real allocable costs. Tags map to CUR; basic security group for SSH. Pitfall: Change AMI by region; delete after testing (terraform destroy) to avoid charges.

Extract Costs by Tag with boto3

Use Cost Explorer API: filter by TAG_KEY (e.g., 'Team'), groupBy for granularity. Limit: 500 results; paginate with NextPageToken. For full CUR, query Athena (next).

Python Script: Costs by Tag (Cost Explorer)

cost_by_tag.py
import boto3
import pandas as pd
from datetime import datetime, timedelta

ce = boto3.client('ce')

# Période : dernier mois
end = datetime.now()
start = end - timedelta(days=30)

# Tags à analyser
TAGS = ['Environment', 'Team', 'Project']

results = []
for tag in TAGS:
    resp = ce.get_cost_and_usage(
        TimePeriod={'Start': start.strftime('%Y-%m-%d'), 'End': end.strftime('%Y-%m-%d')},
        Granularity='MONTHLY',
        Metrics=['UnblendedCost'],
        GroupBy=[{'Type': 'DIMENSION', 'Key': 'SERVICE'}, {'Type': 'TAG', 'Key': tag}],
        Filter={'Tags': {'Key': tag, 'Values': ['*']}}
    )
    for group in resp['ResultsByTime'][0]['Groups']:
        results.append({
            'Tag': tag,
            'Service': group['Keys'][1] if len(group['Keys']) > 1 else 'N/A',
            'Value': group['Keys'][0],
            'Cost': float(group['Metrics']['UnblendedCost']['Amount'])
        })

df = pd.DataFrame(results)
df.to_excel('costs_by_tag.xlsx', index=False)
print(df.groupby('Tag').sum())

This script fetches monthly costs by tag/service via Cost Explorer and exports to Excel. Dynamic tag filtering; handles multi-groupBy. Pitfall: Enable Cost Explorer (console); costs >$1/month for visibility; rate limit 10 req/s.

Python Script: Advanced Team Allocation

allocate_teams.py
import boto3
import pandas as pd
from datetime import datetime, timedelta

ce = boto3.client('ce')
athena = boto3.client('athena')
s3 = boto3.client('s3')

# Config Athena (adaptez)
DB = 'cur_db'
TABLE = 'cur_table'
OUTPUT_S3 = 's3://your-cur-bucket/query_results/'

# Query SQL pour allocation Team
query = f"""
SELECT 
  line_item_usage_account_id,
  resource_tags_user_Environment,
  resource_tags_user_Team,
  SUM(line_item_unblended_cost) as total_cost,
  line_item_product_code
FROM {TABLE}
WHERE line_item_usage_start_date >= date('{ (datetime.now() - timedelta(days=30)).strftime('%Y-%m-%d')}')
  AND line_item_usage_start_date < date('{datetime.now().strftime('%Y-%m-%d')}')
  AND resource_tags_user_Team IS NOT NULL
GROUP BY 1,2,3,4
ORDER BY total_cost DESC
"""

# Exécuter query
resp = athena.start_query_execution(
    QueryString=query,
    QueryExecutionContext={'Database': DB},
    ResultConfiguration={'OutputLocation': OUTPUT_S3}
)
query_id = resp['QueryExecutionId']

# Poll status et fetch (simplifié)
import time
while True:
    status = athena.get_query_execution(QueryExecutionId=query_id)['QueryExecution']['Status']['State']
    if status in ['SUCCEEDED', 'FAILED', 'CANCELLED']:
        break
    time.sleep(5)

if status == 'SUCCEEDED':
    results = athena.get_query_results(QueryExecutionId=query_id)
    df = pd.DataFrame([row['Data'] for row in results['ResultSet']['Rows'][1:]])
    df.to_csv('team_allocation.csv', index=False)
    print('Rapport généré : team_allocation.csv')

# Coût total par team via CE pour validation
resp_ce = ce.get_cost_and_usage(
    TimePeriod={'Start': '2026-01-01', 'End': '2026-01-31'},
    Granularity='MONTHLY',
    Metrics=['UnblendedCost'],
    GroupBy=[{'Type': 'TAG', 'Key': 'Team'}]
)
print('Validation CE:', resp_ce)

Hybrid CE + Athena: queries CUR for detailed tags (not fully supported by CE), exports CSV. Adapt DB/TABLE after CUR setup. Pitfall: Parquet schema evolves; test query in Athena console first; query costs ~$0.005/TB scanned.

Athena SQL Query for CUR Allocation

cur_allocation.sql
-- Executable directly in Athena console
SELECT
  'Team' AS allocation_key,
  resource_tags_user_Team AS key_value,
  SUM(CASE WHEN line_item_product_code LIKE '%EC2%' THEN line_item_unblended_cost ELSE 0 END) AS ec2_cost,
  SUM(CASE WHEN line_item_product_code LIKE '%S3%' THEN line_item_unblended_cost ELSE 0 END) AS s3_cost,
  SUM(line_item_unblended_cost) AS total_cost,
  COUNT(*) AS resources_count
FROM cur_table
WHERE
  line_item_usage_start_date >= DATE '2026-01-01'
  AND line_item_usage_start_date < DATE '2026-02-01'
  AND resource_tags_user_Team IS NOT NULL
  AND line_item_line_item_type = 'Usage'
GROUP BY resource_tags_user_Team
HAVING total_cost > 0

UNION ALL

SELECT
  'Environment' AS allocation_key,
  resource_tags_user_Environment AS key_value,
  SUM(line_item_unblended_cost) AS total_cost,
  COUNT(*) AS resources_count
FROM cur_table
WHERE
  line_item_usage_start_date >= DATE '2026-01-01'
  AND line_item_usage_start_date < DATE '2026-02-01'
  AND resource_tags_user_Environment IN ('prod', 'dev')
GROUP BY resource_tags_user_Environment

ORDER BY allocation_key, total_cost DESC;

UNION query for multi-tags: segments EC2/S3 by Team/Env, Usage only filter. Copy-paste into Athena after CUR. Pitfall: Use linked_account for multi-account; add WHERE line_item_usage_type = 'BoxUsage:c6a.4xlarge' for instance granularity.

Best Practices

  • Standard tags: 3-5 max (Env, Team, Owner, App); use AWS Tag Editor for retro-tagging.
  • Automate: Monthly Lambda + CloudWatch on Python scripts + SNS alerts for budget overruns.
  • CUR optimization: Parquet + Glue partitioned by date; Athena Workgroups for query costs.
  • FinOps: Integrate QuickSight CUR dataset; set budgets by tag via Budgets API.
  • Multi-cloud: Replicate with GCP Labels API or Azure Tags.

Common Errors to Avoid

  • Tags not propagated: 24h delay; don't rely on real-time for allocation.
  • Incomplete CUR: Misses RI/SP; enable 'Include RI recommendations'.
  • Athena over-querying: Unnecessary TB scans; partition on dt=YYYY-MM-DD and use WHERE dt>='...'.
  • Permissions: IAM policy missing ce:GetCostAndUsage or athena:StartQueryExecution fails silently.

Next Steps