AWS CloudTrail

AWS CloudTrail records every API call made within an AWS account — from console clicks to SDK invocations to internal service-to-service calls. It is the primary forensic, audit, and compliance log for AWS, and the source-of-truth for answering "who did what, when, from where" across your cloud footprint.

Key Features:

Management Events: Control-plane operations (create/modify/delete on resources). Enabled by default; the last 90 days are queryable in the CloudTrail console for free.
Data Events: Data-plane operations such as S3:GetObject, Lambda:Invoke, and DynamoDB item-level reads/writes. Off by default — extra cost, very high volume.
Insights Events: Anomaly detection on management API call rates and error rates; surfaces unusual surges (e.g., a sudden spike in RunInstances).
Trails: Persistent delivery of events to an S3 bucket and optionally CloudWatch Logs; multi-region and organization trails fan in across all regions and accounts.
CloudTrail Lake: Managed event data store with SQL-based queries — no S3 + Athena setup needed; retention up to 10 years.
Log File Integrity Validation: SHA-256 digests signed by AWS prove logs haven't been tampered with.
EventBridge Integration: Real-time routing of management events to alarms, Lambda, or SIEMs.

Common Use Cases:

Security Forensics: Reconstruct a breach timeline — which IAM principal called CreateAccessKey for the compromised user, from which IP, when?
Compliance Evidence: SOC 2, HIPAA, PCI-DSS auditors expect tamper-proof audit trails of administrative actions.
Operational Troubleshooting: Identify the change that broke production — usually a recent UpdateStack, PutBucketPolicy, or ModifySecurityGroup.
Threat Detection: Feed CloudTrail into Amazon GuardDuty, Security Lake, or third-party SIEMs for IOC matching.
Cost & Drift Investigation: Pair with Cost Explorer to attribute unexpected spend to specific provisioning events.

Trail Types:

Single-region trail: Legacy — captures only one region's events. Avoid.
Multi-region trail: Recommended baseline — captures every region the account uses, including future regions.
Organization trail: Created in the management account; aggregates all member accounts to a central S3 bucket. The standard pattern for AWS Organizations.

Service Limits & Quotas:

Trails per region per account: 5 (hard).
Event history (free): 90 days of management events queryable in the console without a trail.
Event size: max 256 KB per event.
CloudTrail Lake retention: 7 days to 10 years.
Log delivery latency: typically within 5 minutes; SLA up to 15 minutes.

Pricing Model:

Management events: first copy delivered to S3 is free; additional trails $2 per 100,000 events.
Data events: $0.10 per 100,000 events delivered to S3 (these add up fast for chatty S3/Lambda workloads).
Insights events: $0.35 per 100,000 events analyzed.
CloudTrail Lake: $2.50 per GB ingested + $0.005 per GB scanned by queries (similar to Athena).
Common cost surprise: Enabling S3 data events on a high-traffic bucket — billions of GetObject calls turn into a five-figure monthly bill. Scope data event selectors to only the buckets and prefixes you actually need to audit.
S3 storage of log files is billed separately; lifecycle to Glacier after 90 days for long retention.

Code Example — Create an Org-Wide Multi-Region Trail:


aws cloudtrail create-trail \
  --name org-audit-trail \
  --s3-bucket-name central-audit-logs-111122223333 \
  --is-multi-region-trail \
  --is-organization-trail \
  --enable-log-file-validation \
  --kms-key-id arn:aws:kms:us-west-2:111122223333:key/abcd-ef01

aws cloudtrail start-logging --name org-audit-trail

# Add S3 data events for one sensitive bucket only
aws cloudtrail put-event-selectors \
  --trail-name org-audit-trail \
  --advanced-event-selectors '[{
    "Name": "Sensitive S3 reads",
    "FieldSelectors": [
      {"Field": "eventCategory", "Equals": ["Data"]},
      {"Field": "resources.type", "Equals": ["AWS::S3::Object"]},
      {"Field": "resources.ARN", "StartsWith": ["arn:aws:s3:::pii-vault/"]}
    ]
  }]'

Query a CloudTrail Lake Event Data Store:


SELECT eventTime, userIdentity.arn, sourceIPAddress, eventName
FROM ABCD-1234-EVENT-DATA-STORE-ID
WHERE eventName = 'ConsoleLogin'
  AND responseElements.ConsoleLogin = 'Failure'
  AND eventTime > '2026-04-18T00:00:00Z'
ORDER BY eventTime DESC
LIMIT 100;

Common Interview Questions:

What's the difference between management events and data events?

Management events are control-plane (create, modify, delete) and free for the first copy. Data events are data-plane reads/writes on resources like S3 objects and Lambda invokes — disabled by default because the volume and cost are large. Most production accounts log all management events globally and only data events on sensitive resources.

How do you make CloudTrail logs tamper-evident?

Enable log file integrity validation, send logs to a write-once S3 bucket in a dedicated audit account with object lock and MFA delete, and restrict bucket policy to only the CloudTrail service principal. SCPs prevent member-account admins from disabling the org trail.

CloudTrail Lake vs. S3 + Athena — which to use?

Lake is faster to set up (no Glue catalog, partitions, or schema), supports up to 10 years retention with built-in queries, and bills per GB ingested + scanned. S3 + Athena is cheaper at very high event volumes if you already have a lake architecture. Pick Lake unless you have an existing data lake pipeline.

How quickly do CloudTrail events appear?

Typically within 5 minutes of the API call, with a documented SLA of up to 15 minutes for delivery to S3. Real-time alerting requires routing through EventBridge instead of waiting for log files.

What's an organization trail and why does it matter?

A trail created in the AWS Organizations management account that automatically applies to every member account, present and future. Member accounts cannot disable it. This is the standard control for ensuring no account in the org operates without audit logging.

What's the difference between CloudTrail and CloudWatch Logs?

CloudTrail records AWS API calls (who did what to AWS resources). CloudWatch Logs collects application and OS logs (what the workloads themselves printed). They overlap when CloudTrail is configured to also stream events into a CloudWatch Logs group for alarming.

CloudTrail is the non-negotiable audit foundation. Configure an organization-wide multi-region trail with log file validation on day one, restrict who can disable it via SCP, and add data events surgically where the data sensitivity justifies the cost.