Terraform apply hangs on state lock because CI role lacks dynamodb:PutItem on the lock table

2026-11-14 · 3 min read

Terraform apply hangs on state lock because CI role lacks dynamodb:PutItem on the lock table

Your pipeline passes plan cleanly, then stalls on apply. The log ends at "Acquiring state lock." No error. Ten minutes later the job times out, or an operator cancels it and reruns the same job, which now fails fast with a stale lock. Multiply that by every merge to main and you're burning CI minutes on runs that never had a chance of succeeding.

The root cause is usually a policy change nobody connected to Terraform: the CI role can read and write the state object in S3 but cannot write the lock item. DynamoDB denies PutItem, the backend retries until its retry budget runs out, and Terraform reports the lock as still held.

Who writes the lock item

The S3 backend acquires a lock by calling DynamoDB PutItem on the lock table with the state's key as the item key. plan and apply both do this. The table lives in the backend config, not in your module code, so it's absent from the diff reviewers see.

Typical paths to this failure:

Confirm it is the permissions, not a held lock

Run this before changing anything. A stale lock and a denied PutItem look identical in the console.

1. In the failing job's log, find the timestamp of "Acquiring state lock."

2. Open CloudTrail Event history, filter on the lock table name, and pull events within one minute of that timestamp.

3. Look for PutItem with the CI role's ARN. If errorCode reads AccessDenied, you've found it.

4. Rule out a real lock: have the lock holder release it with terraform force-unlock <LOCK_ID>, then re-run. If it hangs again, it's permissions.

5. Reproduce in isolation rather than against a production workspace: AWS_PROFILE=<ci-role-profile> terraform plan -lock-timeout=30s. The short timeout surfaces the retry-exhausted message instead of a long hang.

Fix Terraform DynamoDB state lock permissions

Grant the file-scoped actions to the CI role and nothing more:


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem",
        "dynamodb:DescribeTable"
      ],
      "Resource": "arn:aws:dynamodb:eu-west-1:123456789012:table/tfstate-locks"
    }
  ]
}

Two details matter. Scope the Resource to the exact table ARN — GetItem covers reading an existing lock and DeleteItem covers release, so PutItem alone leaves you with a lock nobody can clear. And when you shorten -lock-timeout for fast diagnosis, shorten it only in the debug run; keep the default for real applies so a transient hold doesn't fail a legitimate deploy.

Attach the policy, then re-run the same apply and watch for the run progressing past "Acquiring state lock."

When NOT to do this

If your lock table also backs other tooling — multiple Terraform roots, a Terragrunt setup, a separate platform team — don't widen this role to cover all of them. Add a per-table statement, or split the tables per root, so one over-broad grant doesn't quietly become the CI role's permanent permission set.

Also skip this fix if your state backend has moved to S3 native locking (use_lockfile = true). Native locking uses the S3 bucket and a conditional write, not a DynamoDB table. Extra DynamoDB permissions for a table nothing writes to just widen the role for no reason.

Checklist

[link needed]