AWS IAM Awards: the best (and the worst) AWS Services
To celebrate another AWS Re:Invent, we are launching the IAM awards for AWS service teams, namely:
- “Our TPM is AWSome” award
- “My first DeepRacer model” award
- “We walk the talk” award
- “BuT dEpENDenCIeS” award
- “I like to spoil the surprise” award
As you will see, those awards can be acted upon. So we won’t be talking about API Gateway (doing its own thing, whatever it is) or Neptune + DocumentDB (using the rds namespace, because why bother).
We will be focusing on IAM SAR. See, each AWS service documents its IAM permissions in the IAM Service Authorization Reference (or SAR). Like many, we are using the IAM SAR to help our customers (doing things like Amazon S3 ThreatModel), and many AWS customers are using SAR directly via community automation (policy_sentry, parliament, permissions.cloud, trackiam, or our very own awsiamchanges.com).
AWS Service teams generally do a good job updating their IAM SAR page at each release: 69% get it done in less than 7 days. But some don’t (and poorly), and we hope that an AWS exec will see this and starts to ask questions like:
- “can we launch IAM updates consistently?”
- “can we update the IAM policy generator at the same time?”
- leading to “how can we have an API already?”
The response should include “available to the public” (please).
The fastest services to update IAM SAR — "Our TPM is AWSome" award
The “Our TPM is AWSome” award is for the high-velocity IAM SAR delivery, based on the average of the amounts of times the API releases have taken per service in the past year:
- Winner: Transcribe, with the average time-to-update, is a lightning fast 3 hours and 26 minutes.
- Runner Up: DataExchange, with the average time-to-update of 14 hours and 54 minutes.
# | Services | Average | Number of releases |
---|---|---|---|
1 | transcribe | 3 hours, 26 minutes | 10 |
2 | dataexchange | 14 hours, 54 minutes | 3 |
3 | codebuild | 1 day, 4 hours, 55 minutes | 5 |
4 | workmail | 1 day, 5 hours, 37 minutes | 2 |
5 | nimble | 1 day, 5 hours, 40 minutes | 3 |
6 | redshift-data | 1 day, 6 hours, 8 minutes | 2 |
7 | imagebuilder | 1 day, 9 hours, 27 minutes | 2 |
8 | backup | 1 day, 10 hours, 5 minutes | 5 |
9 | frauddetector | 1 day, 19 hours, 17 minutes | 5 |
10 | kms | 1 day, 21 hours, 30 minutes | 3 |
The worst services to update IAM SAR — "My first DeepRacer model" award
Remember when I had this idea for my DeepRacer model, that sounded amazing (especially in the after-hours of Re:Invent). And then my hopes were crushed. The following award-winning services seem to be in the wall as well:
# | Services | Average | Number of releases |
---|---|---|---|
94 | mediapackage | 107 days, 12 hours, 34 minutes | 2 |
93 | medialive | 102 days, 14 hours, 46 minutes | 6 |
92 | codepipeline | 102 days, 8 hours, 35 minutes | 3 |
91 | datasync | 97 days, 14 hours, 38 minutes | 5 |
90 | iotevents | 95 days, 16 hours, 16 minutes | 2 |
89 | personalize | 84 days, 12 hours, 11 minutes | 2 |
88 | appstream | 58 days, 1 hour, 36 minutes | 3 |
87 | directconnect | 53 days, 1 hour, 57 minutes | 2 |
86 | snowball | 50 days, 18 hours, 37 minutes | 2 |
85 | kendra | 48 days, 3 hours, 42 minutes | 9 |
84 | wafv2 | 43 days, 4 hours, 27 minutes | 10 |
Special — "We walk the talk" award
A special shout-out to SecurityHub for launching new APIs and IAM actions to remove “master”.
Honorable mention: KMS, for introducing new API parameters + IAM condition by removing the word Customer Master Key. Looking forward to the deprecation notice on the older parameters/conditions!
Special — "BuT dEpENDenCIeS" award
It is almost 2022, and those Services still have non inclusive wording in their APIs, IAM or docs:
- RDS (Master user)
- RedShift (Master username)
- EMR (Master nodes)
- AWS Control Tower (Master name in the stack)
- OpenSearch (Master nodes): This 6-month-old issue needs support.
- Organizations (Master Account): Fixing documentation (poorly) is not enough.
Special "I like to spoil the surprise" award
Congrats to Amazon Inspector V2 (non-Classic) for announcing itself early. Our bet is on Adam talking of “fast innovation, security is priority number 1, and always improving our services”, before dropping the news.
Appendix - Methodology and full list for IAM SAR Awards
This data is a result of a compiled list of API releases, and IAM permissions releases, sorted chronologically. The results start from the best and fastest service teams and going all the way to the service teams missing out on the IAM permissions on time, or still pending. The results exclude the services with only a single release in the past year.
Below is the full list for all AWS services that launch API releases, followed by subsequent IAM release, along with the average of the time taken between those releases:
# | Services | Average | Number of releases |
---|---|---|---|
1 | transcribe | 3 hours, 26 minutes | 10 |
2 | dataexchange | 14 hours, 54 minutes | 3 |
3 | codebuild | 1 day, 4 hours, 55 minutes | 5 |
4 | workmail | 1 day, 5 hours, 37 minutes | 2 |
5 | nimble | 1 day, 5 hours, 40 minutes | 3 |
6 | redshift-data | 1 day, 6 hours, 8 minutes | 2 |
7 | imagebuilder | 1 day, 9 hours, 27 minutes | 2 |
8 | backup | 1 day, 10 hours, 5 minutes | 5 |
9 | frauddetector | 1 day, 19 hours, 17 minutes | 5 |
10 | kms | 1 day, 21 hours, 30 minutes | 3 |
11 | chime-sdk-messaging | 2 days, 49 minutes | 2 |
12 | rds | 2 days, 1 hour, 55 minutes | 14 |
13 | ecr | 2 days, 2 hours, 29 minutes | 3 |
14 | rekognition | 2 days, 5 hours, 36 minutes | 6 |
15 | license-manager | 2 days, 5 hours, 39 minutes | 4 |
16 | comprehend | 2 days, 8 hours, 55 minutes | 8 |
17 | amp | 2 days, 12 hours, 29 minutes | 3 |
18 | servicecatalog | 2 days, 14 hours, 31 minutes | 5 |
19 | iotwireless | 2 days, 14 hours, 53 minutes | 3 |
20 | fsx | 2 days, 16 hours, 49 minutes | 8 |
21 | s3control | 2 days, 17 hours, 27 minutes | 6 |
22 | kafka | 2 days, 20 hours, 42 minutes | 7 |
23 | acm-pca | 3 days, 2 hours, 9 minutes | 3 |
24 | auditmanager | 3 days, 2 hours, 57 minutes | 2 |
25 | s3 | 3 days, 3 hours, 49 minutes | 10 |
26 | proton | 3 days, 8 hours, 1 minute | 2 |
27 | mediatailor* | 3 days, 16 hours, 37 minutes | 4 |
28 | globalaccelerator | 4 days, 34 minutes | 2 |
29 | organizations | 4 days, 34 minutes | 2 |
30 | sns | 4 days, 34 minutes | 2 |
31 | logs | 4 days, 34 minutes | 2 |
32 | guardduty | 4 days, 34 minutes | 2 |
33 | iam | 4 days, 34 minutes | 2 |
34 | shield | 4 days, 34 minutes | 2 |
35 | sts | 4 days, 34 minutes | 2 |
36 | resourcegroupstaggingapi | 4 days, 34 minutes | 2 |
37 | apprunner | 4 days, 34 minutes | 2 |
38 | cloudwatch | 4 days, 34 minutes | 3 |
39 | dynamodb | 4 days, 34 minutes | 3 |
40 | sso | 4 days, 34 minutes | 3 |
41 | secretsmanager | 4 days, 34 minutes | 3 |
42 | ds* | 4 days, 34 minutes | 3 |
43 | codeartifact | 4 days, 34 minutes | 3 |
44 | route53 | 4 days, 34 minutes | 3 |
45 | fms | 4 days, 34 minutes | 4 |
46 | lightsail | 4 days, 34 minutes | 4 |
47 | config | 4 days, 34 minutes | 4 |
48 | elbv2 | 4 days, 34 minutes | 4 |
49 | devops-guru | 4 days, 34 minutes | 4 |
50 | managedblockchain | 4 days, 34 minutes | 4 |
51 | polly | 4 days, 34 minutes | 5 |
52 | storagegateway | 4 days, 34 minutes | 6 |
53 | gamelift | 4 days, 34 minutes | 6 |
54 | ssm | 4 days, 34 minutes | 7 |
55 | eks | 4 days, 34 minutes | 9 |
56 | lambda | 4 days, 34 minutes | 10 |
57 | iot | 4 days, 10 hours, 22 minutes | 7 |
58 | sagemaker* | 4 days, 14 hours, 51 minutes | 28 |
59 | securityhub | 4 days, 16 hours, 19 minutes | 2 |
60 | events | 5 days, 9 hours, 7 minutes | 7 |
61 | efs | 5 days, 11 hours, 49 minutes | 4 |
62 | connect | 5 days, 22 hours, 18 minutes | 7 |
63 | sagemaker-runtime | 6 days, 2 hours, 10 minutes | 8 |
64 | sesv2 | 7 days, 7 hours, 58 minutes | 2 |
65 | route53resolver | 7 days, 10 hours, 28 minutes | 5 |
66 | redshift | 8 days, 11 hours, 50 minutes | 7 |
67 | vpc | 12 days, 22 hours, 25 minutes | 3 |
68 | cloudformation | 13 days, 2 hours, 40 minutes | 7 |
69 | chime | 13 days, 13 hours, 29 minutes | 7 |
70 | textract | 13 days, 18 hours, 36 minutes | 3 |
71 | ec2 | 14 days, 7 hours, 19 minutes | 36 |
72 | cloudfront | 14 days, 14 hours, 36 minutes | 8 |
73 | workspaces | 15 days, 14 hours, 17 minutes | 6 |
74 | mediaconnect | 16 days, 9 hours, 17 minutes | 5 |
75 | compute-optimizer | 17 days, 19 hours, 42 minutes | 4 |
76 | mediaconvert | 18 days, 5 hours, 23 minutes | 5 |
77 | quicksight | 26 days, 5 hours, 46 minutes | 6 |
78 | emr | 30 days, 1 hour, 19 minutes | 3 |
79 | servicediscovery | 30 days, 10 hours, 57 minutes | 2 |
80 | macie2 | 31 days, 3 hours, 27 minutes | 6 |
81 | outposts | 33 days, 3 hours, 8 minutes | 2 |
82 | iotsitewise | 33 days, 15 hours, 6 minutes | 7 |
83 | autoscaling | 34 days, 19 hours, 19 minutes | 10 |
84 | wafv2 | 43 days, 4 hours, 27 minutes | 10 |
85 | kendra | 48 days, 3 hours, 42 minutes | 9 |
86 | snowball | 50 days, 18 hours, 37 minutes | 2 |
87 | directconnect | 53 days, 1 hour, 57 minutes | 2 |
88 | appstream | 58 days, 1 hour, 36 minutes | 3 |
89 | personalize | 84 days, 12 hours, 11 minutes | 2 |
90 | iotevents | 95 days, 16 hours, 16 minutes | 2 |
91 | datasync | 97 days, 14 hours, 38 minutes | 5 |
92 | codepipeline | 102 days, 8 hours, 35 minutes | 3 |
93 | medialive | 102 days, 14 hours, 46 minutes | 6 |
94 | mediapackage | 107 days, 12 hours, 34 minutes | 2 |
(*) Services with API releases in the past 6 months but no IAM SAR update on the API