Exposing the Weakness: How We Identified a Flaw in Bedrock's Foundation Model Access Control
During research into AWS Marketplace, we found a flaw in how the aws-marketplace:ProductId condition key worked for Bedrock foundation models. AWS confirmed a fix has been implemented.
Service-specific condition keys such as aws-marketplace:ProductId can be used in IAM policies and SCPs to prevent principals from subscribing to unauthorized foundation models in Amazon Bedrock. During our research, we found a flaw where this condition didn't work for certain models, allowing unauthorized subscriptions and exposing organizations to compliance and financial risk.
AWS Security confirmed a fix has been implemented, affected customers were notified, and additional pipeline testing was added to prevent similar issues in the future.
aws-marketplace:ProductId condition key now operates as intended for all Bedrock foundation models.Introduction
At TrustOnCloud, we threat model cloud services. By diving deep into each service, we often find interesting or abnormal behaviors affecting the service we are researching, or other dependent services. In this post, we'll discuss how, during the creation of our AWS Marketplace threat model, we found a flaw in how the aws-marketplace:ProductId condition key behaved, and how it could allow unauthorized access to Bedrock's foundation models in an organization.
Understanding Bedrock's Foundation Models
Foundation models are large, pre-trained machine learning models that can be adapted or fine-tuned for many tasks without training from scratch. Amazon Bedrock supports various providers including Anthropic, Stability, and Meta, across text, chat, and image generation modalities.
Since each model has specific capabilities, specialties, and pricing, there are risks associated with using them:
- Denial of wallet: Depending on the model and usage, costs can scale dramatically. Some models cost 940x more than others. As shown by Sysdig's LLMJacking research, the worst case can exceed $46,000 per day.
- Legal exposure: See Section 50.10 in AWS Service Terms or each model's EULA, linked next to the model in the "Model access" section of the Bedrock console.
Access to models is not granted by default. An IAM entity with sufficient permissions must request access through the console. The corresponding permission is aws-marketplace:Subscribe, and the condition key aws-marketplace:ProductId can be used to limit access to authorized foundation models.
To make models available, go to Amazon Bedrock's "Model access" section and click "Modify model access":
Internally, this subscribes to the model in AWS Marketplace and generates CloudTrail events for AcceptAgreementRequest and CreateFoundationModelAgreement:
<omitted for brevity>
"eventSource": "agreement-marketplace.amazonaws.com",
"eventName": "AcceptAgreementRequest",
"awsRegion": "us-east-1",
"sourceIPAddress": "AWS Internal",
"userAgent": "Coral/Netty4",
"requestParameters": {
"agreementRequestId": "ar-din9hncr70g4tbfuoq6apj4j0"
},
"responseElements": {
"agreementId": "agmt-108pnb96uq2qghxxxx9m4o3m"
},
<omitted for brevity>
<omitted for brevity>
"eventSource": "bedrock.amazonaws.com",
"eventName": "CreateFoundationModelAgreement",
"awsRegion": "us-east-1",
"sourceIPAddress": "xxx.xxx.xxx.xxx",
"userAgent": "xxxxxxxx",
"requestParameters": {
"modelId": "meta.llama2-13b-chat-v1"
},
"responseElements": {
"modelId": "meta.llama2-13b-chat-v1"
},
"requestID": "a48383a5-8a6a-4878-8377-66b25e38964a",
<omitted for brevity>
Understanding AWS Marketplace
AWS Marketplace offers a curated digital catalog allowing customers to find, buy, and immediately use software and services within their AWS environment. To ensure only authorized products are used, IAM policies can implement the condition key aws-marketplace:ProductId, which filters access to specific products.
AWS documentation shows an example of how to do this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow|Deny",
"Action": [
"aws-marketplace:Subscribe"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws-marketplace:ProductId": [
"model-product-id-1",
"model-product-id-2",
"..."
]
}
}
},
{
"Effect": "Allow|Deny",
"Action": [
"aws-marketplace:Unsubscribe",
"aws-marketplace:ViewSubscriptions"
],
"Resource": "*"
}
]
}
The Issue: productId Condition Key
After testing the condition in Bedrock's foundation models, we discovered that the productId condition key in AWS Marketplace was not functioning as expected for all product IDs. Specifically, when we attempted to use this condition to deny access to specific foundation models, the policy ignored the intended restrictions.
Testing
The behavior looked inconsistent at first, with no clear pattern. We ran limited tests to understand what was happening.
Consider the following deny policy for 4 of the available models:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Action": ["aws-marketplace:Subscribe"],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws-marketplace:ProductId": [
"1d288c71-65f9-489a-a3e2-9c7f4f6e6a85",
"cc0bdd50-279a-40d8-829c-4009b77a1fcc",
"216b69fd-07d5-4c7b-866b-936456d68311",
"prod-2lvuzn4iy6n6o"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"aws-marketplace:Subscribe",
"aws-marketplace:Unsubscribe",
"aws-marketplace:ViewSubscriptions",
"bedrock:CreateFoundationModelAgreement",
"bedrock:DeleteFoundationModelAgreement",
"bedrock:GetFoundationModel",
"bedrock:GetFoundationModelAvailability",
"bedrock:ListFoundationModels",
"bedrock:ListFoundationModelAgreementOffers"
],
"Resource": "*"
}
]
}
| productId | Foundation Model |
|---|---|
1d288c71-65f9-489a-a3e2-9c7f4f6e6a85 | Jurassic-2 Mid (Amazon Bedrock Edition) |
cc0bdd50-279a-40d8-829c-4009b77a1fcc | Jurassic-2 Ultra (Amazon Bedrock Edition) |
216b69fd-07d5-4c7b-866b-936456d68311 | Cohere Generate Model – Command-Light (Amazon Bedrock Edition) |
prod-2lvuzn4iy6n6o | SDXL V1.0 (Amazon Bedrock Edition) |
| Foundation Model | Expected | Actual |
|---|---|---|
| Jurassic-2 Mid | Deny | Deny |
| Jurassic-2 Ultra | Deny | Deny |
| Cohere Generate Model – Command-Light | Deny | ⚠ Allow |
| SDXL V1.0 | Deny | ⚠ Allow |
Jurassic-2 models respect the deny. Cohere and SDXL don't: the condition is being ignored for those product IDs.
To confirm this wasn't specific to deny policies, we tested the same policy using "Allow":
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["aws-marketplace:Subscribe"],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws-marketplace:ProductId": [
"1d288c71-65f9-489a-a3e2-9c7f4f6e6a85",
"cc0bdd50-279a-40d8-829c-4009b77a1fcc",
"216b69fd-07d5-4c7b-866b-936456d68311",
"prod-2lvuzn4iy6n6o"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"aws-marketplace:Unsubscribe",
"aws-marketplace:ViewSubscriptions",
"bedrock:CreateFoundationModelAgreement",
"bedrock:DeleteFoundationModelAgreement",
"bedrock:GetFoundationModel",
"bedrock:GetFoundationModelAvailability",
"bedrock:ListFoundationModels",
"bedrock:ListFoundationModelAgreementOffers"
],
"Resource": "*"
}
]
}
| Foundation Model | Expected | Actual |
|---|---|---|
| Jurassic-2 Mid | Allow | Allow |
| Jurassic-2 Ultra | Allow | Allow |
| Cohere Generate Model – Command-Light | Allow | ⚠ Deny (not allowed by policy) |
| SDXL V1.0 | Allow | ⚠ Deny (not allowed by policy) |
The error message confirmed no policy was allowing aws-marketplace:Subscribe for those two models, even though they were explicitly listed in the allow policy:
What Is Happening?
We ran additional tests to understand the pattern:
- Testing the condition with services other than Bedrock and Marketplace: the condition was still ignored for the same product IDs.
- Checking for size limits by varying position of product IDs in the policy: the same strings were acknowledged or ignored regardless of position.
- Using
ForAnyValue:StringLikewith wildcard matching: the same strings were acknowledged or ignored.
After completing the full list of product IDs from AWS documentation and testing them all, a clear pattern emerged:
The condition worked for some models and was completely ignored for others. The full list of affected product IDs for which the condition was ignored:
- 216b69fd-07d5-4c7b-866b-936456d68311Cohere Generate Model – Command-Light (Amazon Bedrock Edition)
- b7568428-a1ab-46d8-bab3-37def50f6f6aCohere Embed Model – English (Amazon Bedrock Edition)
- 38e55671-c3fe-4a44-9783-3584906e7cadCohere Embed Model – Multilingual (Amazon Bedrock Edition)
- prod-ariujvyzvd2qyMeta Llama 2 Chat 13B (Amazon Bedrock Edition)
- prod-2lvuzn4iy6n6oSDXL V1.0 (Amazon Bedrock Edition)
Why It Matters
In restricted environments, it's often expected that image generation models are blocked due to denial-of-wallet or content risks. With this flaw, SDXL V1.0 would have been subscribed to when it shouldn't have been. As Sysdig's research showed, letting these models run ungoverned can be extremely costly.
Conclusion
The discovery of this flaw highlights the importance of testing and continuous monitoring of cloud security configurations. When using IAM policies to restrict access, especially with condition keys, it's critical to run both positive and negative test cases to confirm the desired behavior is actually in place.
To prevent subscription and invocation of unauthorized foundation models, there are two proposed mitigations:
Preventing subscription via AWS Marketplace
- Use the
aws-marketplace:ProductIdcondition in IAM policies or SCPs for the permissionaws-marketplace:Subscribe(now works as intended per AWS). - Enforce AWS Private Marketplace in your organization, allowing only authorized foundation models.
Preventing invocation
- Use an IAM policy with
bedrock:InvokeModelandbedrock:InvokeModelResponseStream, filtered by specific model ARNs in the resource field.
Vendor Notification Timeline
"The issue resulted in the 'aws-marketplace:ProductId' condition key being enforced incorrectly for certain third-party Bedrock models. As a result, a user in your account could successfully subscribe to 'Llama 2 Chat 13B' and 'Llama 2 13B' Bedrock models when the subscription request should have failed. On January 18, 2024, AWS resolved the issue and can confirm the service is operating as expected."
Additionally, AWS recommended reviewing the IAM policy in the Bedrock user guide and removing access to Llama 2 models if the intention was to deny subscription.
Additional Report: Marketplace Bypass Behavior
Before publishing this post, we reported an additional behavior to ensure AWS was aware. If you know a foundation model's product ID in AWS Marketplace, you can subscribe directly in Marketplace, bypassing the Bedrock-specific permissions (bedrock:CreateFoundationModelAgreement and bedrock:PutFoundationModelEntitlement), as long as the IAM entity has aws-marketplace:Subscribe permission (with the correct productId condition if used).
We reported this to AWS Security on May 8th. Their response:
"We do not believe the behavior you describe in this report presents a security concern, rather, it is expected behavior. To control a role's permissions to request access to Amazon Bedrock models, attach an IAM policy to the role using any of the AWS Marketplace actions mentioned in the documentation. For the
aws-marketplace:Subscribeaction only, you can use theaws-marketplace:ProductIdcondition key to limit subscription to specific models."
