Skip to main content

AWS Bedrock

AWS Bedrock configuration​

AWS Bedrock provides access to foundation models from Amazon and third-party providers like Anthropic Claude, Meta Llama, and Mistral.

Prerequisites​

  1. An AWS account with Bedrock access enabled
  2. IAM credentials with permissions to invoke Bedrock models
  3. The desired models enabled in your AWS Bedrock console

Environment variables​

When running Faheem Code with the official Docker image, add these options to the documented docker run command:

--env LLM_AWS_ACCESS_KEY_ID="your-access-key-id" \
--env LLM_AWS_SECRET_ACCESS_KEY="your-secret-access-key" \
--env LLM_AWS_REGION_NAME="us-east-1"

The official ghcr.io/smart-national-solution/faheem-code-app:latest image includes the AWS SDK for Python (boto3).

UI configuration​

In Faheem Code:

  1. Open Settings > LLM and enable the Advanced options.
  2. Set Custom Model to the Bedrock model or inference profile ID. See Model IDs.
  3. Leave Base URL empty because Bedrock uses AWS endpoints automatically.
  4. Leave API Key empty because authentication is handled through your AWS credentials.
  5. Save the profile and start a new conversation to test it.

See Manage LLM Profiles for more information about profile settings.

Model IDs​

Bedrock model IDs are managed by AWS and may change over time. Use the exact Model ID from the AWS Console or the AWS documentation (no bedrock/ prefix).

Example format:

  • Custom Model: anthropic.claude-3-5-sonnet-20241022-v2:0

For a complete list of available models, see the AWS Bedrock documentation.

Cross-region inference​

Some models must be invoked through a cross-region inference profile rather than their direct foundation model ID. Inference profile IDs include a geographic prefix such as us..

For example, use:

  • Custom Model: us.anthropic.claude-sonnet-4-5-20250929-v1:0

instead of the direct model ID:

  • anthropic.claude-sonnet-4-5-20250929-v1:0

No additional environment variables are required. Keep using the AWS region where you configured Bedrock access and your existing credentials. See Increase throughput with cross-region inference for supported profiles and regions.

Using IAM roles (alternative to access keys)​

If running Faheem Code on AWS infrastructure such as EC2, ECS, or Lambda, you can use IAM roles instead of access keys:

  1. Attach an IAM role with Bedrock permissions to your compute resource.
  2. Omit the LLM_AWS_ACCESS_KEY_ID and LLM_AWS_SECRET_ACCESS_KEY environment variables.
  3. The AWS SDK automatically uses the instance role credentials.

Troubleshooting​

"No module named 'boto3'" error​

If you encounter this error:

litellm.APIConnectionError: No module named 'boto3'
ModuleNotFoundError: No module named 'boto3'

First identify how you installed Faheem Code:

  • Docker: The current ghcr.io/smart-national-solution/faheem-code-app:latest image includes boto3. Pull the latest image and recreate the container:

    docker pull ghcr.io/smart-national-solution/faheem-code-app:latest

  • npm or npx: The Python environment managed by the npm distribution may not include the optional Bedrock dependency. Follow Faheem Code issue #16578 for the package fix. Use the official Faheem Code Docker image if you need Bedrock while that issue remains open.

Do not install boto3 into a temporary uv archive environment because Faheem Code may recreate that environment.

On-demand throughput is not supported​

Some foundation model IDs cannot be invoked directly and return an error similar to:

Invocation of model ID ... with on-demand throughput isn't supported.
Retry your request with the ID or ARN of an inference profile that contains this model.

Use the corresponding inference profile ID or ARN, such as us.anthropic.claude-sonnet-4-5-20250929-v1:0. This error does not indicate a credential, model access, or boto3 problem.

Access denied errors​

If you receive access denied errors:

  1. Verify your IAM credentials have the bedrock:InvokeModel permission
  2. Check that the model is enabled in your AWS Bedrock console
  3. Ensure you're using the correct AWS region where the model is available

Model not found​

If the model is not found:

  1. Verify the model ID is correct (check AWS documentation)
  2. Ensure the model is enabled in your Bedrock model access settings
  3. Check that the model is available in your selected AWS region