Light Stemcell Builder for AWS

This tool takes a raw machine image and a configuration file and creates a collection of AMIs. Any AWS region including China is supported.

AWS Setup for Publishing

1. Create an S3 bucket for intermediate artifacts (e.g. light-stemcells-for-project-XXX)

2. Create an AWS IAM policy based on the JSON contained in builder-policy.json

3. Replace the bucket placeholder in your policy with the bucket created in step 1

     "Resource": [
   -    "arn:aws:s3:::<disk-image-file-bucket>",
   -    "arn:aws:s3:::<disk-image-file-bucket>/*"
   +    "arn:aws:s3:::light-stemcells-for-project-XXX",
   +    "arn:aws:s3:::light-stemcells-for-project-XXX/*"
     ]

   Note: The arn for AWS GovCloud region is aws-us-gov. It looks like this: "arn:aws-us-gov:s3:::<disk-image-file-bucket>"

4. Create an AWS IAM user and attach the policy created in steps 2, 3.

5. Create the vmimport AWS role as detailed here, specifying the previously created bucket in place of <disk-image-file-bucket>; see example IAM policy.

   • Updated docs are split over vm-import and roles now.

6. Replicate these steps in a separate AWS China account if publishing to China.

IAM User Setup for Integration Testing

1. Follow steps in “AWS Setup for Publishing”
2. Create an IAM policy based on the JSON contained in integration-test-policy.json
3. Attach the policy you created in step 2 to the existing publishing user

Testing

Unit testing:

ginkgo -r --skipPackage driver,integration

Example Usage

Example config:

{
  "ami_configuration": {
    "description":          "Your description here",
    "virtualization_type":  "hvm",
    "visibility":           "public",
    "tags" : {
      "distro":               "distro name, e.g. ubuntu-jammy",
      "version":              "e.g. 1.0.0"
    }
  },
  "ami_regions": [
    {
      "name":               "us-east-1",
      "credentials": {
        "access_key":       "US_ACCESS_KEY_ID",
        "secret_key":       "US_ACCESS_SECRET_KEY"
      },
      "bucket_name":        "US_BUCKET_NAME",
      "destinations":       ["us-west-1", "us-west-2"]
    },
    {
      "name":               "cn-north-1",
      "credentials": {
        "access_key":       "CN_ACCESS_KEY_ID",
        "secret_key":       "CN_ACCESS_SECRET_KEY"
      },
      "bucket_name":        "CN_BUCKET_NAME"
    }
  ]
}

Non-standard AWS partitions (custom endpoint domain)

Some AWS partitions use a different endpoint domain than the default amazonaws.com. For example, the AWS EU Sovereign Cloud (EUSC) uses amazonaws.eu.

Set endpoint_base on the region entry to override the endpoint domain for all services (EC2, S3, KMS):

{
  "ami_regions": [
    {
      "name":           "eusc-de-east-1",
      "endpoint_base":  "amazonaws.eu",
      "credentials": {
        "access_key":   "ACCESS_KEY_ID",
        "secret_key":   "ACCESS_SECRET_KEY"
      },
      "bucket_name":    "BUCKET_NAME"
    }
  ]
}

Service endpoints are constructed as https://<service>.<region>.<endpoint_base>, e.g. https://ec2.eusc-de-east-1.amazonaws.eu.

Usage:

./light-stemcell-builder -c config.json --image root.img --manifest stemcell.MF > updated-stemcell.MF

Example Output:

name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent
version: "3202"
bosh_protocol: "1"
sha1: f0c10bb5e8b7fee9c29db15bbb4ae481e398eab6
operating_system: ubuntu-trusty
stemcell_formats:
- aws-light
cloud_properties:
  ami:
    cn-north-1: ami-69ae6504
    us-east-1: ami-e62f158c
    us-west-1: ami-947e0df4
    us-west-2: ami-54328238

Troubleshooting

If the vmimport role is not present, you will receive this error from the light stemcell builder:

Error publishing AMIs to us-east-1: creating snapshot: creating import snapshot task: InvalidParameter: The sevice role does not exist or does not have sufficient permissions for the service to continue
status code: 400, request id: