AWS
06 Apr 2017think of some name that you would use throughout all the services that identifies your project - say, foo. then name Amazon resources as follows:
- bucket: foo.bucket
- policy: foo.policy
- user: foo.user
account (aka AWS account)
create Amazon account if you don’t have one (this is a single account for all Amazon services including shop itself).
account identifiers
<your_name> (menu in top right corner) → My Security Credentials |
Accound Identifiers |
each AWS account is assigned 2 unique IDs:
- AWS account ID (used to construct ARNs)
- canonical user ID (used for cross-account access to S3 bucket)
generate access keys
<your_name> (menu in top right corner) → My Security Credentials |
Access Keys (Access Key ID and Secret Access Key) → Create New Access Key (button) |
these keys (just like the keys of any service created in IAM service) can be used to make programmatic requests to AWS services (say, to S3 service).
NOTE: save generated credentials (Access Key ID
and Secret Access key
) -
secret access key won’t be available later (just like in IAM service).
S3
bucket names
https://docs.rightscale.com/faq/clouds/aws/What_are_valid_S3_bucket_names.html
Although Amazon will allow you to use capital letters and periods in the namespace, it is not recommended because of the naming restrictions that are enforced by DNS. In order to conform with DNS requirements, we recommend following these additional guidelines when creating buckets:
- Bucket names should not contain upper-case letters
- Bucket names should not contain underscores (_)
- Bucket names should not end with a dash
- Bucket names should be between 3 and 63 characters long
- Bucket names cannot contain dashes next to periods (my-.bucket.com)
- Bucket names cannot contain periods
access bucket
-
virtual-hosted-style URL:
http://foo.bucket.s3.eu-central-1.amazonaws.com
http://foo.bucket.s3-eu-central-1.amazonaws.com
-
path-style URL:
http://s3.eu-central-1.amazonaws.com/foo.bucket
http://s3-eu-central-1.amazonaws.com/foo.bucket
both ways are equivalent except that when using virtual-hosted-style URLs:
- it’s possible to omit region name (
http://foo.bucket.s3.amazonaws.com
) - request will be routed to appropriate region where bucket resides - bucket name must be DNS-compliant
create bucket
1) Name and region
- Bucket name: foo.bucket (must be unique across all buckets names in S3)
- Region: EU (Frankfurt) (closest to your location)
2) Set properties
leave default settings (Versioning
and Logging
are disabled).
3) Set permissions
give required permissions to users or groups (for Object access
only).
permissions can also be managed after bucket creation:
<bucket> → Permissions (tab) → Access Control List (button) |
giving all permissions to Everyone
group still doesn’t allow
anonymous user to access the bucket (neither read nor write) -
set bucket policy (see below) if you need to make bucket public.
TODO: why?
grant read-only permission to anonymous user
<bucket> → Permissions (tab) → Bucket Policy (button) |
paste policy (specify actual bucket ARN) and save it:
{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"AddPerm",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::foo.bucket/*"]
}
]
}
IAM
create policy
Policies (left sidebar) → Create Policy (button) |
1) Step 1: Create Policy
select Create Your Own Policy
.
2) Step 2: Set permissions
omitted when creating your own policy.
3) Step 3: Review Policy
Policy Name
: foo.policyPolicy Document
: policy itself (specify actual bucket ARN)
create user
Users (left sidebar) → Add user (button) |
1) Details
User name
: foo.userAccess type
: [x]Programmatic access
2) Permissions
- select
Attach existing policies directly
- find and check foo.policy
3) Review
4) Complete
NOTE: save generated credentials (Access Key ID
and Secret Access key
) -
secret access key won’t be available later.
awscli
-
install
awscli
$ brew install awscli
-
configure
aws
select the same region as for S3 bucket.
$ aws configure AWS Access Key ID: *** AWS Secret Access Key: *** Default region name: eu-central-1 Default output format: None
-
test copying to S3 bucket
$ aws s3 cp test.txt s3://foo.bucket/ upload: ./test.txt to s3://foo.bucket/test.txt $ aws s3 ls s3://foo.bucket/ 2017-04-07 03:14:59 4 test.txt $ aws s3 rm s3://foo.bucket/test.txt delete: s3://foo.bucket/test.txt
troubleshooting
error saving bucket policy (the policy must contain a valid version string)
The policy must contain a valid version string
policy versions are not arbitrary strings - they are predefined by AWS.
A conflicting conditional operation is currently in progress against this resource. Please try again.
This error usually occurs when a Bucket is deleted and a new bucket is created in the same name as the old bucket. I believe we would need to wait for certain amount of time until we can create a new bucket in the same name.
About an hour later, my attempt to create the bucket succeeded.