How to become an infrastructure-as-code ninja, using AWS CDK - part 2
If you have not read part 1 in this series of articles, you may want to have a look at that first. , Otherwise, let us get started with defining some AWS infrastructure!
The first infrastructure - virtual machine
First of all we should create an AWS CDK project. You should already have the CDK command-line tool installed. If not, go to the section about AWS CDK installation (in part 1) first to install the tool, then get back here. We will wait!
Our goals
First, let us clarify what we want to accomplish here:
- An EC2 instance
- The instance should run Linux, we will pick Amazon Linux for simplicity
- The instance should not be reachable from the internet
- We should be able to login and access the machine from a command-line prompt in the AWS Console.
- We should use an existing VPC and its subnets.
- We do not care about which availability zone the machine ends up in.
However, we will take smaller steps to reach this goal. Our first goal is simply to get an EC2 instance of any type up and running, in any VPC, subnet, availability zone - just get it up.
Let us see how we can accomplish these goals! We will run into some trouble on the way, which is intentional. There are a few hurdles and learnings to get started with AWS CDK. If you follow along, we should get through them one by one!
This article will be a bit long, since there are a few steps to get started with the AWS CDK. So hang in there, and we will get through all of these steps.
Update:
This article series uses Typescript as an example language. However, there are repositories with example code for multiple languages.
- https://github.com/cloudgnosis/iac-ninja-aws-cdk-ts
- https://github.com/cloudgnosis/iac-ninja-aws-cdk-python
- https://github.com/cloudgnosis/iac-ninja-aws-cdk-go
The repositories will contain all the code examples from the articles series, implemented in different languages with the AWS CDK. You can view the code from specific parts and stages in the series, by checking out the code tagged with a specific part and step in that part, see the README file in each repository for more details.
Initialize our project
Before writing any code, we should initialise our AWS CDK project to get started.
In a command-line shell, create a directory to contain the project, and go to that directory:
mkdir my-cdk-infrastructure
cd my-cdk-infrastructure
In this directory, we will use the CDK command-line tool to initialise a project for Typescript. We use the init app sub-command with the —language option set to typescript.
❯ cdk init app --language=typescript
Applying project template app for typescript
# Welcome to your CDK TypeScript project!
This is a blank project for TypeScript development with CDK.
The `cdk.json` file tells the CDK Toolkit how to execute your app.
## Useful commands
* `npm run build` compile typescript to js
* `npm run watch` watch for changes and compile
* `npm run test` perform the jest unit tests
* `cdk deploy` deploy this stack to your default AWS account/region
* `cdk diff` compare deployed stack with current state
* `cdk synth` emits the synthesized CloudFormation template
Initializing a new git repository...
Executing npm install...
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN my-cdk-infrastructure@0.1.0 No repository field.
npm WARN my-cdk-infrastructure@0.1.0 No license field.
✅ All done!
❯
The resulting file structure in the project, excluding installed components in node_modules directory and hidden files, looks like this:
❯ tree -I node_modules
.
├── README.md
├── bin
│ └── my-cdk-infrastructure.ts
├── cdk.json
├── jest.config.js
├── lib
│ └── my-cdk-infrastructure-stack.ts
├── package-lock.json
├── package.json
├── test
│ └── my-cdk-infrastructure.test.ts
└── tsconfig.json
Feel free to look at the different files. We will for the most part ignore the contents of these files initially, since we do no need to know about it. We will even delete the sample code included, and start from scratch with the code we write. So feel free to delete (or keep) the following files:
bin/my-cdk-infrastructure.tslib/my-cdk-infrastructure-stack.tstest/my-cdk-infrastructure.test.ts
We will replace the content of the bin/my-cdk-infrastructure.ts file though, so feel free to copy it elsewhere, if you want. You can always get the same code by initialising another CDK project also.
Our first CDK code
Delete or empty the content of the bin/my-cdk-infrastructure.ts file, if you have not done that already. Open this file in a text editor of your choice.
In the AWS CDK introduction section in part 1, we covered some basic concepts of the AWS CDK, which include a CDK App. This is represented in Typescript as a class. We need to import it in the code, and create an object instance of that class:
import { App } from 'aws-cdk-lib';
const app = new App();
All AWS CDK application solutions need an App. It is not yet a useful piece of code that will produce anything we can deploy to AWS, as we can see if we run the AWS CDK command-line tool:
❯ cdk synth
This app contains no stacks
❯
The cdk synth command compiles the Typescript code, and runs it. The expected result is the CloudFormation that we can deploy to AWS. CloudFormation deploys stacks from templates, which we refer to as stacks in Aws CDK. So let us add a stack also, and associate that one with our App. AWS CDK has a Stack class for that purpose.
import { App, Stack } from 'aws-cdk-lib';
const app = new App();
const stack = new Stack(app, 'my-stack');
To create the stack, we specify two parameters - the AWS CDK app that the stack belongs to, and the name of the stack. The name is important - it is the same name that the corresponding CloudFormation stack will have. This means that the name must be unique within an AWS account and region combination.
If your friends or colleagues share an AWS account, and do the same exercise, you should set your individual names to the stack! Otherwise things may get a bit messy if you would overwrite each others stacks…
If we run the cdk synth command again, we get some output:
❯ cdk synth
Resources:
CDKMetadata:
Type: AWS::CDK::Metadata
Properties:
Analytics: v2:deflate64:H4sIAAAAAAAA/yXGQQ5AMBAAwLe4dxeNxgP8gBfQVrItu0lbHMTfRZxmNDbYQLKoTTVfGayLsNGC91RmG9Ww8uizHMn674Owo0LCj2JxHkOuz7bDtkdThUwE6eBCu8fx9wX28ZTwXgAAAA==
Metadata:
aws:cdk:path: my-stack/CDKMetadata/Default
Condition: CDKMetadataAvailable
Conditions:
CDKMetadataAvailable:
Fn::Or:
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- af-south-1
- Fn::Equals:
- Ref: AWS::Region
- ap-east-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-2
- Fn::Equals:
- Ref: AWS::Region
- ap-south-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-2
- Fn::Equals:
- Ref: AWS::Region
- ca-central-1
- Fn::Equals:
- Ref: AWS::Region
- cn-north-1
- Fn::Equals:
- Ref: AWS::Region
- cn-northwest-1
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- eu-central-1
- Fn::Equals:
- Ref: AWS::Region
- eu-north-1
- Fn::Equals:
- Ref: AWS::Region
- eu-south-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-2
- Fn::Equals:
- Ref: AWS::Region
- eu-west-3
- Fn::Equals:
- Ref: AWS::Region
- me-south-1
- Fn::Equals:
- Ref: AWS::Region
- sa-east-1
- Fn::Equals:
- Ref: AWS::Region
- us-east-1
- Fn::Equals:
- Ref: AWS::Region
- us-east-2
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- us-west-1
- Fn::Equals:
- Ref: AWS::Region
- us-west-2
Parameters:
BootstrapVersion:
Type: AWS::SSM::Parameter::Value<String>
Default: /cdk-bootstrap/hnb659fds/version
Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store.
Rules:
CheckBootstrapVersion:
Assertions:
- Assert:
Fn::Not:
- Fn::Contains:
- - "1"
- "2"
- "3"
- "4"
- "5"
- Ref: BootstrapVersion
AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.
❯
This looks like a lot, but this is only some generic metadata essentially that the AWS CDK always includes. By default, the cdk synth command outputs the CloudFormation template that it has generated, if it is a single template. Since the AWS CDK both compiles and runs the code, it can be a simple sanity check to see if what we have written might be something we can deploy, at least until we have unit tests, pipelines, etc in place.
In most cases, we do not want to see the CloudFormation template output though to do this check. We can avoid that by using the --quiet option:
❯ cdk synth --quiet
❯
Now we have a less verbose feedback loop here.
Create the EC2 instance
Ok, so we now have a foundation to actually deploy something to AWS, but we only have an empty stack, and we need to fill that with something - our EC2 instance.
The AWS CDK library has a number of submodules, one for each AWS service it supports (plus a few more). We want to use the aws-ec2 submodule. In that submodule we want to use the Instance class, that describes an EC2 instance resource.
We need to associate the instance with the stack we want to deploy, and we should give the instance a name as well. We use the same pattern as when we associated the stack with the AWS CDK App:
import { App, Stack } from 'aws-cdk-lib';
import { Instance } from 'aws-cdk-lib/aws-ec2';
const app = new App();
const stack = new Stack(app, 'my-stack');
const instance = new Instance(stack, 'my-ec2');
The name itself for the instance be an internal name in the AWS CDK App, it will not literally show up with the same name in AWS. The code above will not compile properly.
❯ cdk synth --quiet
⨯ Unable to compile TypeScript:
bin/my-cdk-infrastructure.ts:8:18 - error TS2554: Expected 3 arguments, but got 2.
8 const instance = new Instance(stack, 'my-ec2');
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
node_modules/aws-cdk-lib/lib/aws-ec2/lib/instance.d.ts:335:47
335 constructor(scope: Construct, id: string, props: InstanceProps);
~~~~~~~~~~~~~~~~~~~~
An argument for 'props' was not provided.
Subprocess exited with error 1
❯
Many resources will require additional parameters. In Typescript, you provide these as a third parameter which is a structure of some kind - different for each type of resource. In this case it is a structure of type InstanceProps.
To open up the AWS CDK documentation, you can use the command cdk docs. This command will open up a browser window/tab with the home page too the AWS CDK Api documentation. From there you can drill down to the aws-ec2 submodule, and in there find the InstanceProps structure. (Or use the link here)
In the documentation we can see that there are three thing that must be specified with InstanceProps:
- instanceType - the type of instance to launch
- machineImage - the Amazon Machine Image (AMI) to use
- vpc - the VPC to launch the EC2 instance in
We need to supply these three properties. Let us tackle them, one by one:
An instance type describes a combination of memory, CPU, disk and networking capacity for a virtual machine. AWS divides these into various families of virtual machines, and within each family there are multiple sizes of machines. For this setup we are going to go with an instance type which is part of the AWS free tier, which means if you have an AWS account that is less than 12 months, you should not need to pay anything for it. The family here is burstable compute generation 2, also known as T2 , and the size is micro. It a nice general purpose and small type of machine. In AWS CDK the instance family is also referred to as instance class. We can use the of() function for InstanceType to get the right instance type based on parameters we provide.
const instance = new Instance(stack, 'my-ec2', {
instanceType: InstanceType.of(InstanceClass.T2, InstanceSize.MICRO),
});
Next, we should get the appropriate machine image to use. Since we do not care so much about specific versions at this point, we just want to get the latest Amazon Linux machine image. This is something AWS CDK can handle for us.
const instance = new Instance(stack, 'my-ec2', {
instanceType: InstanceType.of(InstanceClass.T2, InstanceSize.MICRO),
machineImage: MachineImage.latestAmazonLinux(),
});
Last but not least, we need a VPC (Virtual Private Cloud - actually a network infrastructure) to launch the virtual machine into. We can create a new VPC or reuse an existing one. For simplicity, we are going to use the default VPC in the account. All AWS accounts have a default VPC, unless it has been explicitly removed. We assume for now that your account has a default VPC.
If you had been using CloudFormation, you would need to look for the default VPC in your account and note its VPC ID value, perhaps via AWS Console. In AWS CDK, we let the CDK itself do that work for us and do the lookup. We need to associate the lookup data with the stack we want to put the instance into also.
const vpc = Vpc.fromLookup(stack, 'my-vpc', {
isDefault: true,
});
This means that when the AWS CDK code runs, it will use the current AWS credentials to look up VPC information, based on criteria we specify. In this case, we just specify that we want the default VPC. We keep that in a variable named vpc, which we then pass in as property to our new instance.
const vpc = Vpc.fromLookup(stack, 'my-vpc', {
isDefault: true,
});
const instance = new Instance(stack, 'my-ec2', {
instanceType: InstanceType.of(InstanceClass.T2, InstanceSize.MICRO),
machineImage: MachineImage.latestAmazonLinux(),
vpc,
});
Now we have everything in place to create our EC2 instance it seems! Let us get it running!
Running the CDK code
To recap where we are, the complete code we have written so far looks like this, in Typescript:
import {App, Stack} from 'aws-cdk-lib';
import {Instance, InstanceClass, InstanceSize, InstanceType, MachineImage, Vpc } from 'aws-cdk-lib/aws-ec2';
const app = new App();
const stack = new Stack(app, 'my-stack');
const vpc = Vpc.fromLookup(stack, 'my-vpc', {
isDefault: true,
});
const instance = new Instance(stack, 'my-ec2', {
instanceType: InstanceType.of(InstanceClass.T2, InstanceSize.MICRO),
machineImage: MachineImage.latestAmazonLinux(),
vpc,
});
As we mentioned in the previous section, our code will look up the default VPC in the target account, based on the credentials we use, and it should create the EC2 instance in that VPC.
In the same way as we tested with the cdk synth command when we had an empty stack, we can test that again now and see if that works ok.
❯ cdk synth
Cannot retrieve value from context provider vpc-provider since account/region are not specified at the stack level. Configure "env" with an account and region when you define your stack.See https://docs.aws.amazon.com/cdk/latest/guide/environments.html for more details.
Subprocess exited with error 1
❯
Oops! What happened here? It could not generate the stack.
The problem here is that we asked AWS CDK to look up the default VPC and include that in the generated CloudFormation. But to do that, it needs to know precisely which AWS account and region this stack is used for - it is not enough to implicitly assume whatever our current credentials refer to.
An optional parameter we can provide to the stack is the AWS environment, which includes the AWS account and the AWS region. With that information, the AWS CDK will be able to figure out exactly what VPC information to look up.
Luckily we do not need to hardcode this setting in the code, the CDK provides two environment variables that will be set for you, based on current AWSS credentials and settings. These environment variables are:
- CDK_DEFAULT_ACCOUNT
- CDK_DEFAULT_REGION
The CDK_DEFAULT_ACCOUNT environment variable contains the current AWS account ID, e.g. 123456789012. The CDK_DEFAULT_REGION will reflect what region setting is current, e.g. eu-west-1. We change the stack creation slightly to include these settings
const stack = new Stack(app, 'my-stack', {
env: {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION,
},
});
Now, when we run cdk synth again, we will get a different output:
❯ cdk synth
current credentials could not be used to assume 'arn:aws:iam::123456789012:role/cdk-hnb659fds-lookup-role-123456789012-eu-west-1', but are for the right account. Proceeding anyway.
Resources:
myec2InstanceSecurityGroup1CDE1A58:
Type: AWS::EC2::SecurityGroup
.. more CloudFormation output ..
The cdk synth command generates a stack, but what is that warning message about?
At this stage, when we are only working with a single AWS account, this message is not something we need to worry about. The AWS CDK have a few built in features to handle multi-account deployment patterns, which will come in handy when we deal looking up information in one AWS account from another AWS account, for example. This is out of the scope of this article and we only use a single account here, so we can safely ignore this message.
If you looked at the generated output from cdk synth, you would see that the VPC Id of the default VPC in your AWS account is in fact in place. You can also see it in a new file in the directory of your project, which name is cdk.context.json. If you look in this file, you will se references to VPC Id, subnets, route tables , etc.
All that is data that has been saved in this file when AWS CDK performed the lookup. This file acts as a cache for various data from the environments you work with. Any data that AWS CDK tries to look up, it will fetch from this file rather than the actual AWS environment, if it is present.
This approach to handle existing data in the environment is part of an approach to make predictable deployments in AWS CDK, which we get into more depth later.
Deploy the EC2 instance
So can we finally deploy that EC2 instance? Yes, we can! Let us run the cdk deploy command
❯ cdk deploy
current credentials could not be used to assume 'arn:aws:iam::069901141591:role/cdk-hnb659fds-deploy-role-069901141591-eu-west-1', but are for the right account. Proceeding anyway.
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
IAM Statement Changes
┌───┬────────────────────────────┬────────┬────────────────┬───────────────────────────┬───────────┐
│ │ Resource │ Effect │ Action │ Principal │ Condition │
├───┼────────────────────────────┼────────┼────────────────┼───────────────────────────┼───────────┤
│ + │ ${my-ec2/InstanceRole.Arn} │ Allow │ sts:AssumeRole │ Service:ec2.amazonaws.com │ │
└───┴────────────────────────────┴────────┴────────────────┴───────────────────────────┴───────────┘
Security Group Changes
┌───┬─────────────────────────────────────────┬─────┬────────────┬─────────────────┐
│ │ Group │ Dir │ Protocol │ Peer │
├───┼─────────────────────────────────────────┼─────┼────────────┼─────────────────┤
│ + │ ${my-ec2/InstanceSecurityGroup.GroupId} │ Out │ Everything │ Everyone (IPv4) │
└───┴─────────────────────────────────────────┴─────┴────────────┴─────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)
Do you wish to deploy these changes (y/n)?
What is this question and what is the data shown here? The AWS CDK has a safety check, if your deployment will perform changes that adds permissions in some way, it will be default ask if you are really sure about this. In this case, we are ok with outbound traffic from the EC2 instance role, which is the security group part. There is no inbound communication open to the EC2 instance, at all. For the IAM permissions, there is right now only a dummy change - it will be possible to assign IAM permissions to this EC2 instance, but we have not added any yet.
So it is safe for us to answer yes here and continue.
Do you wish to deploy these changes (y/n)? y
my-stack: deploying...
current credentials could not be used to assume 'arn:aws:iam::069901141591:role/cdk-hnb659fds-deploy-role-069901141591-eu-west-1', but are for the right account. Proceeding anyway.
❌ my-stack failed: Error: my-stack: SSM parameter /cdk-bootstrap/hnb659fds/version not found. Has the environment been bootstrapped? Please run 'cdk bootstrap' (see https://docs.aws.amazon.com/cdk/latest/guide/bootstrapping.html)
Sigh… Why is it failing now? The error message tells us what the problem likely is - we have not bootstrapped the environment!
What is bootstrapping, and why do we need to do that?
For AWS CDK, bootstrapping is preparing each AWS environment (AWS account + region combination) for deployments. That includes setting up S3 bucket for deployment, set IAM roles to use with single- or multi-account or region deployments, and lookups.
This bootstrapping process is something that you normally do once per account + region combination. To perform a simple single account bootstrap, we just run cdk bootstrap. This will deploy a CloudFormation stack with the name CDKToolkit. You can see the progress of the bootstrap when the command executes.
❯ cdk bootstrap
⏳ Bootstrapping environment aws://123456789012/eu-west-1...
Trusted accounts for deployment: (none)
Trusted accounts for lookup: (none)
Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
CDKToolkit: creating CloudFormation changeset...
✅ Environment aws://123456789012/eu-west-1 bootstrapped.
If you want, you can go to AWS Console under CloudFormation and see what it has created. Anyway, now that we have done the bootstrap, we can try cdk deploy again.
Do you wish to deploy these changes (y/n)? y
my-stack: deploying...
[0%] start: Publishing 48c3b22f5968bc85e081f72eaf153a0aad114bfae184d783e93b8a01f344fe47:069901141591-eu-west-1
[100%] success: Published 48c3b22f5968bc85e081f72eaf153a0aad114bfae184d783e93b8a01f344fe47:069901141591-eu-west-1
my-stack: creating CloudFormation changeset...
✅ my-stack
Stack ARN:
arn:aws:cloudformation:eu-west-1:123456789012:stack/my-stack/2cd99040-40a5-11ec-9c50-0ad46b2ba94d
Now that looks a bit better! Similar to the bootstrap process, AWS CDK reports the progress of the deployment in the command-line interface.
If we take a look in the AWS Console under EC2, we might see something like this:
We have an instance running, and that is great!
However, it does have a public IP address, but we did not want it to be reachable from internet. We do have a security group associated with the instance that does not allow any inbound traffic, so it is still blocked from traffic, despite the public IP address.
But, how are we going to login to the machine and still block public access? This will be what we cover in the next article, which will be shorter than this one.
For now, we will clean up after ourselves and delete this instance, and all associated resources that we created with it. We can do that easily with the cdk destroy command.
❯ cdk destroy
Are you sure you want to delete: my-stack (y/n)? y
my-stack: destroying...
✅ my-stack: destroyed
In the next article, we will continue with the code we have created so far, and improve on that. Until next time!
