Lab: Providing Internet Access to a Private AWS Subnet

Lab - Accessing the internet from a private subnet

In this lab we are going to iteratively build a network containing a public subnet and a private subnet that can access the internet. We are going to start with the most basic network and continue to make modifications and observe the effects of the changes we make.

To create the resources in our AWS account, we’re going to be using CloudFormation. For a reference to the CloudForation resources we add to the templates, see AWS Resource and Property Types Reference. To launch the templates in your account, simply copy the template to a .yaml file and do the following:

  • Navigate to the CloudFormation console and click Create Stack.

create a stack Create a stack from the CloudFormation console.

  • Select Upload a template file and select your .yaml file. Click Next.

upload a template Uploading a CloudFormation template ‘stack.yaml’.

  • Give your stack a name. I’m using “LabStack”. Click Next.
  • You can assign tags to your stack if you’d like to, or just click Next.
  • Finally, click Create Stack.

stack being created Create a stack from the CloudFormation console.

You can update an existing stack by selecting the stack and clicking the Update button. Select Replace Current Template and Upload a template file. Select the updated template file and click Next until you get to the final screen, then click Update.

To familiarize yourself with the terms VPC and Subnets, please read the AWS documenation found here.

In this experiment I refer to subnets as being public or private. Here is how I define those terms:

public subnet - a subnet whose hosted resources can be accessed from the internet.

private subnet - a subnet whose hosted resources cannot be accessed from the internet.

Now that we have a basic understanding of what our top level network resources are, let’s begin our lab!

Phase 1 - A Basic network

The first iteration of the template is a very basic network. It contains two subnets, PublicSubnet and PrivateSubnet. Each of these belong to the parent VPC LabVPC. There isn’t anything that makes either of these subnets public or private at the moment. The only difference between the two is that we enabled MapPublicIpOnLaunch in the PublicSubnet. This will provide any EC2 resource hosted in the subnet to be automatically assigned a public IPv4 address.

The CidrBlock values we chose for the lab are small since we only need two subnets for hosting a couple of EC2 instances. I’m not a network engineer, so I use an ip calculator to help calculate the CidrBlocks.

Template - Phase 1

AWSTemplateFormatVersion: '2010-09-09'
Description: 'VPC with one public subnet and one private subnet'

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/24
      EnableDnsSupport: true
      Tags:
        - Key: 'Lab'
          Value: 'Private subnet internet access'
        - Key: Name
          Value: LabVPC
  PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: us-east-1a
      VpcId: !Ref VPC
      CidrBlock: 10.0.0.0/28
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Lab
          Value: 'Private subnet internet access'
        - Key: Name
          Value: PublicSubnet
  PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: us-east-1a
      VpcId: !Ref VPC
      CidrBlock: 10.0.0.16/28
      Tags:
        - Key: Lab
          Value: 'Private subnet internet access'
        - Key: Name
          Value: PrivateSubnet

Phase 1 - Observations

After the stack has been created, take a look at what we got. From the VPC console, we can see that we have our LabVPC with the 10.0.0.0/24 CidrBlock. Along with the VPC, AWS also created a Route Table, and a Network Access Control List (NACL) for the VPC.

VPC details Details for the LabVPC.

If you click on the route table link, you can see some details about the route table. First, we see that it has been assigned as the “main” route table for the VPC. This means that any subnets that we add to this VPC will be implicitly associated with this route table unless we explicitly associate them with another route table.

default route table details Default route table details.

You can see the subnets associated with a route table on the Subnet Associations tab. Currently our route table has no explicitly associate subnets. This is because we created our subnets from a CloudFormation stack and did not associate them with a route table.

default route table subnet associations Subnet associations for the default route table.

default route table routes Default route table routes.

The NACL that we were provided with has been designated as the default NACL for the VPC. This means that any subnets that are added to the VPC and are not explicitly associated with another NACL, will be associated with this one. As we can see by clicking on the Subnet associations tab, it is associated with both of the subnets we created. If we click on the Inbound Rules and Outbound Rules tabs, we can see that both whitelist all traffic.

NACL details NACL details.

NACL rules Inbound NACL Rules (outbound rules are identical).

To test the internet connectivity we have in both of these subnets, let’s launch an EC2 instance. First let’s go to the EC2 console and click Launch Instance. For this lab, I’m going to use an Amazon Linux 2 AMI.

ami Selecting the Amazon Linux 2 AMI.

Click Next and select a t2.micro for the instance type. Click Next: Configure instance details. Here we will specify our LabVPC and PublicSubnet. You can leave everything else as the default values. Because we are launching the instance into our PublicSubnet and we specified that IPv4 addresses should be auto assigned, this instance will have a public IPv4 address.

public instance configuration Public EC2 instance configuration.

Click Next to add storage. You can use the default values here and click Next to add tags. I’m providing a Name tag with a value of LabInstance. This will make it easier to identify this instance if you already have EC2 instances in your account. Click Next to configure the security group.

public instance tags Tagging the EC2 instance.

A Security Group is a whitelist of allowed traffic to your instance. It differs from a NACL because NACLs allow you to boh whielist and blacklist traffic. In a Security Group, all traffic is blacklisted by default and you simply whitelist the traffic you want o allow. Here we will open the ssh port (22) so that we can connect to the instance, and ICMP to allow us to ping the instance. Name the Security Group LabSecurityGroup because we will use it again later. Click Review and Launch, then Launch.

security group rules Inbound Security Group rules for the public EC2 instance.

AWS will now prompt you to select a Key Pair for connecting to your instance. Select, Create a new key pair and name it LabKP. Click Download Key Pair and Launch Instances. You can click the provided EC2 id link to navigate back to the EC2 console. It will take a few minutes for the EC2 to come online. While waiting for the EC2 to come online, I copied the downloaded LabKP.pem.txt file from my Donloads folder to a better location and removed the .txt extension (I am using a MacBook). We’ll use this file in just a bit to connect to our EC2 instance.

creating ec2 key pair Creating an EC2 key pair.

After the EC2 instance status checks pass, you can select the EC2 instance to view its details. Double check the details to make sure that EC2 instance was launched into the PublicSubnet. You can see in the details that the instance has a public IPv4 address. Let’s test to see if we can ping the instance from the internet.

public instance details Public EC2 instance details.

Testing Internet Connectivity of Public EC2 Instance

Let’s try pinging our instance by opening a terminal and typing ping #.#.#.#, replacing #.#.#.# with the public Ipv4 address of the EC2 instance. Suprisingly, the ping test fails despite the fact that our NACL for the subnet allows all traffic, the Security Group allows ICMP traffic, and the instance has a public IPv4 address.

failed ping to public instance Failed ping attempt to public EC2 instance.

Let’s see if we can connect to the internet from the instance. In a terminal, navigate to the location where you saved your downloaded key pair and type ssh -i LabKP.pem ec2-usr@#.#.#.#, replacing #.#.#.# with the public Ipv4 address of the EC2 instance. The ssh connection fails to connect despite the fact that the Security Group whitelists the ssh port from anywhere.

So why can’t we establish a connection to the instance from the internet? By our definition, the subnet we are hosted in is not a public subnet. This is an important security aspect to keep in mind when you create new VPCs and subnets. By default, an AWS VPC and subnet have no internet connectivity. This usually isn’t understood by new AWS users because the default VPC that is provided for you when you first create an AWS account has all of the resources in place for you to access the internet. What our VPC is missing is an Internet Gateway.

Let’s add this resource to the next phase of our template.

Phase 2 - A Basic network with a public subnet

Our Phase 1 CloudFormation stack was missing an Internet Gateway for providing internet connectivity to our PublicSubnet. In addition to adding an Internet Gateway to our template, we are going to add a Route Table with a route for internet traffic to be routed through the Internet Gateway, and an association to our PublicSubnet. We need to do this in order for us to route internet traffic to the Internet Gateway. We could add this routing to our default Route Table, but that would provide internet connectivity to our PrivateSubnet that we do not want.

Template - Phase 2

AWSTemplateFormatVersion: '2010-09-09'
Description: 'VPC with one public subnet and one private subnet'

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/24
      EnableDnsSupport: true
      Tags:
        - Key: 'Lab'
          Value: 'Private subnet internet access'
        - Key: Name
          Value: LabVPC
  PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: us-east-1a
      VpcId: !Ref VPC
      CidrBlock: 10.0.0.0/28
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Lab
          Value: 'Private subnet internet access'
        - Key: Name
          Value: PublicSubnet
  PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: us-east-1a
      VpcId: !Ref VPC
      CidrBlock: 10.0.0.16/28
      Tags:
        - Key: Lab
          Value: 'Private subnet internet access'
        - Key: Name
          Value: PrivateSubnet
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: LabInternetGateway
  InternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC
  PublicSubnetRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: PublicSubnetRouteTable
  PublicSubnetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet
      RouteTableId: !Ref PublicSubnetRouteTable
  InternetRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PublicSubnetRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

Phase 2 - Observations

After updating the stack, navigate to the newly created Route Table. From the VPC console, select Route Tables from the left sidebar menu. Select the PublicSubnetRouteTable in the list. Explore the details of the Route Table by navigating through the tabs.

public route table routes Public route table routes including internet traffic being routed to the Internet gateway.

Now that we’ve added an Internet Gateway to our VPC and have routed internet traffic in the PublicSubnet to it, let’s try our tests again.

Testing Internet Connectivity of Public EC2 Instance

Let’s try pinging our instance by opening a terminal and typing ping #.#.#.#, replacing #.#.#.# with the public Ipv4 address of the EC2 instance. This time, we are able to ping the EC2 instance proving that we can make a connection to it from the internet.

successful ping to public instance Successful ping of public EC2 instance.

Let’s see if we can connect to the internet from the instance. In a terminal, navigate to the location where you saved your downloaded key pair and type ssh -i LabKP.pem ec2-usr@#.#.#.#, replacing #.#.#.# with the public Ipv4 address of the EC2 instance. Now we can successfully connect to the EC2 instance and ping google.com from the instance proving that the instance has outbound access to the internet.

successful ping to google from public instance Successful ping of google.com from public EC2 instance.

We now have internet connectivity for our PublicSubnet. The traffic can be controlled by blacklisting and whitelisting traffic in the NACL and we can also protect instances in this subnet by maintaining strict Security Group whitelisting rules. If desired, you can create a NACL specifically for the PublicSubnet so that the rules for that subnet can differ from the rules for other subnets in the VPC.

So how about the PrivateSubnet? The PrivateSubnet has no internet connectivity at all because it has no route to an Internet Gateway. But what if we want to provide outbound internet connectivity? Couldn’t we simply add a route to the Internet Gateway just as we did with the PublicSubnet? We could do this and keep it “private” by keeping the NACL rules more restrictive than the rules for the PublicSubnet NACL. The instances should be private also because we don’t intend on assigning them public IPv4 addresses either.

While this would provide us with a “private” subnet, it is not the recommended approach. Even though the subnet would technically not have any connections from the internet, the possibility of a connection still exists. A simple mistake like assigning a public IPv4 address, and a liberal Security Group could create a security risk. The better solution would be to provide the PrivateSubnet with outbound only internet connectivity using a NAT Gateway.

Phase 3 - A Basic network with a public subnet and private subnet with outbound internet access

While our phase 1 stack gave us a public subnet that had outbound and inbound internet connectivity, it left us with a private subnet that had no internet access. To provide the PrivateSubnet with outbound internet access, we are going to add a NAT Gateway, a Route Table with a route for internet traffic to the NAT Gateway, and associate it with our PrivateSubnet. This is all very similar to how we added an Internet Gateway to our PublicSubnet.

The NAT Gateway allows outbound internet traffic to be routed out to the internet without being exposed to inbound internet connections. It is associated with an Elastic IP Address, making it a static endpoint that is used as the destination for outbound internet traffic. In a way, the NAT Gateway acts as a jump box or bastion in the public subnet that your private instances use to connect to the internet. NACL rules and Security Group rules can then be applied to control the allowed outbound internet traffic. When modifying NACL rules for a private subnet, be aware that the rules must allow reply traffic from outbound calls. These replies use waht are known as Ephemeral ports. Inbound rules in the NACL have no affect on inbound internet traffic since no inbound connections can be made via a NAT Gateway. Even with a rule allowing all inbound traffic, no internet connections can be made. We will test this in this phase.

NOTE: There is one gotch in the template when defining the Elastic IP that is associated with the NAT Gateway. You must specify DependsOn to the AWS::EC2::VPCGatewayAttachment resource when creating the stack. This reinforces the understanding that the NAT Gateway uses the Internet Gateway to access the internet. Since we are updating an existing stack, and the gateway attachment already exists, this is not required, but it is specified in the template.

Template - Phase 3

AWSTemplateFormatVersion: '2010-09-09'
Description: 'VPC with one public subnet and one private subnet'

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/24
      EnableDnsSupport: true
      Tags:
        - Key: 'Lab'
          Value: 'Private subnet internet access'
        - Key: Name
          Value: LabVPC
  PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: us-east-1a
      VpcId: !Ref VPC
      CidrBlock: 10.0.0.0/28
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Lab
          Value: 'Private subnet internet access'
        - Key: Name
          Value: PublicSubnet
  PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: us-east-1a
      VpcId: !Ref VPC
      CidrBlock: 10.0.0.16/28
      Tags:
        - Key: Lab
          Value: 'Private subnet internet access'
        - Key: Name
          Value: PrivateSubnet
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: LabInternetGateway
  InternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC
  PublicSubnetRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: PublicSubnetRouteTable
  PublicSubnetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet
      RouteTableId: !Ref PublicSubnetRouteTable
  InternetRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PublicSubnetRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway
  NATElasticIPAddress:
    Type: AWS::EC2::EIP
    DependsOn: InternetGatewayAttachment
    Properties:
      Domain: vpc
  NATGateway:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt NATElasticIPAddress.AllocationId
      SubnetId: !Ref PublicSubnet
      Tags:
        - Key: Name
          Value: LabNATGateway
  PrivateSubnetRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: PrivateSubnetRouteTable
  PrivateSubnetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet
      RouteTableId: !Ref PrivateSubnetRouteTable
  NATRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PrivateSubnetRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref NATGateway

Phase 3 - Observations

After deloying our latest stack changes, you should be able to see that each subnet is associated with its own Route Table now, leaving the default Route Table with no subnet associations. Navigate to the VPC console and select Route Tables from the left sidebar menu. You should see the PublicSubnetRouteTable and PrivateSubnetRouteTable in the route table list. The PublicSubnetRouteTable routes internet traffic to the Internet Gateway and the PrivateSubnetRouteTable routes internet traffic to the NAT Gateway.

private route table routes Private route table routes including internet traffic being routed to the NAT Gateway.

From the VPC console, click on the NAT Gateways link in the left sidebar menu. You should see our LabNATGateway in the list and it should be hosted in the PublicSubnet. To see what kind of internet connectivity our PrivateSubnet has now, let’s run some tests.

nat gateway details NAT Gateway details.

Testing Internet Connectivity of Private EC2 Instance

Let’s start by launching a second EC2 instance in the PrivateSubnet. We will follow the same steps as we did for our first EC2 instance, except this time we will specify the PrivateSubnet and we will choose to assign a public IPv4 address. Typically you would not do this for an instance in a private subnet, but we are doing this to prove a point. Use the same Security Group and key pair as the first instance.

private instance configuration Private EC2 instance configuration.

private instance tags Private EC2 instance tags.

private instance security group Private EC2 instance Security Group.

private instance key pair Private EC2 instance key pair.

First let’s prove that a NAT Gateway does not allow connectiond from the internet. When we created our private EC2 instance, we gave it a public IPv4 address. Let’s try to ping that by typing ping #.#.#.# in a terminal, replacing #.#.#.# with the public Ipv4 address of the private EC2 instance. The ping fails even though the NACL and Security Group rules are identical to the public instance. This demonstrates the higher level of security that you get using a NAT Gateway instead of diretly using an Internet Gateway and trying to protect your instances by NACL and Security Group rules alone.

failed ping to private instance Failed ping attempt to private EC2 instance.

Next, let’s check to see if the instance can access the internet. We will use our public instance as a bastion to connect to our private instance. To do this we are going to copy the .pem file to the public instance so that we can use it to ssh to the private instance. This practice is not recommended, but we will be deleting these instances as soon as we are finished with this lab. To copy the .pem file to the public instance type scp -i LabKP.pem ./LabKP.pem ec2-user@#.#.#.#:. in a terminal, replacing #.#.#.# with the public Ipv4 address of the public instance. After the file is copied, ssh into the public instance as we did previously.

After you’ve connected to the public instance, type ls to show the files in the root directory. You should see the LabKP.pem file that we copied. Next ssh into the private instance by typing ssh -i LabKP.pem ec2-user@#.#.#.#, replacing #.#.#.# with the private IPv4 address of the private instance. Ping Google.com like we did in our private instance test. As you can see, our private instance now has outbound internet access.

successful ping to google from private instance Successful ping of google.com from private EC2 instance.

Conclusions

Based on our observations, we have proven the follwoing:

  • Without a route to an Internet Gateway, subnets have no internet connectivity.
  • NAT Gateways only provide outbound internet connectivity.
  • Private subnets can be provided with outbound only internet connectivity by routing internet traffic to a NAT Gateway.

Clean up

To clean up after this lab, delete the two EC2 instances and Security Group that we created. This can be done from the EC2 console. Select Instances in the left sidebar menu, select the two EC2 instances and from the Actions dropdown, select Instance State > treminate. Netx, select Security Groups in the left sidebar menu, select the Security Group and from the Actions dropdown, select Delete Security Groups. Do the same for the key pair by selecting Key Pairs in the left sidebar menu. Select the key pair and then Delete.

The rest of the resources can be deleted by deleting the CloudFormation stack in the CloudFormation console.

Tags

aws labs network security

Recent posts

Lab: Providing Internet Access to a Private AWS Subnet

Providing Internet Access to a Private AWS Subnet


Related posts

Providing Internet Access to a Private AWS Subnet


Archives

2019 (2)