Archive
AWS and IPv6 only solution
Are you interested in bypassing the complexities of Network Address Translation (NAT) and deploying containers without the need for IP address translation? The solution lies in acquiring ample public IP addresses, although the high cost of each IPv4 address makes it impractical to assign one to every container. Conversely, IPv6 offers an abundance of addresses, allowing for unique assignments to multiple containers without constraints.
Originally defined in 1981, the Internet Protocol (IP) utilized fixed-length addresses composed of four octets (32 bits), known as IPv4 addresses. As the depletion of IPv4 addresses became apparent in the early 1990s, reusable private IP addresses were introduced in March 1994 to conserve address space. This necessitated Network Address Translation (NAT) for communication between internal and external hosts, which was standardized shortly after.
In response to IPv4 limitations, IPv6 was introduced in 1995, featuring expanded addressing capabilities with a 128-bit address size. However, the transition to IPv6 has been sluggish due to the lack of backward compatibility, with current adoption standing at around 22% after over two decades.
The objective of this post is to illustrate the process of deploying containers on a Cloud Provider (AWS) using IPv6, building upon the simplified Kubernetes multi-cluster networking discussed previously. The proposed topology involves creating EC2 instances with Elastic Network Interfaces (ENI) attached, enabling the allocation of a contiguous block of IPv6 addresses to each instance.
Although the allocation of smaller subnets (/64) to AWS instances is not currently feasible, Elastic Network Interfaces facilitate the association of a block of IPv6 addresses with each instance. Despite the less-than-optimal IPv6 prefix length, this approach enables the deployment of containers exclusively using IPv6.
The process entails several steps:
- Creation of EC2 instances with ENIs attached, allocating a block of IPv6 addresses.
- Installation of Docker and configuration of IPv6 addressing on the instances.
- Running containers exclusively using IPv6 addresses.
- Testing connectivity between containers and to external hosts over IPv6.
Here’s the coding part:
# Create EC2 instances with ENIs attached eni1=`aws ec2 create-network-interface \ --subnet-id $subnetId \ --description "My IPv6 ENI 1" \ --groups $sgId \ --ipv6-addresses \ Ipv6Address=2600:1f18:47b:ca03::1:1 \ Ipv6Address=2600:1f18:47b:ca03::8 \ Ipv6Address=2600:1f18:47b:ca03::9 \ Ipv6Address=2600:1f18:47b:ca03::a \ Ipv6Address=2600:1f18:47b:ca03::b \ --query 'NetworkInterface.NetworkInterfaceId' \ --output text` # Repeat for the second ENI eni2=`aws ec2 create-network-interface \ --subnet-id $subnetId \ --description "My IPv6 ENI 2" \ --groups $sgId \ --ipv6-addresses \ Ipv6Address=2600:1f18:47b:ca03::2:2 \ Ipv6Address=2600:1f18:47b:ca03::c \ Ipv6Address=2600:1f18:47b:ca03::d \ Ipv6Address=2600:1f18:47b:ca03::e \ Ipv6Address=2600:1f18:47b:ca03::f \ --query 'NetworkInterface.NetworkInterfaceId' \ --output text` # Launch instances with ENI attached vm1=`aws ec2 run-instances \ --key-name $AWS_SSH_KEY \ --image-id ami-0ac019f4fcb7cb7e6 \ --instance-type r5d.large \ --network-interfaces DeviceIndex=0,NetworkInterfaceId=$eni1 \ --query 'Instances[0].InstanceId' \ --output text` # Similarly for instance-2 vm2=`aws ec2 run-instances \ --key-name $AWS_SSH_KEY \ --image-id ami-0ac019f4fcb7cb7e6 \ --instance-type r5d.large \ --network-interfaces DeviceIndex=0,NetworkInterfaceId=$eni2 \ --query 'Instances[0].InstanceId' \ --output text` # Get public IPv6 addresses for instances ip1=`aws ec2 describe-instances \ --filter Name=instance-id,Values=$vm1 \ --output text \ --query 'Reservations[].Instances[].NetworkInterfaces[].\ Ipv6Addresses[0].Ipv6Address'` # Similarly for instance-2 ip2=`aws ec2 describe-instances \ --filter Name=instance-id,Values=$vm2 \ --output text \ --query 'Reservations[].Instances[].NetworkInterfaces[].\ Ipv6Addresses[0].Ipv6Address'`
These steps involve creating EC2 instances with ENIs attached, launching instances with the ENIs, and retrieving public IPv6 addresses for each instance.
The process continues with configuring IPv6 addressing and installing Docker on the instances, followed by running containers exclusively using IPv6 addresses. Testing connectivity between containers and external hosts over IPv6 ensures the successful deployment of IPv6-only containers in the cloud.
Install Docker on Amazon Linux old AMIs
UPDATE (March 2020, thanks @ic): I don’t know the exact AMI version but yum install docker now works on the latest Amazon Linux 2. The instructions below may still be relevant depending on the vintage AMI you are using.
Amazon changed the install in Linux 2. One no-longer using ‘yum’ See: https://aws.amazon.com/amazon-linux-2/release-notes/
Docker CE Install
sudo amazon-linux-extras install docker
sudo service docker start
sudo usermod -a -G docker ec2-user
Make docker auto-start
sudo chkconfig docker on
Because you always need it….
sudo yum install -y git
Reboot to verify it all loads fine on its own.
sudo reboot
docker-compose install
Copy the appropriate docker-compose binary from GitHub:
sudo curl -L https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
NOTE: to get the latest version (thanks @spodnet): sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
Fix permissions after download:
sudo chmod +x /usr/local/bin/docker-compose
Verify success:
docker-compose version
Update npm with nvm for windows
Rename npm.cmd to npm1.cmd and then deleted other npm files: npm, npx.cmd, npx. Then I ran npm1 install -g npm
Pay attention, v8.40+ of npm not working with windows
EC2 Mac
#
# On your laptop, connect to the Mac instance with SSH (similar to Linux instances)
#
ssh -i
#
# On the Mac
#
# Set a password for ec2-user
sudo passwd ec2-user
# Enable VNC Server (thanks [email protected] for the feedback and tests)
sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart \
-activate -configure -access -on \
-configure -allowAccessFor -specifiedUsers \
-configure -users ec2-user \
-configure -restart -agent -privs -all
sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart \
-configure -access -on -privs -all -users ec2-user
exit
#
# On your laptop
# Create a SSH tunnel to VNC and connect from a vnc client using user ec2-user and the password you defined.
#
ssh -L 5900:localhost:5900 -C -N -i
# open another terminal
open vnc://localhost
#
# On the mac, resize the APFS container to match EBS volume size
#
PDISK=$(diskutil list physical external | head -n1 | cut -d” ” -f1)
APFSCONT=$(diskutil list physical external | grep “Apple_APFS” | tr -s ” ” | cut -d” ” -f8)
sudo diskutil repairDisk $PDISK
# Accept the prompt with “y”, then paste this command
sudo diskutil apfs resizeContainer $APFSCONT 0
Linux check occupied space in folder
hidden files
sudo du -sch /home/ec2-user/.[!.]* |sort -h
all files except hidden
sudo du -sch /home/ec2-user/* |sort -h