Gitlab CI/CD and terraform to deploy AWS resources (EC2/VPC/peering/subnets)

In this article, we will deploy the same AWS infrastructure as in the Github article.
We will use the same main.tf file, we’ve changed the name of the .tfstate terraform state file in the S3 bucket.

The .gitlab-ci.yml file in the root of the project contains the 3 stages, plan/apply and destroy. The plan/apply stages are triggered by any changes committed to the main.tf terraform file. They can also be run manually, from the Build/pipelines section, where 3 checkmarks will appear under the Stages header, for the pipeline. The destroy stage can be invoked manually by clicking on the last checkmark and on the run button at the end.
The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables need to be added manually to the Settings > CI/CD section. Choose Visibility > Masked and Protect variable.

Github Actions to deploy AWS infrastructure with terraform

Introduction

In this article we will use Github Actions to deploy terraform infrastructure.
An s3 bucket will be used to save and retrieve the terraform state file, in case we need to remove infrastructure via terraform destroy.
For the bucket, the AWS account will need to have the permissions below (attached to the bucket via an inline policy).

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:ListBucket",
                "s3:DeleteObject"  
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*"
            ]
        }
    ]
}

Steps to Set up GitHub Actions for Terraform:

AWS - Using AMI and VPC peering

Introduction
Resources that will be created
Terraform init/plan/apply
Testing connectivity with ssh and ping
Resource removal
The reference terraform file

Introduction

In this article we will use terraform and AWS cloud infrastructure to create two Linux VMs in two separate VPCs, each VM in its separate subnet, with private and public IP addresses. The machines will use a public AWS AMI with Debian 12 and will be able to access one another (test via ping), by using VPC peering.

AWS - using an access key, AMI templates, disk snapshots to safely upgrade software

Introduction
Creating an account and access key
Creating a disk snapshot
Upgrading linux software with dpkg tool
Using AMI templates and restoring a disk snapshot to an EC2 instance
Removing AMI templates and disk snapshots

Introduction

Let’s say we want to upgrade the software on a linux VM on AWS EC2, but first we would like to back up the VM’s boot disk as a snapshot, in case the upgrade causes issues and we want to restore from backup.
We will use the AWS web console to create a user in IAM, and we will use the AWS CLI to take a snapshot of the VM’s disk and restore it, if needed.

Azure Terraform - Create a virtual machine and enable SSH and ping

Introduction
Explanation of the terraform yaml file
Previewing, applying infrastructure changes with terraform
Connecting to the virtual machine via ssh and using ping
Deleting created infrastructure
The reference .tf Terraform configuration file

Introduction

In this tutorial we will deploy a Linux VM to Azure cloud, using terraform.
The virtual machine will have a private and public IP, and we will enable remote login via ssh with a public/private keypair. We will also enable ping.
The first step is to create the .tf terraform file, which is yaml syntax. There should be a single .tf file in the working directory.

Installing Active Directory role on Windows Server

Installation of AD DS role
Authorisation of DHCP server on domain
Testing name resolution via FQDN
Getting AD domain information

Installation of AD DS role

Let’s install the directory service role, so that we can create a domain and promote winsrv-1 to domain controller.

PS > Install-WindowsFeature -Name AD-Domain-Services -IncludeManagementTools

Success Restart Needed Exit Code      Feature Result                               
------- -------------- ---------      --------------                               
True    No             Success        {Active Directory Domain Services, Group P...
Install-ADDSForest -DomainName "ad.georgetech.co.uk" -DomainNetbiosName "ad" -InstallDns -Force

Authorisation of DHCP server on domain

We will need to authorise the DHCP server on winsrv-1 for it to allow to distribute IP addresses to computers such as winsrv-2 in the newly created domain ad.georgetech.co.uk:

Using AWS CLI and SES (simple email service) to send emails for alerts

Introduction
Domain validation and AWS SES configuration
Creating a user and assigning policies in the AWS console
Creating an access key and using the AWS CLI to send an email

Introduction

Let’s say we are running a bash script and we would like to be notified of success or failure after it is executed.
We could send an email out via Amazon SES (simple email service), by using the AWS CLI binaries.
We would also need an AWS service account that has permissions to send emails.
The emails will be sent from an email address in domain that we own (alerts@georgetech.co.uk).
The alerts will be sent to a GMail email address, for example, as we do not have an email server of our own.

Using tar, rclone, bash, AWS CLI, systemd services, timers - to schedule a website back up to an S3 bucket

Using AWS CLI to create an S3 bucket and to attach IAM policies
Configuring rclone to access the AWS S3 bucket
Creating a bash script to back up the website folder and upload it to an S3 bucket
Using a systemd service and timer to schedule website backup

Using AWS CLI to create an S3 bucket and to attach IAM policies

In this tutorial, we will back up the web site to a file and we will copy the resulting backup file to an AWS S3 bucket with rclone, at 3AM every day, via a systemd timer and service.

Windows Server - static IP address, DHCP, DNS server

Network topology
Renaming computers
Setting static IP address
Adding the DNS role
Adding the DHCP role and defining a DHCP scope
Testing DHCP IP assignment

Enabling ping via firewall rules and testing connectivity

Network topology

We have a topology of 2 Windows Server 2022 VMs, which we will call winsrv-1 and winsrv2-2.
winsrv-1 is directly connected to the router via the IP 192.168.1.10.
It will also be DHCP server for the 172.16.1.0/24 subnet and will connect to winsrv-2, which will connect to winsrv-1 via a the latter’s second network card.

Powershell by example

User management
Network management and connectivity
Install and connect via SSH to Windows server
Service management
Customising your Powershell profile
Getting help on a command
Practical example: search for Windows updates pending install

User management

To create a local user in Powershell, let’s use the New-LocalUser command:

PS > New-LocalUser -Name 'george' `
	-Description 'Local admin.' `
	-Password (ConvertTo-SecureString -String 'somestring' `
	-AsPlainText -Force)

Name   Enabled Description
----   ------- -----------
george True    Local admin.

To add the new user to the local Administrators group, we can use Add-LocalGroupMember; -WhatIf does not run the command, it shows what would be done:

Terminate a process from linux terminal by name - ssh

Let’s say we want to end the process ssh in another terminal because the connection is frozen due to network disconnect.
To do this, we need to identify the id of the ssh process, and end the process.
We will use:

  • pipes (|) to redirect the output of one program to the next.
  • ps ax to list all processes in all terminals
  • grep to search for the ssh text in the output of the ps command
  • awk to print the first word {print $1} of the first line NR==1
  • kill command to end the process identified in the awk line

To explain the concept of pipes (|): we execute ps ax to list all processes, and with the | we pass the resulting output to the grep command, which searched for the word ssh:

Using PHP and MySQL to build web pages

Before we begin, let’s allow the user george to connect to MySQL/MariaDB from any host, including the server that hosts php and the web server.
We will use % to allow connection from all hosts:

MariaDB [(none)]> GRANT ALL PRIVILEGES ON `publications`.* TO "george"@"%";
Query OK, 0 rows affected (0.010 sec)

We will be creating the login.php file which we will be using in other phpfiles.
This code will help us connect to the MySQL database with the user and password, and the publications database will be selected.
This code is visible and executed only by the server, it is not visible by the web browser.