Gitlab +Jenkins +S3 部署静态页面

Posted 2020-11-30

一、Introduction

This article provides instructions to setup a basic deployment using Jenkins to Amazon S3 by pulling the source code from GitLab and pushing it to a S3 bucket. Deployment automation using Jenkins saves us from manually upload files to the S3 bucket using AWS online console. In addition to saving time, this method pushes the exact code that was committed in the latest Git push, therefore avoiding upload of wrong files. This is useful for static web site stored on S3 having a large number of files.

二、Pre-requisite

OS: Mac OS but the main ideas are OS agnostic.

It is assumed that you already have a static webpage on AWS S3. If not, you can check out instructions from AWS documentation: http://docs.aws.amazon.com/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html

You should also have your project added in GitLab. If not, you can have a look at their documentation: https://docs.gitlab.com/ee/gitlab-basics/create-project.html

Install Jenkins

• 1. Download Jenkins and install (Current LTS version as of this writing is 2.89.2): https://jenkins.io/download
• 2. After the installation, change the Update Site url from https to http: Jenkins->Plugin Manager->Advanced (tab)->Update Site: http://updates.jenkins.io/update-center.json
• 3. Install Git Plugin (3.7.0) Jenkins->Plugin Manager->Available (tab) Optional. Install Green Balls plugin to make the blue success ball green.

Setup to pull project from GitLab

Establish SSH authentication between your local machine and GitLab.

• 1. Get the Private Key Inside the File /.ssh/id_rsa and Create a Git Credential.
     This site shows you how to create a Pair of Ssh Key if you do not have one: https://docs.gitlab.com/ce/ssh/README.html
• 2. Create a Git credential in Jenkins.

Jenkins->Credentials->System->Global credentials (unrestricted) -> Add Credentials

Kind: SSH Username with private key.
Select Private Key and paste the private key.
Press OK

• 3. Add Git username and email to Jenkins Git properties.

Manage Jenkins->Configure System
Add your Git username and email under Git plugin section

• 4. Create a project/job in Jenkins

Jenkins->New Item

Enter any name for this Job
Select Freestyle Project
Press Ok

• 5. Configure Source Code Management to pull the project from Git.

Fill in Git project URL and select credential to use (created in step 2) under Source Code Management tab.
Press Save.

• 6. At this point you can test to see if the Jenkins job can retrieve your project from Git by clicking Build Now.

If successful, you should see a blue or green ball. You can check the logs by clicking on the blue/green ball.

To check if the project is correctly pulled from git to Jenkins workspace, click on Workspace. You should see the same files that are in your Git repository.

Pushing project to Amazon S3

• 7. Create a user to allow access to your S3 bucket.
     In the online AWS console->IAM (service)->Users
     Click Add user
     Enter any username you want
     Check Programmatic access
     Click Next Permission
     Select tab Attach Policy Directly
     Search for ‘s3‘
     Select AmazonS3FullAccess
     Click Next Review
     Click Create user
     Write down the user Access Key ID and Secret Access Key as they will not be available anymore after this point.
     Click Close

If you server is EC2, you can use IAM Role to attach to your EC2.

• 8. Install and configure aws command (command line client to access S3)

apt install awscli

https://docs.aws.amazon.com/zh_cn/cli/latest/userguide/cli-chap-install.html

• 9. Update the Jenkins job to use awscli to push code to S3

Return to the job created in Jenkins and click configure
Go to the Build section
Enter:

aws s3 sync . s3://beauty.wzlinux.com/ --region eu-west-3 --delete --exclude ".git/*";

explanation: the first line tells Jenkins where to find s3cmd command by adding its location to PATH
the second line uses s3cmd to copy all (.) to a S3 bucket name optimus.com
--delete-removed will remove deleted files from S3
--exclude ‘.git/‘ excludes the git directory to sync to S3.
Click Save
Click Build Now

By checking the job’s log, you should see that Jenkins pulled the code from Git and pushed it to AWS S3.
That’s it! I hope this has helped you.

License
This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)