Using Hugo and Travis CI To Deploy Blog To Github Pages Automatically

Blog
2543 words | 漢文版

Table of Contents

My personal static blog is hosted on Github. I used Hexo before, but as its complicated package dependence and cumbersome deployment process, I have to encapsulate the whole deployment environment into Docker image for rapid deployment, but it was still too complex. Now I switch to Hugo which is faster, simpler.

This article documents how to automatically synchronize blog contents generated by Hugo to Github through Travis CI.

New: I have migrated my blog from Github to GitLab on May 24, 2019 due to GitHub and Export Controls. Blog is Migrating Personal Hugo Static Blog From GitHub Pages To GitLab Pages.

This article is one of my personal static blog build series:

  1. Using Hugo and Travis CI To Deploy Blog To Github Pages Automatically
  2. Using Cloudflare Free SSL In GitHub Pages With Custom Domain
  3. Setting Up Slack Build Notification in Travis CI for Github Project

Attention: The following operation process will not involve account registration, Git configuration.

Official Docs

The relevant operations in this document are based on the official document as the operation guides:

GitHub Pages Limits

GitHub Pages is a static site hosting service designed to host your personal, organization, or project pages directly from a GitHub repository.

GitHub Pages sites are subject to the following usage limits:

  1. GitHub Pages source repositories have a recommended limit of 1GB.
  2. Published GitHub Pages sites may be no larger than 1 GB.
  3. GitHub Pages sites have a soft bandwidth limit of 100GB per month.
  4. GitHub Pages sites have a soft limit of 10 builds per hour.

More details in official document What is GitHub Pages?.

Prerequisite

  1. Registering a Github account;
  2. Registering a Travis CI account (login with Github account);
  3. Installing git, hugo in your system;
  4. Personal domain (optional, here my domain is axdlog.com);

Hugo Installation

Install Hugo is the official installation document of Hugo. If you’re using GNU/Linux, you may consider using my Python script to install it quickly.

Current release version info

1
2
3
4
5

# which hugo
/usr/local/bin/hugo

# hugo version
Hugo Static Site Generator v0.55.5-A83256B9 linux/amd64 BuildDate: 2019-05-02T13:03:36Z

Python Script

This Python script is used to install or upgrade Hugo on GNU/Linux.

operating process

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

$ sudo python3 ~/hugo.py
uccessfully download pack /tmp/hugo_0.55.4_Linux-64bit.tar.gz!
Successfully install Hugo v0.55.4!

Symlink info:
lrwxrwxrwx 1 root root 14 Apr 26 12:26 /usr/local/bin/hugo -> /opt/Hugo/hugo

Hugo info:
Hugo Static Site Generator v0.55.4-57900417 linux/amd64 BuildDate: 2019-04-25T07:38:50Z

$ sudo python3 ~/hugo.py
Latest version 0.55.4 existed.

Hugo info:
Hugo Static Site Generator v0.55.4-57900417 linux/amd64 BuildDate: 2019-04-25T07:38:50Z


# update operation
$ sudo python3 hugo.py
Local version 0.55.4 < latest version 0.55.5.
Successfully download pack /tmp/hugo_0.55.5_Linux-64bit.tar.gz!
Successfully install Hugo v0.55.5!

Symlink info:
lrwxrwxrwx 1 root root 14 May  2 11:21 /usr/local/bin/hugo -> /opt/Hugo/hugo

Hugo info:
Hugo Static Site Generator v0.55.5-A83256B9 linux/amd64 BuildDate: 2019-05-02T13:03:36Z

Create A New Site

You can reference official document Get Started、Quick Start to know how to create a new site through Hugo.

If this is your first time using Hugo and you’ve already installed Hugo on your machine, we recommend the quick start.

Here I name the site to quickstart (it is just the directory name, not the final site name).

1

hugo new site quickstart

Directory structure generated by Hugo:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# tree -L 2
.
├── archetypes
│   └── default.md
├── config.toml
├── content
├── data
├── layouts
├── static
└── themes

6 directories, 2 files

Attention: These files are not the final blog contents, they are just used to generate the blog contents.

Directory content is used to store your blog file (Markdown file).

If you wanna preview the final effect, just executing command hugo server to set up a local server which is listening port 1313. And opening the link in your browser.

1
2

Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop

Executing command hugo in your project directory, it will generate a directory named public. This directory contains the final blog contents. What you need to do is push all the files in this directory to the branch master of your GitHub repository.

The demonstration process is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# hugo

                   | EN   
+------------------+-----+
  Pages            | 426  
  Paginator pages  |  24  
  Non-page files   |   0  
  Static files     |  34  
  Processed images |   0  
  Aliases          |  95  
  Sitemaps         |   1  
  Cleaned          |   0  

Total in 403 ms

# tree -L 1
.
├── archetypes
├── config.toml
├── content
├── data
├── layouts
├── public
├── resources
├── static
└── themes

8 directories, 1 file

Github Repository Operation

Attention: The handling ideas and operations of this section are very important. Git setting procedure in official documents GitHub Help、Git Cheat Sheets.

If you use GitHub Pages to host blog, you need to create a repository named <username>.github.io. Then putting the files generated by Hugo to the branch master of this repository.

I create 2 branches via command git checkout --orphan to store source files and blog contents in directory public separately.

  • branch code stores source files;
  • branch master stores blog contents in directory public;

Attention please, this method is different from official document Configuring a publishing source for GitHub Pages.

Create Empty Repository

My GitHub account is MaxdSre, so I need to create a repository named maxdsre.github.io.

After creating the repository, it will prompts

or create a new repository on the command line

1
2
3
4
5
6

echo "# maxdsre.github.io" >> README.md
git init
git add README.md
git commit -m "first commit"
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git
git push -u origin master

or push an existing repository from the command line

1
2

git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git
git push -u origin master

Pushing Repository

There are 4 steps:

  1. Initializing Git repository via command git init in project directory;
  2. Creating branch code, pushing source files to remote repository;
  3. Executing command hugo to generate public directory; Transferring all the files in this directory to another temporary directory, empty project directory, then move the files store in temporary directory back to project directory;
  4. Creating branch master via command git checkout --orphan, then pushing all files store in current directory to remote repository.

Entering the target directory (here is /PATH/quickstart/), then executing the following commands in sequence.

Branch code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# Initialized empty Git repository in /PATH/.git/
git init

# Switched to a new branch 'code', equal to 'git branch code && git checkout code'
git checkout -b code

# exclude dir public/
echo '/public/' >> .gitignore
sed -r -i '/^\/public\/$/{$!d}' .gitignore

# Add file contents to the index
git add .

# Show the working tree status
# git status

# Record changes to the repository
git commit -m "Hugo generator code"

# Manage set of tracked repositories
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git

# Update remote refs along with associated objects to branch 'code'
git push -u origin code

Branch master

Executing command hugo in project directly to generate directly public. If you wanna check the branch info of this repository, just executing command git branch -a.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# execute command hugo
hugo

# create temporary dir
HUGO_TEMP_DIR=$(mktemp -d)
cp -R public/* "$HUGO_TEMP_DIR"

# create orphan branch 'master'
git checkout --orphan master

# empty current dir
rm .git/index
git clean -fdx

# copy back contents in dir public/
cp -R "$HUGO_TEMP_DIR"/. .

# add, commit, push
git add .
# git status
git commit -m 'initial blog content'
git push -u origin master

# remove temp dir
[[ -d "$HUGO_TEMP_DIR" ]] && rm -rf "$HUGO_TEMP_DIR"

Branch image

This section is optional. In order to store images in GitHub directly, I create the third branch image. All images used in blog will be saved in this branch.

Notice: GitHub provides 1 GB storage space for every repository, single file must less then 100 MB in size, more details in official document File and repository size limitations。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

git checkout --orphan image
rm .git/index
git clean -fdx

# add content to this branch

git add .
git commit -m 'initial blog images'
git push -u origin image

# fatal: The current branch image has no upstream branch.
# To push the current branch and set the remote as upstream, use
# git push --set-upstream origin image

Switching branch

If you wanna switch your local branch to remote branch remotes/origin/image, you may consider executing the following command

1
2
3
4
5
6
7
8

# git checkout -b <branch> --track <remote>/<branch>
# -b <new_branch>    Create a new branch named <new_branch> and start it at <start_point>;
# -t, --track     When creating a new branch, set up "upstream" configuration.

git checkout -t remotes/origin/image

# local branch, switch from other branch to branch 'code'
git checkout code

Demonstration example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

$ git branch
* code

$ git branch -a
* code
  remotes/origin/HEAD -> origin/code
  remotes/origin/code
  remotes/origin/image
  remotes/origin/master

$ git checkout -t remotes/origin/image
Branch 'image' set up to track remote branch 'image' from 'origin'.
  Switched to a new branch 'image'

$ git branch
  code
* image

$ git branch -a
  code
* image
  remotes/origin/HEAD -> origin/code
  remotes/origin/code
  remotes/origin/image
  remotes/origin/master

Here remote default branch is code, if you wanna change to branch master

1
2

# change HEAD from origin/code to origin/master
git remote set-head origin master

Deleting branch

Deleting branches has three types: remote branch, local branch, local remote-tracking branch, more details in How do I delete a Git branch both locally and remotely?.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# 1 - Delete a remote branch
$ git push origin --delete <branch> # Git version 1.7.0 or newer
$ git push origin :<branch> # Git versions older than 1.7.0

# 2 - Delete a local branch
$ git branch --delete <branch>
$ git branch -d <branch> # Shorter version

$ git branch -D <branch> # Force delete un-merged branches

# error: The branch 'image' is not fully merged.
# If you are sure you want to delete it, run 'git branch -D image'.

# 3 - Delete a local remote-tracking branch
$ git branch --delete --remotes <remote>/<branch>
$ git branch -dr <remote>/<branch> # Shorter

$ git fetch <remote> --prune # Delete multiple obsolete tracking branches
$ git fetch <remote> -p # Shorter

Travis CI Configuration

GitHub Pages Deployment is the official document of Travis CI。

Github Access Token Generation

You’ll need to generate a personal access token with the public_repo or repo scope (repo is required for private repositories). Since the token should be private, you’ll want to pass it to Travis securely in your repository settings or via encrypted variables in .travis.yml.

Generating personal access tokens in GitHub Pages Developer settings. If it is a private repository, choose repo. Otherwise, just choose public_repo. Here I choose public_repo, repo:status, repo_deployment.

The length of the generated token is 40, please keep is private.

Environment Variables Setting

Generated token is used in Travis CI, the page url of target repository in Travis CI is https://travis-ci.org/MaxdSre/maxdsre.github.io/settings.

Don’t forget to check Build only if .travis.yml is present, Build pushed branches, Build pushed pull requests.

The default name of token defined in official document GitHub Pages Deployment is GITHUB_TOKEN, this document alse lists other directives.

Notice that the values are not escaped when your builds are executed. Special characters (for bash) should be escaped accordingly.

Defining variables, it will be used in file .travis.yml latter.

Environment Variables Value
CNAME_URL axdlog.com
GITHUB_USERNAME MaxdSre
GITHUB_EMAIL [email protected] (pseudo value)
GITHUB_TOKEN 75e8b7y318ebf48df0bc35cp4affd79e167b2489 (pseudo value)

.travis.yml directives

Hugo is written by Golang. If I chose language: go in .travis.yml, the process of build Golang environment costed too much time (a few minutes).

I find someone use Python to deploy successfully, then chose Python as operation environment. As the containers of Travis CI is based on Ubuntu, I can use command dpkg, apt-get, but it needs sudo privilege.

As Python failed to build frequently, because GitHub forbided HTTP request from Travis container. I solve this problem via tor proxy.

The final code is as follow

Python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

# https://docs.travis-ci.com/user/deployment/pages/
# https://docs.travis-ci.com/user/reference/xenial/
# https://docs.travis-ci.com/user/languages/python/
# https://docs.travis-ci.com/user/customizing-the-build/

dist: xenial
language: python
python:
  - "3.7"

before_install:
  - sudo apt-get update -qq
  - sudo apt-get -yq install apt-transport-https tor curl

# install - install any dependencies required
install:
    # install latest release version
    # Github may forbid request from travis container, so use tor proxy
    - sudo systemctl start tor
    # - curl -fsL --socks5-hostname 127.0.0.1:9050 https://api.github.com/repos/gohugoio/hugo/releases/latest | sed -r -n '/browser_download_url/{/Linux-64bit.deb/{[email protected][^:]*:[[:space:]]*"([^"]*)".*@\[email protected];p;q}}' | xargs wget
    - download_command='curl -fsSL -x socks5h://127.0.0.1:9050' # --socks5-hostname
    - $download_command -O $($download_command https://api.github.com/repos/gohugoio/hugo/releases/latest | sed -r -n '/browser_download_url/{/Linux-64bit.deb/{[email protected][^:]*:[[:space:]]*"([^"]*)".*@\[email protected];p;q}}')
    - sudo dpkg -i hugo*.deb
    - rm -rf public 2> /dev/null

# script - run the build script
script:
    - hugo
    - echo "$CNAME_URL" > public/CNAME

deploy:
  provider: pages
  skip-cleanup: true
  github-token: $GITHUB_TOKEN  # Set in travis-ci.org dashboard, marked secure
  email: $GITHUB_EMAIL
  name: $GITHUB_USERNAME
  verbose: true
  keep-history: true
  local-dir: public
  target_branch: master  # branch contains blog content
  on:
    branch: code  # branch contains Hugo generator code

Golang (deprecated)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

# https://docs.travis-ci.com/user/deployment/pages/
# https://docs.travis-ci.com/user/reference/xenial/
# https://docs.travis-ci.com/user/languages/go/
# https://docs.travis-ci.com/user/customizing-the-build/

dist: xenial
language: go
go:
    - master

# before_install
# install - install any dependencies required
install:
    - go get github.com/gohugoio/hugo    # consume time 70.85s

before_script:
    - rm -rf public 2> /dev/null

# script - run the build script
script:
    - hugo
    - echo "$CNAME_URL" > public/CNAME

deploy:
  provider: pages
  skip-cleanup: true
  github-token: $GITHUB_TOKEN  # Set in travis-ci.org dashboard, marked secure
  email: $GITHUB_EMAIL
  name: $GITHUB_USERNAME
  verbose: true
  keep-history: true
  local-dir: public
  target_branch: master  # branch contains blog content
  on:
    branch: code  # branch contains Hugo generator code

Writing it into file .travis.yml, and push file .travis.yml to the branch code of remote repository.

Continuous Integration

After all operations are finished. If you push your changed source file to branch code. Travis CI will automatically execute directives defined in file .travis.yml, then pushing the generated blog contents to branch master.

Deploying log

References

Change logs

  • 2018.04.11 00:20 Wed America/Boston
    • first draft
  • 2018.07.11 12:20 Wed America/Boston
    • add python script for hugo installation
  • 2018.08.03 08:44 Fri Asia/Shanghai
    • add switch branch
  • 2018.12.04 11:46 Tue America/Boston
    • use Golang in travis
  • 2019.01.30 09:38 Wed America/Boston
    • still use Python in travis, short time consuming
  • 2019.04.05 22:44 Fri America/Boston
    • update hugo version to v0.54.0
  • 2019.04.26 12:33 Fri America/Boston
    • update hugo version to v0.55.4
  • 2019.05.02 11:26 Thu America/Boston
    • update hugo version to v0.55.5
  • 2019.05.07 15:47 Tue America/Boston
    • add GitHub Pages limits introduction

Author MaxdSre

LastMod May 2, 2019

License CC BY-NC-ND 4.0