利用Travis CI和Hugo將Blog自動部署到Github Pages

文章目錄

個人Blog採用靜態Blog形式託管在Github上。此前使用的是Hexo,因其包依賴關係複雜,部署流程繁瑣,故將整個部署環境封裝到Docker鏡像中以實現快速部署,但仍較爲繁瑣。現轉用執行速度快、操作簡便的Hugo。本文記錄如何在GNU/Linux中通過Travis CIHugo生成的Blog內容自動同步到Github,實現持續集成、部署。

注意:本文旨在記錄關鍵操作,不涉及帳號註冊、Git安裝配置等過程。

準備工作

  1. Github註冊帳號;
  2. Travis CI註冊帳號(可通過Github帳號登入);
  3. 本地系統安裝githugo
  4. 個人域名(可選,此處指定域名axdlog.com);

官方文檔

本文中的相關操作皆以官方文檔爲操作憑據

安裝 Hugo

Hugo官方的安裝文檔頁Install Hugo,如果使用的是GNU/Linux,可考慮使用本人寫的Python script安裝。

當前版本信息

1
2
3
4
5
# which hugo
/usr/local/bin/hugo

# hugo version
Hugo Static Site Generator v0.43 linux/amd64 BuildDate: 2018-07-09T10:00:08Z

Python腳本

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Writer: MaxdSre
# Date: Wed 2018-07-11 12:10:23 -0400 EDT

# import requests
from urllib.request import urlopen
#from bs4 import BeautifulSoup
import json
import re
import os
import tarfile
# remove unempty dirs
from shutil import rmtree
import subprocess

utility_name = 'Hugo'
hugo_release_api='https://api.github.com/repos/gohugoio/hugo/releases/latest'
symlink_path = '/usr/local/bin/hugo'
pack_save_dir = '/tmp'
target_dir = "/opt/" + utility_name
hugo_binary_path = target_dir + "/hugo"
is_latest_version = True

# - extract release info from api
# raw_data = requests.get(hugo_release_api, timeout=0.5).text
raw_data = urlopen(hugo_release_api).read().decode()

json_data = json.loads(raw_data)

release_version = json_data['tag_name'].lstrip('v')
release_date = json_data['published_at']
release_pack_link=''
release_pack_size=''

for item in json_data['assets']:
    if re.search(r"Linux-64bit.*tar.gz$",item['name']):
        release_pack_link=item['browser_download_url']
        release_pack_size=item['size']
        break
    else:
        continue

# - if existed latest version
if os.path.exists(hugo_binary_path):
    version_info = subprocess.getoutput(hugo_binary_path + " version")
    version_num = re.search(r".*?v([\d.]+).*", version_info).group(1)

    if version_num == release_version:
        print("Latest version {} existed.".format(release_version))
    else:
        is_latest_version = False
        # remove target dir
        if os.path.exists(target_dir) and os.path.isdir(target_dir):
            rmtree(target_dir)
        print("Local version {} < latest version {}.".format(version_num, release_version))
else:
    is_latest_version = False

if is_latest_version == False:
    # remove target dir
    if os.path.exists(target_dir) and os.path.isdir(target_dir):
        rmtree(target_dir)

if not os.path.exists(target_dir):
    # - download
    pack_save_path = pack_save_dir.rstrip("/") + "/" + release_pack_link.split("/")[-1]

    if os.path.exists(pack_save_path) and os.path.getsize(pack_save_path) == release_pack_size:
        print("Find existed pack {}.".format(pack_save_path))
    else:
        if os.path.exists(pack_save_path):
            os.remove(pack_save_path)
        # https://stackoverflow.com/questions/7243750/download-file-from-web-in-python-3
        with open(pack_save_path, "wb") as file:
            # response = requests.get(release_pack_link)
            # file.write(response.content)

            response = urlopen(release_pack_link)
            file.write(response.read())

            if os.path.exists(pack_save_path) and os.path.getsize(pack_save_path) == release_pack_size:
                print("Successfully download pack {}!".format(pack_save_path))

    # - decompress
    tar = tarfile.open(pack_save_path, 'r:gz')
    # 1. extract all file
    tar.extractall(path=target_dir)

    # 2. extract needed file
    # for item in tar:
    #     if not item.name.endswith(".md"):
    #         tar.extract(item, path=target_dir)

    # - create symlink
    if os.path.exists(hugo_binary_path):
        print("Successfully install {} v{}!".format(utility_name, release_version))

        if os.path.islink(symlink_path):
            os.unlink(symlink_path)
            # os.remove(symlink_path)
        os.symlink(hugo_binary_path, symlink_path)
        print("\nSymlink info: \n{}".format(subprocess.getoutput("ls -lh " + symlink_path)))

    # - remove package
    if os.path.exists(pack_save_path):
        os.remove(pack_save_path)

if os.path.exists(hugo_binary_path):
    print("\nHugo info: \n{}".format(subprocess.getoutput(hugo_binary_path + " version")))

# Script End

演示過程

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# sudo python3 ~/hugo.py
Successfully download pack /tmp/hugo_0.43_Linux-64bit.tar.gz!
Successfully install Hugo v0.43!

Symlink info:
lrwxrwxrwx 1 root staff 14 Jul 11 12:09 /usr/local/bin/hugo -> /opt/Hugo/hugo

Hugo info:
Hugo Static Site Generator v0.43 linux/amd64 BuildDate: 2018-07-09T10:00:08Z

# sudo python3 ~/hugo.py
Latest version 0.43 existed.

Hugo info:
Hugo Static Site Generator v0.43 linux/amd64 BuildDate: 2018-07-09T10:00:08Z

創建新項目

如何通過Hugo創建一個新的站點,詳見官方文檔 Get StartedQuick Start

If this is your first time using Hugo and you’ve already installed Hugo on your machine, we recommend the quick start.

此處且將站點名稱命名爲quickstart(其實就是目錄名,不一定是最終的網站名稱)。

Hugo生成的目錄結構如下

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# tree -L 2
.
├── archetypes
│   └── default.md
├── config.toml
├── content
│   ├── about.md
│   └── post
├── data
├── layouts
├── static
└── themes
    └── even

8 directories, 3 files

注意:這些不是最終的Blog內容,是用於生成Blog內容的文件。

目錄content中存放自己需要發佈的Blog (Markdown文件);

如果要預覽Blog最終效果,可通過命令hugo server在本地開啓一個Web服務器,端口號1313,在瀏覽器中打開即可。

1
2
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop

在項目所在目錄中執行命令hugo,會生成名爲public的目錄,該目錄存放着最終的Blog內容文件。將public目錄中的所有內容上傳至Github的master分支中即可。

演示過程如下

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# hugo

                   | EN   
+------------------+-----+
  Pages            | 426  
  Paginator pages  |  24  
  Non-page files   |   0  
  Static files     |  34  
  Processed images |   0  
  Aliases          |  95  
  Sitemaps         |   1  
  Cleaned          |   0  

Total in 403 ms

# tree -L 1
.
├── archetypes
├── config.toml
├── content
├── data
├── layouts
├── public
├── static
└── themes

7 directories, 1 file

配置 Github 倉庫

重要:本部分的處理思路、操作非常重要,Git配置步驟省略。

使用Github Page託管Blog內容,需要創建名爲<username>.github.io的倉庫,並將Hugo生成的Blog內容提交到倉庫的master中。

因Hugo生成的內容分爲兩部分,源文件和public目錄中的Blog內容。故通過git checkout --orphan在同一倉庫中創建2個分支:

  • 分支code用於存放源文件;
  • 分支master用於存放public目錄中的Blog內容;

處理思路與官方文檔 Configuring a publishing source for GitHub Pages不同,需要注意。

創建空倉庫

本人GitHub帳號名爲MaxdSre,故創建倉庫maxdsre.github.io

創建倉庫後,出現如下提示信息

or create a new repository on the command line

1
2
3
4
5
6
echo "# maxdsre.github.io" >> README.md
git init
git add README.md
git commit -m "first commit"
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git
git push -u origin master

or push an existing repository from the command line

1
2
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git
git push -u origin master

分支操作

操作分爲4步:

  1. 初始化Git倉庫(git init);
  2. 創建code分支,提交源文件代碼到倉庫;
  3. 在目標目錄中執行hugo生成public目錄,將public目錄中內容轉移到臨時目錄中, 清空目標目錄,將臨時目錄中文件複製到該目錄中;
  4. 通過命令git checkout --orphan創建master分支,將目錄中內容提交到倉庫。

進入目標目錄(此處爲/PATH/quickstart/)後,順序執行如下操作命令

code 分支

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# Initialized empty Git repository in /PATH/.git/
git init

# Switched to a new branch 'code', equal to 'git branch code && git checkout code'
git checkout -b code

# exclude dir public/
echo '/public/' >> .gitignore
sed -r -i '/^\/public\/$/{$!d}' .gitignore

# Add file contents to the index
git add .

# Show the working tree status
# git status

# Record changes to the repository
git commit -m "Hugo generator code"

# Manage set of tracked repositories
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git

# Update remote refs along with associated objects to branch 'code'
git push -u origin code

master 分支

在目錄中執行hugo生成public目錄。通過命令git branch -a可查看倉庫的分支信息。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# execute command hugo
hugo

# create temporary dir
HUGO_TEMP_DIR=$(mktemp -d)
cp -R public/* "$HUGO_TEMP_DIR"

# create orphan branch 'master'
git checkout --orphan master

# empty current dir
rm .git/index
git clean -fdx

# copy back contents in dir public/
cp -R "$HUGO_TEMP_DIR"/. .

# add, commit, push
git add .
# git status
git commit -m 'initial blog content'
git push -u origin master

# remove temp dir
[[ -d "$HUGO_TEMP_DIR" ]] && rm -rf "$HUGO_TEMP_DIR"

image 分支

此操作爲可選操作,本人創建第3個分支image用作圖牀,本文中的圖片即存放在該分支中。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
git checkout --orphan image
rm .git/index
git clean -fdx

# add content to this branch

git add .
git commit -m 'initial blog images'
git push -u origin image

# fatal: The current branch image has no upstream branch.
# To push the current branch and set the remote as upstream, use
# git push --set-upstream origin image

切換分支

如果要將本地分支切換到遠程分支remotes/origin/image,可通過如下命令實現

1
2
3
4
5
6
7
8
# git checkout -b <branch> --track <remote>/<branch>
# -b <new_branch>    Create a new branch named <new_branch> and start it at <start_point>;
# -t, --track     When creating a new branch, set up "upstream" configuration.

git checkout -t remotes/origin/image

# 本地分支,從其它分支切換到本地分支code
git checkout code

演示過程如下

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# git branch
* code

# git branch -a
* code
  remotes/origin/HEAD -> origin/code
  remotes/origin/code
  remotes/origin/image
  remotes/origin/master

# git checkout -t remotes/origin/image
Branch 'image' set up to track remote branch 'image' from 'origin'.
  Switched to a new branch 'image'

# git branch
  code
* image

# git branch -a
  code
* image
  remotes/origin/HEAD -> origin/code
  remotes/origin/code
  remotes/origin/image
  remotes/origin/master

刪除分支

可分爲刪除遠程分支、本地分支、追蹤分支三種情況,具體見How do I delete a Git branch both locally and remotely?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# 1 - 刪除遠程分支
$ git push origin --delete <branch> # Git version 1.7.0 or newer
$ git push origin :<branch> # Git versions older than 1.7.0

# 2 - 刪除本地分支
$ git branch --delete <branch>
$ git branch -d <branch> # Shorter version

$ git branch -D <branch> # Force delete un-merged branches

# error: The branch 'image' is not fully merged.
# If you are sure you want to delete it, run 'git branch -D image'.

# 3 - 刪除本地追蹤分支
$ git branch --delete --remotes <remote>/<branch>
$ git branch -dr <remote>/<branch> # Shorter

$ git fetch <remote> --prune # Delete multiple obsolete tracking branches
$ git fetch <remote> -p # Shorter

Travis CI 配置

Travis CI官方說明文檔 GitHub Pages Deployment

生成 Github 訪問 Token

You’ll need to generate a personal access token with the public_repo or repo scope (repo is required for private repositories). Since the token should be private, you’ll want to pass it to Travis securely in your repository settings or via encrypted variables in .travis.yml.

在Github的Developer settings頁面中生成personal access tokens,若是私有倉庫,勾選repo;若是公開倉庫,勾選public_repo。此處本人選擇了 public_repo, repo:status, repo_deployment三項。

生成的是長度爲40的隨機字符串,注意請勿泄漏。

設置環境變量

生成的token須在Travis CI的目標Repo中設置,頁面地址爲 https://travis-ci.org/MaxdSre/maxdsre.github.io/settings

確保General中的Build only if .travis.yml is present, Build pushed branches, Build pushed pull requests已啓用。

官方文檔GitHub Pages Deployment定義的默認token名爲GITHUB_TOKEN,其它指令詳見文檔。

Notice that the values are not escaped when your builds are executed. Special characters (for bash) should be escaped accordingly.

此處定義相關變量,稍後會寫入.travis.yml文件。

Environment Variables Value
GITHUB_USERNAME MaxdSre
GITHUB_EMAIL [email protected]
GITHUB_TOKEN 75e8b72618ebf48df0b235cp4affd79e167b2489 (假設值)

.travis.yml 指令

Hugo是用Golang構建的,故最開始在.travis.yml中用language: go,但Travis CI構建Golang環境耗時太長(數分鐘),且出現hugo安裝失敗的清空。看到他人教程中有用Python的,恰巧最近在學習Python,便選擇Python做爲構建環境。因Travis CI的容器環境是基於Ubuntu的,故可以使用dpkgapt-get等命令,但需要添加sudo

經過反復測試,最終指令如下

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# https://docs.travis-ci.com/user/deployment/pages/
# https://docs.travis-ci.com/user/languages/python/
language: python

python:
    - "3.6"

install:
    # install latest release version
    #- wget $(wget -qO- https://api.github.com/repos/gohugoio/hugo/releases/latest | sed -r -n '/browser_download_url/{/Linux-64bit.deb/{[email protected][^:]*:[[:space:]]*"([^"]*)".*@\[email protected];p}}')
    - wget -qO- https://api.github.com/repos/gohugoio/hugo/releases/latest | sed -r -n '/browser_download_url/{/Linux-64bit.deb/{[email protected][^:]*:[[:space:]]*"([^"]*)".*@\[email protected];p}}' | xargs wget
    - sudo dpkg -i hugo*.deb
    - pip install Pygments
    - rm -rf public 2> /dev/null

script:
    - hugo
    - echo 'axdlog.com' > public/CNAME

deploy:
  provider: pages
  skip-cleanup: true
  github-token: $GITHUB_TOKEN  # Set in travis-ci.org dashboard, marked secure
  email: $GITHUB_EMAIL
  name: $GITHUB_USERNAME
  verbose: true
  keep-history: true
  local-dir: public
  target_branch: master  # branch contains blog content
  on:
    branch: code  # branch contains Hugo generator code

將指令寫入到文件.travis.yml中,並將該.travis.yml上傳至repo的code分支中。

持續集成

所有操作完成後,只需將需要發佈的Blog上傳到repo的code分支中,Travis CI會自動進行部署操作,將更新的Blog內容更新到repo的master分支中,實現Blog的持續集成,自動部署。

部署過程日誌

參考資料

更新日誌

  • 2018-04-11 00:20 Wed America/Boston
    • 初稿完成
  • 2018.07.11 12:20 Wed America/Boston
    • 添加安裝hugo的python腳本
  • 2018.08.03 08:44 Fri Asia/Shanghai
    • 添加切換分支
顯示 Disqus 評論