My personal static blog is hosted on Github. I used Hexo before, but as its complicated package dependence and cumbersome deployment process, I have to encapsulated the whole deploying environment into Docker image for rapid deployment. But it was still too complex. Now I switch to Hugo which is faster, simpler.
This article documents how to automatically synchronize blog contents generated by Hugo to Github through Travis CI.
Attention: The following operation process will not involve account registration, Git configuration.
Prerequisite
- Registering a Github account;
- Registering a Travis CI account (login with Github account);
- Installing
git, hugo in your system;
- Personal domain (optional, here my domain is
axdlog.com);
Official Docs
The relevant operations in this document are based on the official document as the operation credentials:
Hugo Installation
Install Hugo is the official installation document of Hugo. If you’re using GNU/Linux, you may consider using my Python script to install it.
Current release version info
1
2
3
4
5
|
# which hugo
/usr/local/bin/hugo
# hugo version
Hugo Static Site Generator v0.43 linux/amd64 BuildDate: 2018-07-09T10:00:08Z
|
Python Script
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
|
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Writer: MaxdSre
# Date: Wed 2018-07-11 12:10:23 -0400 EDT
# import requests
from urllib.request import urlopen
#from bs4 import BeautifulSoup
import json
import re
import os
import tarfile
# remove unempty dirs
from shutil import rmtree
import subprocess
utility_name = 'Hugo'
hugo_release_api='https://api.github.com/repos/gohugoio/hugo/releases/latest'
symlink_path = '/usr/local/bin/hugo'
pack_save_dir = '/tmp'
target_dir = "/opt/" + utility_name
hugo_binary_path = target_dir + "/hugo"
is_latest_version = True
# - extract release info from api
# raw_data = requests.get(hugo_release_api, timeout=0.5).text
raw_data = urlopen(hugo_release_api).read().decode()
json_data = json.loads(raw_data)
release_version = json_data['tag_name'].lstrip('v')
release_date = json_data['published_at']
release_pack_link=''
release_pack_size=''
for item in json_data['assets']:
if re.search(r"Linux-64bit.*tar.gz$",item['name']):
release_pack_link=item['browser_download_url']
release_pack_size=item['size']
break
else:
continue
# - if existed latest version
if os.path.exists(hugo_binary_path):
version_info = subprocess.getoutput(hugo_binary_path + " version")
version_num = re.search(r".*?v([\d.]+).*", version_info).group(1)
if version_num == release_version:
print("Latest version {} existed.".format(release_version))
else:
is_latest_version = False
# remove target dir
if os.path.exists(target_dir) and os.path.isdir(target_dir):
rmtree(target_dir)
print("Local version {} < latest version {}.".format(version_num, release_version))
else:
is_latest_version = False
if is_latest_version == False:
# remove target dir
if os.path.exists(target_dir) and os.path.isdir(target_dir):
rmtree(target_dir)
if not os.path.exists(target_dir):
# - download
pack_save_path = pack_save_dir.rstrip("/") + "/" + release_pack_link.split("/")[-1]
if os.path.exists(pack_save_path) and os.path.getsize(pack_save_path) == release_pack_size:
print("Find existed pack {}.".format(pack_save_path))
else:
if os.path.exists(pack_save_path):
os.remove(pack_save_path)
# https://stackoverflow.com/questions/7243750/download-file-from-web-in-python-3
with open(pack_save_path, "wb") as file:
# response = requests.get(release_pack_link)
# file.write(response.content)
response = urlopen(release_pack_link)
file.write(response.read())
if os.path.exists(pack_save_path) and os.path.getsize(pack_save_path) == release_pack_size:
print("Successfully download pack {}!".format(pack_save_path))
# - decompress
tar = tarfile.open(pack_save_path, 'r:gz')
# 1. extract all file
tar.extractall(path=target_dir)
# 2. extract needed file
# for item in tar:
# if not item.name.endswith(".md"):
# tar.extract(item, path=target_dir)
# - create symlink
if os.path.exists(hugo_binary_path):
print("Successfully install {} v{}!".format(utility_name, release_version))
if os.path.islink(symlink_path):
os.unlink(symlink_path)
# os.remove(symlink_path)
os.symlink(hugo_binary_path, symlink_path)
print("\nSymlink info: \n{}".format(subprocess.getoutput("ls -lh " + symlink_path)))
# - remove package
if os.path.exists(pack_save_path):
os.remove(pack_save_path)
if os.path.exists(hugo_binary_path):
print("\nHugo info: \n{}".format(subprocess.getoutput(hugo_binary_path + " version")))
# Script End
|
operating process
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
# sudo python3 ~/hugo.py
Successfully download pack /tmp/hugo_0.43_Linux-64bit.tar.gz!
Successfully install Hugo v0.43!
Symlink info:
lrwxrwxrwx 1 root staff 14 Jul 11 12:09 /usr/local/bin/hugo -> /opt/Hugo/hugo
Hugo info:
Hugo Static Site Generator v0.43 linux/amd64 BuildDate: 2018-07-09T10:00:08Z
# sudo python3 ~/hugo.py
Latest version 0.43 existed.
Hugo info:
Hugo Static Site Generator v0.43 linux/amd64 BuildDate: 2018-07-09T10:00:08Z
|
Create A New Site
You can reference official document Get Started、Quick Start to know how to create a new site through Hugo.
If this is your first time using Hugo and you’ve already installed Hugo on your machine, we recommend the quick start.
Here I name the site to quickstart (it is just the directory name, not the final site name).
Generated directory structure by Hugo:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
# tree -L 2
.
├── archetypes
│ └── default.md
├── config.toml
├── content
│ ├── about.md
│ └── post
├── data
├── layouts
├── static
└── themes
└── even
8 directories, 3 files
|
Attention: These files are not the final blog contents, they are just used to generate the blog contents.
Directory content is used to store your blog file (Markdown file).
If you wanna preview the final effect, just executing command hugo server to set up a local server which is listening port 1313. And opening the link in your browser.
1
2
|
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop
|
Executing command hugo in your project directory, it will generate a directory named public. This directory contains the final blog contents. What you need to do is push all the files in this directory to the branch master of your GitHub repository.
The demonstration process is as follows:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
# hugo
| EN
+------------------+-----+
Pages | 426
Paginator pages | 24
Non-page files | 0
Static files | 34
Processed images | 0
Aliases | 95
Sitemaps | 1
Cleaned | 0
Total in 403 ms
# tree -L 1
.
├── archetypes
├── config.toml
├── content
├── data
├── layouts
├── public
├── static
└── themes
7 directories, 1 file
|
Github Repository Operation
Important: The handling ideas and operations of this section are very important. Not involving Git setting procedure.
If you use Github Page to host blog, you need to create a repository named <username>.github.io. Then putting the files generated by Hugo to the branch master of this repository.
I create 2 branches via command git checkout --orphan to store source files and blog contents in directory public separately.
- branch
code stores source files;
- branch
master stores blog contents in directory public;
Attention please, this method is different from official document Configuring a publishing source for GitHub Pages.
Create Empty Repository
My GitHub account is MaxdSre, so I need to create a repository named maxdsre.github.io.
After creating the repository, it will prompts
or create a new repository on the command line
1
2
3
4
5
6
|
echo "# maxdsre.github.io" >> README.md
git init
git add README.md
git commit -m "first commit"
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git
git push -u origin master
|
or push an existing repository from the command line
1
2
|
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git
git push -u origin master
|
Pushing Repository
There are 4 steps:
- Initializing Git repository via command
git init in project directory;
- Creating branch
code, pushing source files to remote repository;
- Executing command
hugo to generate public directory; Transferring all the files in this directory to another temporary directory, empty project directory, then move the files store in temporary directory back to project directory;
- Creating branch
master via command git checkout --orphan, then pushing all files store in current directory to remote repository.
Entering the target directory (here is /PATH/quickstart/), then executing the following commands in sequence.
Branch code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
# Initialized empty Git repository in /PATH/.git/
git init
# Switched to a new branch 'code', equal to 'git branch code && git checkout code'
git checkout -b code
# exclude dir public/
echo '/public/' >> .gitignore
sed -r -i '/^\/public\/$/{$!d}' .gitignore
# Add file contents to the index
git add .
# Show the working tree status
# git status
# Record changes to the repository
git commit -m "Hugo generator code"
# Manage set of tracked repositories
git remote add origin [email protected]:MaxdSre/maxdsre.github.io.git
# Update remote refs along with associated objects to branch 'code'
git push -u origin code
|
Branch master
Executing command hugo in project directly to generate directly public. If you wanna check the branch info of this repository, just executing command git branch -a.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
# execute command hugo
hugo
# create temporary dir
HUGO_TEMP_DIR=$(mktemp -d)
cp -R public/* "$HUGO_TEMP_DIR"
# create orphan branch 'master'
git checkout --orphan master
# empty current dir
rm .git/index
git clean -fdx
# copy back contents in dir public/
cp -R "$HUGO_TEMP_DIR"/. .
# add, commit, push
git add .
# git status
git commit -m 'initial blog content'
git push -u origin master
# remove temp dir
[[ -d "$HUGO_TEMP_DIR" ]] && rm -rf "$HUGO_TEMP_DIR"
|
Branch image
This section is optional. In order to store images in GitHub directly, I create the third branch image. All images used in blog will be saved in this branch.
1
2
3
4
5
6
7
8
9
10
11
12
13
|
git checkout --orphan image
rm .git/index
git clean -fdx
# add content to this branch
git add .
git commit -m 'initial blog images'
git push -u origin image
# fatal: The current branch image has no upstream branch.
# To push the current branch and set the remote as upstream, use
# git push --set-upstream origin image
|
Switching branch
If you wanna switch your local branch to remote branch remotes/origin/image, you may consider executing the following command
1
2
3
4
5
6
7
8
|
# git checkout -b <branch> --track <remote>/<branch>
# -b <new_branch> Create a new branch named <new_branch> and start it at <start_point>;
# -t, --track When creating a new branch, set up "upstream" configuration.
git checkout -t remotes/origin/image
# local branch, switch from other branch to branch 'code'
git checkout code
|
Demonstration example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
# git branch
* code
# git branch -a
* code
remotes/origin/HEAD -> origin/code
remotes/origin/code
remotes/origin/image
remotes/origin/master
# git checkout -t remotes/origin/image
Branch 'image' set up to track remote branch 'image' from 'origin'.
Switched to a new branch 'image'
# git branch
code
* image
# git branch -a
code
* image
remotes/origin/HEAD -> origin/code
remotes/origin/code
remotes/origin/image
remotes/origin/master
|
Deleting branch
Deleting branches has three types: remote branch, local branch, local remote-tracking branch, more details in How do I delete a Git branch both locally and remotely?.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
# 1 - Delete a remote branch
$ git push origin --delete <branch> # Git version 1.7.0 or newer
$ git push origin :<branch> # Git versions older than 1.7.0
# 2 - Delete a local branch
$ git branch --delete <branch>
$ git branch -d <branch> # Shorter version
$ git branch -D <branch> # Force delete un-merged branches
# error: The branch 'image' is not fully merged.
# If you are sure you want to delete it, run 'git branch -D image'.
# 3 - Delete a local remote-tracking branch
$ git branch --delete --remotes <remote>/<branch>
$ git branch -dr <remote>/<branch> # Shorter
$ git fetch <remote> --prune # Delete multiple obsolete tracking branches
$ git fetch <remote> -p # Shorter
|
Travis CI Configuration
GitHub Pages Deployment is the official document of Travis CI。
Github Access Token Generation
You’ll need to generate a personal access token with the public_repo or repo scope (repo is required for private repositories). Since the token should be private, you’ll want to pass it to Travis securely in your repository settings or via encrypted variables in .travis.yml.
Generating personal access tokens in GitHub page Developer settings. If it is a private repository, choose repo. Otherwise, just choose public_repo. Here I choose public_repo, repo:status, repo_deployment.
The length of the generated token is 40, please keep is private.
Environment Variables Setting
Generated token is used in Travis CI, the page url of target repository in Travis CI is https://travis-ci.org/MaxdSre/maxdsre.github.io/settings.
Don’t forget to check Build only if .travis.yml is present, Build pushed branches, Build pushed pull requests.
The default name of token defined in official document GitHub Pages Deployment is GITHUB_TOKEN, this document alse lists other directives.
Notice that the values are not escaped when your builds are executed. Special characters (for bash) should be escaped accordingly.
Defining variables, it will be used in file .travis.yml latter.
| Environment Variables |
Value |
| GITHUB_USERNAME |
MaxdSre |
| GITHUB_EMAIL |
[email protected] |
| GITHUB_TOKEN |
75e8b72618ebf48df0b235cp4affd79e167b2489 (假設值) |
.travis.yml directives
Hugo is written by Golang. So I chose language: go in .travis.yml primitively, but the process of build Golang environment costed too much time (a few minutes). What worse is the container fail to install hugo. I find someone use Python to deploy successfully, then chose Python as operation environment. As the containers of Travis CI is based on Ubuntu, I can use command dpkg, apt-get, but it needs sudo privilege.
Through repeated tests, the final code is as follow
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
# https://docs.travis-ci.com/user/deployment/pages/
# https://docs.travis-ci.com/user/languages/python/
language: python
python:
- "3.6"
install:
# install latest release version
# - wget $(wget -qO- https://api.github.com/repos/gohugoio/hugo/releases/latest | sed -r -n '/browser_download_url/{/Linux-64bit.deb/{[email protected][^:]*:[[:space:]]*"([^"]*)".*@\[email protected];p}}')
- wget -qO- https://api.github.com/repos/gohugoio/hugo/releases/latest | sed -r -n '/browser_download_url/{/Linux-64bit.deb/{[email protected][^:]*:[[:space:]]*"([^"]*)".*@\[email protected];p}}' | xargs wget
- sudo dpkg -i hugo*.deb
- pip install Pygments
- rm -rf public 2> /dev/null
script:
- hugo
- echo 'axdlog.com' > public/CNAME
deploy:
provider: pages
skip-cleanup: true
github-token: $GITHUB_TOKEN # Set in travis-ci.org dashboard, marked secure
email: $GITHUB_EMAIL
name: $GITHUB_USERNAME
verbose: true
keep-history: true
local-dir: public
target_branch: master # branch contains blog content
on:
branch: code # branch contains Hugo generator code
|
Writing it into file .travis.yml, and push file .travis.yml to the branch code of remote repository.
Continuous Integration
After all operations are finished. If you push your changed source file to branch code. Travis CI will automatically execute directives defined in file .travis.yml, then pushing the generated blog contents to branch master.
Deploying log
References
Change logs
- 2018.04.11 00:20 Wed America/Boston
- 2018.07.11 12:20 Wed America/Boston
- add python script for hugo installation
- 2018.08.03 08:44 Fri Asia/Shanghai