Extracting Percona Version Lists For Per Supported GNU/Linux Distribution

Percona支持RHEL/CentOS/Debian/Ubuntu等GNU/Linux發行版，在 Installing Percona Server from Repositories 中列出了具體支持的發行版本，但並未顯式地說明Percona Server for MySQL的每一個釋出版本(如8.0、5.6、5.7)具體支持哪些GNU/Linux發行版本。

本文通過 Percona Software Downloads 提取各個可供下載的Percona版本的下載頁鏈接，再通過下載頁的HTML代碼提取各版本具體支持的GNU/Linux發行版，最後通過Shell Script代碼實現。

爲實現 MySQL Variants (MySQL、MariaDB、Percona)在GNU/Linux中的自動安裝、配置，本人通過Shell腳本提取其對各GNU/Linux發行版本的具體支持信息：

數據庫系統安裝腳本代碼託管在GitLab，腳本同時支持在Debian/Ubuntu/CentOS/Fedora/OpenSUSE/SLES等發行版中安裝MySQL、MariaDB、Percona。

1
2
3
4


# curl -fsL / wget -qO-

# if need help info, specify '-h'
wget -qO- https://gitlab.com/Maxdong/axd-ShellScript/raw/master/assets/software/MySQLVariants.sh | sudo bash -s --

Announcement

Platform End of Life (EOL) Announcement for RHEL 5 and Ubuntu 12.04 LTS

RHEL 5 was EOL as of March 31st, 2017 and Ubuntu 12.04 LTS was end of life as of April 28th, 2017.

End of Life Announcement for Query Analyzer and MySQL Configuration Generator

Percona will no longer be offering the free services of Query Analyzer and MySQL Configuration Generator hosted at tools.percona.com. We made this decision based on observed low usage numbers, outdated technology, and lack of support for MySQL 5.7+, MongoDB, and PostgreSQL.

Shell Script

整個操作過程已通過Shell腳本實現，代碼託管在GitLab，通過如下命令執行

1
2
3


download_tool='wget -qO-'  # curl -fsL / wget -qO-
# if need help info, specify '-h'
$download_tool https://gitlab.com/Maxdong/axd-ShellScript/raw/master/assets/tool/mysqlVariantsVersionAndLinuxDistroRelationTable.sh | bash -s -- -d percona

提取結果

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


Percona Server|xenial|8.0 5.7 5.6 5.5
Percona Server|trusty|5.7 5.6 5.5
Percona Server|stretch|8.0 5.7 5.6 5.5
Percona Server|squeeze|5.1
Percona Server|rhel7|8.0 5.7 5.6 5.5|rhel centos
Percona Server|rhel6|8.0 5.7 5.6 5.5 5.1|rhel centos
Percona Server|rhel5|5.1|rhel centos
Percona Server|precise|5.1
Percona Server|lucid|5.1
Percona Server|jessie|5.7 5.6 5.5
Percona Server|cosmic|8.0 5.7 5.6
Percona Server|bionic|8.0 5.7 5.6 5.5
Percona XtraDB Cluster|xenial|5.7 5.6
Percona XtraDB Cluster|wheezy|5.5
Percona XtraDB Cluster|trusty|5.7 5.6 5.5
Percona XtraDB Cluster|stretch|5.7 5.6
Percona XtraDB Cluster|rhel7|5.7 5.6 5.5|rhel centos
Percona XtraDB Cluster|rhel6|5.7 5.6 5.5|rhel centos
Percona XtraDB Cluster|rhel5|5.5|rhel centos
Percona XtraDB Cluster|precise|5.5
Percona XtraDB Cluster|jessie|5.7 5.6
Percona XtraDB Cluster|cosmic|5.7 5.6
Percona XtraDB Cluster|bionic|5.7 5.6

Requirement

提取每一個Percona所支持的GNU/Linux發行版本具體支持的Percona版本列表

Percona具體支持哪些GNU/Linux發行版；
每一個GNU/Linux發行版具體支持Percona哪些版本；

Analysis

通過分析Percona Software Downloads頁面，提取Percona目前提供下載的版本號及對應的下載頁鏈接。

1
2
3
4
5
6
7


$download_tool https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{[email protected]* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\[email protected];p}}'

# 8.0 https://www.percona.com/downloads/Percona-Server-LATEST/
# 5.7 https://www.percona.com/downloads/Percona-Server-5.7/LATEST/
# 5.6 https://www.percona.com/downloads/Percona-Server-5.6/LATEST/
# 5.5 https://www.percona.com/downloads/Percona-Server-5.5/LATEST/
# 5.1 https://www.percona.com/downloads/Percona-Server-5.1/LATEST/

通過下載頁鏈接，提取具體支持的GNU/Linux發行版。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


# Latest 8.0
$download_tool https://www.percona.com/downloads/Percona-Server-LATEST/ | sed -r -n '/Select Software Platform/{[email protected]/@[email protected];[email protected]<\/option>@\[email protected];p}' | sed -r -n '[email protected]*/([^"]*)"[[:space:]]*>.*@\[email protected];/binary|source|select|^$/d;p'

# stretch
# rhel6
# rhel7
# xenial
# bionic
# cosmic

# 多行轉換爲一行
sed -r ':a;N;$!ba;[email protected]\[email protected] @g;'
# stretch rhel6 rhel7 xenial bionic cosmic

Code Snippets

通過兩種方式實現：while循環和parallel並行操作。

Original

利用while循環，較爲耗時

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


#!/usr/bin/env bash

download_tool=${download_tool:-'wget -qO-'}  # curl -fsL
version_list_info=$(mktemp -t tempXXXXX.txt)

# - version|distro_list
$download_tool https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{[email protected]* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\[email protected];p}}' | while read -r version url; do
    distro_list=${distro_list:-}
    distro_list=$($download_tool $url | sed -r -n '/Select Software Platform/{[email protected]/@[email protected];[email protected]<\/option>@\[email protected];p}' | sed -r -n '[email protected]*/([^"]*)"[[:space:]]*>.*@\[email protected];/binary|source|select|^$/d;p' | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
    echo "${version}|${distro_list}" >> "${version_list_info}"
done

cat "${version_list_info}"
# 8.0|stretch rhel6 rhel7 xenial bionic cosmic
# 5.7|jessie stretch rhel6 rhel7 trusty xenial bionic cosmic
# 5.6|jessie stretch rhel6 rhel7 trusty xenial bionic cosmic
# 5.5|jessie stretch rhel6 rhel7 trusty xenial bionic
# 5.1|squeeze rhel5 rhel6 lucid precise

# - distro|version_list
sed -r '[email protected]*\|@@g' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;[email protected] @\[email protected]' | awk '!a[$0]++' | while read -r distro; do
    version_list=$(awk -F\| 'match($NF,/[[:space:]]*rhel6[[:space:]]*/){print $1}' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
    echo "$distro|$version_list"
done

# stretch|8.0 5.7 5.6 5.5 5.1
# rhel6|8.0 5.7 5.6 5.5 5.1
# rhel7|8.0 5.7 5.6 5.5 5.1
# xenial|8.0 5.7 5.6 5.5 5.1
# bionic|8.0 5.7 5.6 5.5 5.1
# cosmic|8.0 5.7 5.6 5.5 5.1
# jessie|8.0 5.7 5.6 5.5 5.1
# trusty|8.0 5.7 5.6 5.5 5.1
# squeeze|8.0 5.7 5.6 5.5 5.1
# rhel5|8.0 5.7 5.6 5.5 5.1
# lucid|8.0 5.7 5.6 5.5 5.1
# precise|8.0 5.7 5.6 5.5 5.1

[[ -f "${version_list_info}" ]] && rm -f "${version_list_info}"
unset version_list_info

Parallel

利用parallel命令並行操作，較節省時間

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61


#!/usr/bin/env bash

download_tool=${download_tool:-'curl -fsL'}
version_list_info=$(mktemp -t tempXXXXX.txt)
export version_list_info="${version_list_info}"

# - version|distro_list
funcDistroListForPerVersion(){
    # $1 = 5.7 https://www.percona.com/downloads/Percona-Server-LATEST/
    local item=${1:-}
    if [[ -n "${item}" ]]; then
        local version=${version:-}
        local version_url=${version_url:-}
        local distro_list=${distro_list:-}
        version="${item%% *}"
        version_url="${item##* }"
        distro_list=$($download_tool $version_url | sed -r -n '/Select Software Platform/{[email protected]/@[email protected];[email protected]<\/option>@\[email protected];p}' | sed -r -n '[email protected]*/([^"]*)"[[:space:]]*>.*@\[email protected];/binary|source|select|^$/d;p' | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
        echo "${version}|${distro_list}" >> "${version_list_info}"
    fi
}

export -f funcDistroListForPerVersion
export download_tool="${download_tool}"
$download_tool https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{[email protected]* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\[email protected];p}}' | parallel -k -j 0 funcDistroListForPerVersion 2> /dev/null

cat "${version_list_info}"
# 5.5|jessie stretch rhel6 rhel7 trusty xenial bionic
# 5.1|squeeze rhel5 rhel6 lucid precise
# 8.0|stretch rhel6 rhel7 xenial bionic cosmic
# 5.6|jessie stretch rhel6 rhel7 trusty xenial bionic cosmic
# 5.7|jessie stretch rhel6 rhel7 trusty xenial bionic cosmic


# - distro|version_list
funcVersionListPerDistro(){
    local distro=${1:-}
    local version_list=${version_list:-}
    version_list=$(awk -F\| 'match($NF,/[[:space:]]*'"${distro}"'[[:space:]]*/){print $1}' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;')
    echo "$distro|$version_list"
}

export version_list_info="${version_list_info}"
export -f funcVersionListPerDistro
sed -r '[email protected]*\|@@g' "${version_list_info}" | sed -r ':a;N;$!ba;[email protected]\[email protected] @g;[email protected] @\[email protected]' | awk '!a[$0]++' | parallel -k -j 0 funcVersionListPerDistro 2> /dev/null

# jessie|5.5 5.6 5.7
# stretch|5.5 8.0 5.6 5.7
# rhel6|5.5 5.1 8.0 5.6 5.7
# rhel7|5.5 8.0 5.6 5.7
# trusty|5.5 5.6 5.7
# xenial|5.5 8.0 5.6 5.7
# bionic|5.5 8.0 5.6 5.7
# squeeze|5.1
# rhel5|5.1
# lucid|5.1
# precise|5.1
# cosmic|8.0 5.6 5.7


[[ -f "${version_list_info}" ]] && rm -f "${version_list_info}"
unset version_list_info

Performance Comparison

使用bash內置命令time對操作進行計時

1
2
3
4
5
6
7
8
9


# Original Time
real	0m3.334s
user	0m0.090s
sys	0m0.023s

# Parallel Time
real	0m1.730s
user	0m0.443s
sys	0m0.128s

可以看到使用parallel，操作耗時節省接近一半時間。

Reference

Change Logs

2017.08.13 21:26 Sun Asia/Shanghai
- 初稿完成
2018.07.27 15:39 Fri America/Boston
- 勘誤，更新，遷移到新Blog
2019.04.26 13:09 Fri America/Boston
- 提取結果更新，增加8.0

Table of Contents