码迷,mamicode.com
首页 > 其他好文 > 详细

在隔离环境中搭建 Mirror 站点

时间:2020-09-23 23:33:55      阅读:38      评论:0      收藏:0      [点我收藏+]

标签:output   pes   etc   今天   level   fedora   size   fun   false   

搭建 Mirror 站点的工具有很多,但从根源上讲,他们都使用了 rsync 作为远程同步的工具和手段,因此可称这一类镜像工具为 rsync 镜像工具。在诸多 rsync 镜像工具,使用和配置的便捷性当属 清华大学 开放出的 tunasync 工具,当然它也是一个 Golang 开发而成的 rsync 镜像工具。

之所以会展示这篇操作文档,是因为今天有个我去年带过的学生需要在隔离环境中搭建一个 对阿里云CentOS站点进行镜像的站点,由于操作不当导致镜像失败、且无法Web访问服务。

以下是我使用 tunasync 搭建隔离环境中的 Mirror 站点的过程:

OS环境为 CentOS 7 1804 64位

先对 CentOS 7 做初始化,初始化脚本如下:

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.original
wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
yum clean all && yum makecache
yum -y update
systemctl stop firewalld && systemctl disable firewalld
sed -i ‘s/^SELINUX=./SELINUX=disabled/‘ /etc/selinux/config && setenforce 0
wget https://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm
yum install -y https://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
sed -i ‘s|^#baseurl=https://download.fedoraproject.org/pub|baseurl=https://mirrors.aliyun.com|‘ /etc/yum.repos.d/epel

sed -i ‘s|^metalink|#metalink|‘ /etc/yum.repos.d/epel*
yum clean all && yum makecache
yum -y update
yum install -y ntpdate
wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
yum localinstall -y google-chrome-stable_current_x86_64.rpm
ntpdate cn.ntp.org.cn
yum install -y tree
who | grep googlebigtable | sed -n ‘1p‘ | cut -d‘ ‘ -f 1 | sort | uniq
DescriptionUser=$(who | grep googlebigtable | sed -n ‘1p‘ | cut -d‘ ‘ -f 1 | sort | uniq)
echo $DescriptionUser
echo "$DescriptionUser ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
init 6

创建用户mirrors及用户组mirrorgroup
[googlebigtable@localhost tunasyncMirrorWeb]$ sudo su

[root@localhost tunasyncMirrorWeb]# groupadd mirrorgroup

[root@localhost tunasyncMirrorWeb]# useradd -g mirrorgroup mirrors

[root@localhost tunasyncMirrorWeb]# id mirrors

uid=1001(mirrors) gid=1001(mirrorgroup) groups=1001(mirrorgroup)

[root@localhost tunasyncMirrorWeb]# passwd mirrors

Changing password for user mirrors.

New password:

BAD PASSWORD: The password fails the dictionary check - it is based on a dictionary word

Retype new password:

passwd: all authentication tokens updated successfully.

[root@localhost tunasyncMirrorWeb]# id mirrors

uid=1001(mirrors) gid=1001(mirrorgroup) groups=1001(mirrorgroup)

[root@localhost tunasyncMirrorWeb]#

建立应用程序及数据目录
[root@localhost tunasyncMirrorWeb]# mkdir -p /opt/mirrors/tunasync/{conf,db,log}

[root@localhost tunasyncMirrorWeb]# ls -F /opt/mirrors/tunasync/

conf/ db/ log/

[root@localhost tunasyncMirrorWeb]#

[root@localhost tunasyncMirrorWeb]# mkdir -p /opt/mirrors/mirrorsData

[root@localhost tunasyncMirrorWeb]# ls -F /opt/mirrors/

mirrorsData/ tunasync/

[root@localhost tunasyncMirrorWeb]# chown -R mirrors:mirrorgroup /opt/mirrors/mirrorsData

[root@localhost tunasyncMirrorWeb]# ll /opt/mirrors/

total 0

drwxr-xr-x 2 mirrors mirrorgroup 6 Sep 20 06:03 mirrorsData

drwxr-xr-x 4 root root 28 Sep 20 06:00 tunasync

[root@localhost tunasyncMirrorWeb]#

部署 tunasync
[root@localhost tunasyncMirrorWeb]# ls -F

tunasync-linux-amd64-bin.tar.gz

[root@localhost tunasyncMirrorWeb]# chmod 777 tunasync-linux-amd64-bin.tar.gz

[root@localhost tunasyncMirrorWeb]# tar -xzvf tunasync-linux-amd64-bin.tar.gz -C /opt/mirrors/tunasync/

tunasync

tunasynctl

[root@localhost tunasyncMirrorWeb]# ls -F /opt/mirrors/tunasync/

conf/ db/ tunasync tunasynctl

[root@localhost tunasyncMirrorWeb]#

配置环境变量
[root@localhost tunasyncMirrorWeb]# chown -R mirrors:mirrorgroup /opt/mirrors

[root@localhost tunasyncMirrorWeb]# ll /opt/mirrors

total 0

drwxr-xr-x 2 mirrors mirrorgroup 6 Sep 20 06:03 mirrorsData

drwxr-xr-x 4 mirrors mirrorgroup 62 Sep 20 06:07 tunasync

[root@localhost tunasyncMirrorWeb]# ll /opt/mirrors/tunasync/

total 28032

drwxr-xr-x 2 mirrors mirrorgroup 6 Sep 20 06:00 conf

drwxr-xr-x 2 mirrors mirrorgroup 6 Sep 20 06:00 db

-rwxr-xr-x 1 mirrors mirrorgroup 17930410 Sep 11 06:24 tunasync

-rwxr-xr-x 1 mirrors mirrorgroup 10770787 Sep 11 06:24 tunasynctl

[root@localhost tunasyncMirrorWeb]# su - mirrors

[mirrors@localhost ~]$ cat -n ~/.bash_profile

 1 # .bash_profile

 2

 3 # Get the aliases and functions

 4 if [ -f ~/.bashrc ]; then

 5 . ~/.bashrc

 6 fi

 7

 8 # User specific environment and startup programs

 9

10 PATH=$PATH:$HOME/.local/bin:$HOME/bin

11

12 export PATH

[mirrors@localhost ~]$ cp ~/.bash_profile{,.original}

[mirrors@localhost ~]$ echo "TUNAPATH=/home/mirrors/tunasync" >> ~/.bash_profile

[mirrors@localhost ~]$ echo "PATH=$PATH:$TUNAPATH" >> ~/.bash_profile

[mirrors@localhost ~]$ cat -n ~/.bash_profile

 1 # .bash_profile

 2

 3 # Get the aliases and functions

 4 if [ -f ~/.bashrc ]; then

 5 . ~/.bashrc

 6 fi

 7

 8 # User specific environment and startup programs

 9

10 PATH=$PATH:$HOME/.local/bin:$HOME/bin

11

12 export PATH

13 TUNAPATH=/home/mirrors/tunasync

14 PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/mirrors/.local/bin:/home/mirrors/bin:

[mirrors@localhost ~]$ source ~/.bash_profile

[mirrors@localhost ~]$

配置 tunasync 的同步服务
[mirrors@localhost ~]$ ls -F /opt/mirrors/tunasync/

conf/ db/ tunasync tunasynctl

[mirrors@localhost ~]$

[mirrors@localhost ~]$ tee > /opt/mirrors/tunasync/conf/manager.conf << EOF

manager.conf

debug = false

[server]

addr = "127.0.0.1"

port = 14242

ssl_cert = ""

ssl_key = ""

[files]

db_type = "bolt"

db_file = "/opt/mirrors/tunasync/db/manager.db"

ca_cert = ""

EOF

按需创建被同步的镜像配置文件,此处以 CentOS 镜像为例:

[mirrors@localhost ~]$ tee > /opt/mirrors/tunasync/conf/worker-centos.conf << EOF

[global]

name = "centos_worker"

log_dir = "/opt/mirrors/tunasync/log/{{.Name}}"

mirror_dir = "/opt/mirrors/mirrorsData"

concurrent = 10

interval = 1440

[manager]

api_base = "http://localhost:14242"

token = "some_token"

ca_cert = ""

[cgroup]

enable = false

base_path = "/sys/fs/cgroup"

group = "tunasync"

[server]

hostname = "localhost"

listen_addr = "127.0.0.1"

listen_port = 16010

ssl_cert = ""

ssl_key = ""

[[mirrors]]

name = "centos"

provider = "rsync"

upstream = "rsync://mirrors.aliyun.com/centos/"

use_ipv6 = false

EOF

[mirrors@localhost ~]$

mirrors@localhost ~]$ ls -F /opt/mirrors/tunasync/

conf/ db/ log/ tunasync tunasynctl

[mirrors@localhost ~]$ ls -F /opt/mirrors/tunasync/conf/

manager.conf worker-centos.conf

[mirrors@localhost ~]$

启动 tunasync同步程序
启动 tunasync 需先启动 manager,后启动 worker。为了便于监控系统进程情况,建立 /opt/mirrors/tunasync/log/plog/ 目录,所有进程的工作日志在该目录中。

用 mirrors 用户执行:

[mirrors@localhost ~]$ /opt/mirrors/tunasync/tunasync manager --config /opt/mirrors/tunasync/conf/manager.conf >> /opt/mirrors/tunasync/log/plog/manager.log &

[1] 60268

[mirrors@localhost ~]$ /opt/mirrors/tunasync/tunasync worker --config /opt/mirrors/tunasync/conf/worker-centos.conf >> /opt/mirrors/tunasync/log/plog/worker-centos.log &

[2] 60284

[mirrors@localhost ~]$

为内网用户开放 web 访问服务
安装 apache 服务器:

[mirrors@localhost ~]$ su root

Password:

[root@localhost mirrors]# yum -y install httpd

Loaded plugins: fastestmirror, langpacks

Loading mirror speeds from cached hostfile

Installed:

httpd.x86_64 0:2.4.6-93.el7.centos

Dependency Installed:

httpd-tools.x86_64 0:2.4.6-93.el7.centos mailcap.noarch 0:2.1.41-2.el7

Complete!

[root@localhost mirrors]#

[root@localhost mirrors]# tree /etc/httpd/

/etc/httpd/

├── conf

│ ├── httpd.conf

│ └── magic

├── conf.d

│ ├── autoindex.conf

│ ├── README

│ ├── userdir.conf

│ └── welcome.conf

├── conf.modules.d

│ ├── 00-base.conf

│ ├── 00-dav.conf

│ ├── 00-lua.conf

│ ├── 00-mpm.conf

│ ├── 00-proxy.conf

│ ├── 00-systemd.conf

│ └── 01-cgi.conf

├── logs -> ../../var/log/httpd

├── modules -> ../../usr/lib64/httpd/modules

└── run -> /run/httpd

6 directories, 13 files

[root@localhost mirrors]# cp /etc/httpd/conf/httpd.conf{,.original}

[root@localhost mirrors]#

[mirrors@localhost ~]$ grep "^\s[^# \t].$" /etc/httpd/conf/httpd.conf | cat

ServerRoot "/etc/httpd"

Listen 80

Include conf.modules.d/*.conf

User apache

Group apache

ServerAdmin root@localhost

<Directory />

AllowOverride none

Require all denied

</Directory>

DocumentRoot "/var/www/html"

<Directory "/var/www">

AllowOverride None

Require all granted

</Directory>

<Directory "/var/www/html">

Options Indexes FollowSymLinks

AllowOverride None

Require all granted

</Directory>

<IfModule dir_module>

DirectoryIndex index.html

</IfModule>

<Files ".ht*">

Require all denied

</Files>

ErrorLog "logs/error_log"

LogLevel warn

<IfModule log_config_module>

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

LogFormat "%h %l %u %t \"%r\" %>s %b" common

<IfModule logio_module>

  LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

</IfModule>

CustomLog "logs/access_log" combined

</IfModule>

<IfModule alias_module>

ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

</IfModule>

<Directory "/var/www/cgi-bin">

AllowOverride None

Options None

Require all granted

</Directory>

<IfModule mime_module>

TypesConfig /etc/mime.types

AddType application/x-compress .Z

AddType application/x-gzip .gz .tgz

AddType text/html .shtml

AddOutputFilter INCLUDES .shtml

</IfModule>

AddDefaultCharset UTF-8

<IfModule mime_magic_module>

MIMEMagicFile conf/magic

</IfModule>

EnableSendfile on

IncludeOptional conf.d/*.conf

[mirrors@localhost ~]$

修改/etc/httpd/conf/httpd.conf后为:

[mirrors@localhost ~]$ grep "^\s[^# \t].$" /etc/httpd/conf/httpd.conf | cat

ServerRoot "/etc/httpd"

Listen 80

Include conf.modules.d/*.conf

User apache

Group apache

ServerAdmin root@localhost

<Directory />

AllowOverride none

Require all denied

</Directory>

DocumentRoot "/opt/mirrors/mirrorsData/"

<Directory "/var/www">

AllowOverride None

Require all granted

</Directory>

<Directory "/opt/mirrors/mirrorsData/">

Options Indexes FollowSymLinks

AllowOverride None

Require all granted

</Directory>

<IfModule dir_module>

DirectoryIndex index.html

</IfModule>

<Files ".ht*">

Require all denied

</Files>

ErrorLog "logs/error_log"

LogLevel warn

<IfModule log_config_module>

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

LogFormat "%h %l %u %t \"%r\" %>s %b" common

<IfModule logio_module>

  LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

</IfModule>

CustomLog "logs/access_log" combined

</IfModule>

<IfModule alias_module>

ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

</IfModule>

<Directory "/var/www/cgi-bin">

AllowOverride None

Options None

Require all granted

</Directory>

<IfModule mime_module>

TypesConfig /etc/mime.types

AddType application/x-compress .Z

AddType application/x-gzip .gz .tgz

AddType text/html .shtml

AddOutputFilter INCLUDES .shtml

</IfModule>

AddDefaultCharset UTF-8

<IfModule mime_magic_module>

MIMEMagicFile conf/magic

</IfModule>

EnableSendfile on

IncludeOptional conf.d/*.conf

[mirrors@localhost ~]$

[mirrors@localhost ~]$ chmod -R 755 /opt/mirrors/mirrorsData

[mirrors@localhost ~]$ systemctl start httpd.service

[mirrors@localhost ~]$ systemctl enable httpd.service

Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.

[mirrors@localhost ~]$

[root@localhost Public]# chown -R mirrors:mirrorgroup /opt/mirrors/mirrorsData/index.html

[mirrors@localhost ~]$ cat /opt/mirrors/mirrorsData/index.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">;

<html xmlns="http://www.w3.org/1999/xhtml">;

<head>

<meta http-equiv="content-type" content="text/html; charset=utf-8"/>

<link rel="stylesheet" type="text/css" href="/.resource/css/mirror.css" media="screen" />

<script type="text/javascript" src=".resource/js/jquery-1.11.3.min.js" ></script>

<script type="text/javascript" src=".resource/js/mirror.js" ></script>

<script type="text/javascript" src=".resource/js/main.js" ></script>

<title>CentOS</title>

</head>

<body>

<h1>欢迎访问CentOS镜像站</h1>

<table id="distro-table" cellpadding="0" cellspacing="0">

<colgroup>

    <col width="12%"/>

    <col width="15%"/>

    <col width="35%"/>

    <col width="10%"/>

    <col width="8%"/>

    <col width="20%"/>

</colgroup>

<thead>

    <tr>

        <th>镜像名</th>

        <th>上次更新时间</th>

        <th>更新源</th>

        <th>状态</th>

        <th>镜像大小</th>

        <th>使用帮助</th>

    </tr>

</thead>

<tbody>

    <tr class="odd">

        <td><a href="/centos/">centos/</a></td>

        <td class="centos update-time">N/A</td>

        <td class="centos upstream">N/A</td>

        <td class="centos sync-status">N/A</td>

        <td class="centos mirror-size">N/A</td>

        <td><a href="/.help/centos.html">Centos/Redhat 使用帮助</a></td>

    </tr>

</tbody>

</table>

<div id="footer">

</div>

</body>

</html>

[mirrors@localhost ~]$

[mirrors@localhost ~]$ chmod -R 755 /opt/mirrors/mirrorsData

至此 Mirror 站点搭建完成,可以通过浏览器直接访问 站点 IP 访问 :

技术图片

技术图片

在隔离环境中搭建 Mirror 站点

标签:output   pes   etc   今天   level   fedora   size   fun   false   

原文地址:https://blog.51cto.com/6286393/2536308

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!