lsyncd实时同步文件至远程服务器

Lysncd是lua语言封装了 inotify 和 rsync 工具,采用了 linux 内核(2.6.13 及以后)里的 inotify 触发机制,通过inotify或者fsevents监控本地目录的变化事件,然后通过rsync同步。Lsyncd最强大之处在于简单高效传输海量数据并且Lsyncd支持多种工作模式。

安装rsync

[root@localhost ~]# yum install -y rsync
[root@localhost ~]# rsync --version
rsync  version 3.1.2  protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

安装lsyncd所需依赖

[root@localhost ~]# yum install -y gcc gcc-c++ lua lua-devel cmake libxml2 libxml2-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: download.nus.edu.sg
 * elrepo: mirrors.tuna.tsinghua.edu.cn
 * epel: mirrors.tuna.tsinghua.edu.cn
 * extras: download.nus.edu.sg
 * updates: download.nus.edu.sg
Package gcc-4.8.5-44.el7.x86_64 already installed and latest version
Package lua-5.1.4-15.el7.x86_64 already installed and latest version
Package libxml2-2.9.1-6.el7_9.6.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package cmake.x86_64 0:2.8.12.2-2.el7 will be installed
--> Processing Dependency: libarchive.so.13()(64bit) for package: cmake-2.8.12.2-2.el7.x86_64
---> Package gcc-c++.x86_64 0:4.8.5-44.el7 will be installed
--> Processing Dependency: libstdc++-devel = 4.8.5-44.el7 for package: gcc-c++-4.8.5-44.el7.x86_64
---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_9.6 will be installed
--> Processing Dependency: zlib-devel for package: libxml2-devel-2.9.1-6.el7_9.6.x86_64
--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_9.6.x86_64
---> Package lua-devel.x86_64 0:5.1.4-15.el7 will be installed
--> Running transaction check
---> Package libarchive.x86_64 0:3.1.2-14.el7_7 will be installed
---> Package libstdc++-devel.x86_64 0:4.8.5-44.el7 will be installed
---> Package xz-devel.x86_64 0:5.2.2-2.el7_9 will be installed
---> Package zlib-devel.x86_64 0:1.2.7-21.el7_9 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================================================================================
 Package                                                     Arch                                               Version                                                        Repository                                           Size
=========================================================================================================================================================================================================================================
Installing:
 cmake                                                       x86_64                                             2.8.12.2-2.el7                                                 base                                                7.1 M
 gcc-c++                                                     x86_64                                             4.8.5-44.el7                                                   base                                                7.2 M
 libxml2-devel                                               x86_64                                             2.9.1-6.el7_9.6                                                updates                                             1.1 M
 lua-devel                                                   x86_64                                             5.1.4-15.el7                                                   base                                                 21 k
Installing for dependencies:
 libarchive                                                  x86_64                                             3.1.2-14.el7_7                                                 base                                                319 k
 libstdc++-devel                                             x86_64                                             4.8.5-44.el7                                                   base                                                1.5 M
 xz-devel                                                    x86_64                                             5.2.2-2.el7_9                                                  updates                                              46 k
 zlib-devel                                                  x86_64                                             1.2.7-21.el7_9                                                 updates                                              50 k

Transaction Summary
=========================================================================================================================================================================================================================================
Install  4 Packages (+4 Dependent packages)

Total download size: 17 M
Installed size: 61 M
Downloading packages:
(1/8): libarchive-3.1.2-14.el7_7.x86_64.rpm                                                                                                                                                                       | 319 kB  00:00:01     
(2/8): lua-devel-5.1.4-15.el7.x86_64.rpm                                                                                                                                                                          |  21 kB  00:00:00     
(3/8): xz-devel-5.2.2-2.el7_9.x86_64.rpm                                                                                                                                                                          |  46 kB  00:00:00     
(4/8): zlib-devel-1.2.7-21.el7_9.x86_64.rpm                                                                                                                                                                       |  50 kB  00:00:00     
(5/8): libxml2-devel-2.9.1-6.el7_9.6.x86_64.rpm                                                                                                                                                                   | 1.1 MB  00:00:03     
(6/8): libstdc++-devel-4.8.5-44.el7.x86_64.rpm                                                                                                                                                                    | 1.5 MB  00:00:03     
(7/8): gcc-c++-4.8.5-44.el7.x86_64.rpm                                                                                                                                                                            | 7.2 MB  00:00:15     
(8/8): cmake-2.8.12.2-2.el7.x86_64.rpm                                                                                                                                                                            | 7.1 MB  00:00:21     
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                    807 kB/s |  17 MB  00:00:21     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : zlib-devel-1.2.7-21.el7_9.x86_64                                                                                                                                                                                      1/8 
  Installing : xz-devel-5.2.2-2.el7_9.x86_64                                                                                                                                                                                         2/8 
  Installing : libstdc++-devel-4.8.5-44.el7.x86_64                                                                                                                                                                                   3/8 
  Installing : libarchive-3.1.2-14.el7_7.x86_64                                                                                                                                                                                      4/8 
  Installing : cmake-2.8.12.2-2.el7.x86_64                                                                                                                                                                                           5/8 
  Installing : gcc-c++-4.8.5-44.el7.x86_64                                                                                                                                                                                           6/8 
  Installing : libxml2-devel-2.9.1-6.el7_9.6.x86_64                                                                                                                                                                                  7/8 
  Installing : lua-devel-5.1.4-15.el7.x86_64                                                                                                                                                                                         8/8 
  Verifying  : gcc-c++-4.8.5-44.el7.x86_64                                                                                                                                                                                           1/8 
  Verifying  : lua-devel-5.1.4-15.el7.x86_64                                                                                                                                                                                         2/8 
  Verifying  : libarchive-3.1.2-14.el7_7.x86_64                                                                                                                                                                                      3/8 
  Verifying  : libxml2-devel-2.9.1-6.el7_9.6.x86_64                                                                                                                                                                                  4/8 
  Verifying  : cmake-2.8.12.2-2.el7.x86_64                                                                                                                                                                                           5/8 
  Verifying  : libstdc++-devel-4.8.5-44.el7.x86_64                                                                                                                                                                                   6/8 
  Verifying  : xz-devel-5.2.2-2.el7_9.x86_64                                                                                                                                                                                         7/8 
  Verifying  : zlib-devel-1.2.7-21.el7_9.x86_64                                                                                                                                                                                      8/8 

Installed:
  cmake.x86_64 0:2.8.12.2-2.el7                          gcc-c++.x86_64 0:4.8.5-44.el7                          libxml2-devel.x86_64 0:2.9.1-6.el7_9.6                          lua-devel.x86_64 0:5.1.4-15.el7                         

Dependency Installed:
  libarchive.x86_64 0:3.1.2-14.el7_7                        libstdc++-devel.x86_64 0:4.8.5-44.el7                        xz-devel.x86_64 0:5.2.2-2.el7_9                        zlib-devel.x86_64 0:1.2.7-21.el7_9                       

Complete!

源码编译安装lsyncd

[root@localhost ~]# wget https://github.com/lsyncd/lsyncd/archive/refs/tags/v2.3.1.tar.gz
[root@localhost ~]# tar xzvf v2.3.1.tar.gz
[root@localhost lsyncd-2.3.1]# cd lsyncd-2.3.1
[root@localhost lsyncd-2.3.1]# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/lsyncd
[root@localhost lsyncd-2.3.1]# make && make install
[root@localhost lsyncd-2.3.1]# ln -s /usr/local/lsyncd/bin/lsyncd /usr/bin/lsyncd

出现-- Configuring incomplete, errors occurred!安装lua-devel即可,如有提示cmake版本问题请安装3.5以上版本cmake

配置lsyncd.conf文件

[root@localhost ~]# vi /etc/lsyncd.conf
  • ssh远程同步模式
settings {
     logfile = "/var/log/lsyncd.log",
     statusFile = "/var/log/lsyncd.status",
     pidfile = "/var/run/lsyncd.pid",
     statusInterval = 1,
     nodaemon = false,
     maxProcesses = 1,
     maxDelays = 1,
}
sync {
     default.rsyncssh,
     source = "/opt/",
     delete = true,
     host = "192.168.1.2",
     targetdir = "/bak",
     rsync = {
     binary = "/usr/bin/rsync",
     archive = true,
     compress = false,
     owner = true,
     perms = true,
     whole_file = false
     },
     ssh = {
     port = 22
     }
}
  • 本地同步
settings {
     logfile = "/var/log/lsyncd.log",
     statusFile = "/var/log/lsyncd.status",
     pidfile = "/var/run/lsyncd.pid",
     statusInterval = 1,
     nodaemon = false,
     maxProcesses = 1,
     maxDelays = 1,
}
sync {
    default.rsync,
    source = "/opt",
    target = "/bak",
}
  • 远程同步rsync-daemon模式
settings {
     logfile = "/var/log/lsyncd.log",
     statusFile = "/var/log/lsyncd.status",
     pidfile = "/var/run/lsyncd.pid",
     statusInterval = 1,
     nodaemon = false,
     maxProcesses = 1,
     maxDelays = 1,
}
sync {
    default.rsync,
    source    = "/opt",
    target    = "renwole@192.168.1.1::bak",
    delete="true",
    exclude = { ".bak*" },
    delay = 30,
    init = false,
    rsync = {
    binary = "/usr/bin/rsync",
    archive = true,
    compress = true,
    verbose   = true,
    perms = true,
    password_file = "/etc/rsync.password",
    _extra    = {"--bwlimit=200"}
    }
}

lsyncd的config参数详解如下:

  • delay: 设置同步源文件更新到目标机器的延迟时间,单位为秒。
  • maxDelays:同步过程中最大可容忍的延迟次数,超过该次数后将自动退出同步程序。
  • init: 指定lsyncd启动时需要执行的脚本命令。
  • logfile:指定日志输出文件路径和名称,默认为stdout。
  • statusFile:指定状态信息输出文件路径和名称,默认为/tmp/lsyncd.status。
  • statusInterval:指定状态信息输出的时间间隔,单位为秒,默认为10秒。
  • nodaemon: 指定以非守护进程模式运行lsyncd。
  • insist:指定如果出现错误,则重试尝试重新同步文件。
  • rsync: 指定rsync程序的可执行文件路径。
  • ssh: 指定ssh程序的可执行文件路径。
  • exclude: 指定需要排除同步的文件或文件夹。可以使用通配符或正则表达式。
  • include:指定需要包含同步的文件或文件夹。可以使用通配符或正则表达式。
  • protectArgs: 指定需要保护的rsync参数,避免被误删除或覆盖。
  • copyin: 指定在同步之前是否需要将目标机器上的文件复制到源机器上。
  • copyout:指定在同步之后是否需要将源机器上的文件复制到目标机器上。
  • copyback: 指定在同步失败后是否需要将目标机器上的文件复制回源机器。
  • delete: 指定是否需要删除目标机器上已经不存在于源机器上的文件。
  • rsyncArgs: 指定传递给rsync程序的参数。
  • sshArgs: 指定传递给ssh程序的参数。
  • timeout: 指定同步过程中命令执行的超时时间,单位为秒。
  • maxProcesses:指定最大同时运行的进程数。
  • watch: 指定需要监视的文件或文件夹,当发现其有变化时会触发同步。可以使用通配符或正则表达式。
  • sync: 指定需要同步的文件或文件夹,可以使用通配符或正则表达式。

添加lsyncd开机自启动

[root@localhost ~]# vi /etc/sysconfig/lsyncd
LSYNCD_OPTIONS="/etc/lsyncd.conf" 
[root@localhost ~]# vi /usr/lib/systemd/system/lsyncd.service
[Unit]
Description=Live Syncing (Mirror) Daemon
After=network.target

[Service]
Type=simple
EnvironmentFile=-/etc/sysconfig/lsyncd
ExecStart=/usr/local/lsyncd/bin/lsyncd -nodaemon $LSYNCD_OPTIONS

[Install]
WantedBy=multi-user.target
[root@localhost ~]# systemctl start lsyncd
[root@localhost ~]# systemctl enable lsyncd
添加新评论