15308555518/ tunasync
1
0
Fork 0
mirror of https://github.com/tuna/tunasync.git synced 2025-12-25 22:02:46 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
516 Commits 3 Branches 31 Tags 1.6 MiB
Commit Graph

165 Commits

Author SHA1 Message Date
z4yx 1fb9f85862 closing log files where they were opened 2020-03-28 16:26:40 +08:00
Phy 4d461bd172 Use logger to print some debug messages than fmt.print 2020-03-23 01:20:49 -04:00
zyx b1f2679fbf [cmd provider] add support of match size in logs 2020-03-20 10:30:44 +08:00
zyx aee1a705b7 remove "--contimeout=120" from default rsync options 2020-03-16 22:23:47 +08:00
zyx 9eb72c5db0 fix misuse of variables 2020-03-16 21:59:34 +08:00
z4yx b490c22984 add test of rsyncEnv 2020-03-16 21:16:23 +08:00
z4yx ae5ff25d20 in case rsyncEnv is nil 2020-03-16 21:11:15 +08:00
z4yx 365f49e6d3 add support of env config for rsync provider 2020-03-16 20:59:08 +08:00
z4yx c41d7a4038 Bring docker test back 
commit 4540ba24c72cb2d24e2e04870025dfa233dedf30
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 11:16:13 2020 +0800

    wait longer

commit c8f07b81a7fe5fdef9224e8bc187500c4d67f049
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 10:55:49 2020 +0800

    try to Terminate

commit 10d2d4b9d0756cf8f60fe27e1e41ae29b5ea6cbe
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 10:50:26 2020 +0800

    forward the error

commit 38c96ee44d31088b9e6de67ebb745358fac8d49a
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 10:31:39 2020 +0800

    now enable the assertion

commit 3b3c46a065a035d906d4cc5022d42e30b1f52a08
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 10:26:40 2020 +0800

    rm un-related info

commit dd7ef7e3d0a0765c1fc48296d70966b3b4d581dd
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 10:12:01 2020 +0800

    print err of provider.Run

commit 49a7b57dbf52d410c0dfe796be9c2f6213884931
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 09:55:48 2020 +0800

    wait until it exits

commit a3e8f699072e3252b3300c667f1425a966aedb39
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 09:54:19 2020 +0800

    targeting alpine:3.8

commit f30b8565049bb373a1a91a34ad07c8c3df8e1036
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 09:47:27 2020 +0800

    see what happens

commit 8c21229a8be8e2ac0737bbc4bb88ba54e9fb7a20
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Sat Mar 14 09:30:50 2020 +0800

    remove one assertion

commit 123368e6ef07aa63c489bb49bdf370d3abdd17bb
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Fri Mar 13 23:32:45 2020 +0800

    docker test somehow works now

commit 94fa294a9bbedb569e6dd9cc7e4f27e73ed97443
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Fri Mar 13 23:27:12 2020 +0800

    should use -d

commit b35bae2a9cb5e006c513da95377ab9487fc4341a
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Fri Mar 13 23:22:25 2020 +0800

    docker run not working??

commit 9aea0036f434d333087f0cff3ce5165a53554e5f
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Fri Mar 13 23:12:39 2020 +0800

    test if docker works

commit f92578b159587a8bbda296bbf9261fb4c5e2f186
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Fri Mar 13 17:42:00 2020 +0800

    debugging docker_test

commit b649e32f76549711af597ce3a642309a41a08bf9
Author: z4yx <z4yx@users.noreply.github.com>
Date:   Fri Mar 13 17:27:55 2020 +0800

    Revert "remove docker_test.go"

    This reverts commit a517a4bb64.
 2020-03-14 11:23:19 +08:00
z4yx 8b0ef2bb53 fix the test 2020-03-14 11:11:10 +08:00
z4yx b25be80670 extra options should only be applied to the second stage 2020-03-14 11:01:34 +08:00
z4yx a517a4bb64 remove docker_test.go 2020-03-13 17:08:39 +08:00
z4yx b816803eaf Revert "disable docker_test" 
This reverts commit 6d17d6b4ca.
 2020-03-13 17:08:19 +08:00
z4yx 6d17d6b4ca disable docker_test 2020-03-13 17:04:36 +08:00
z4yx 51e7f1d573 add TestRsyncProviderWithOverriddenOptions 2020-03-13 17:02:38 +08:00
z4yx c99095267e [docker test] Wait for docker running 2020-03-13 16:57:07 +08:00
z4yx 5c140035ec [worker teset] Wait for http server starting 2020-03-13 16:41:30 +08:00
z4yx 80ad3247a0 fix build 2020-03-13 15:16:39 +08:00
z4yx 02468e21c0 add an option "rsync_override" 2020-03-13 15:12:52 +08:00
z4yx 908f098c72 [mirror config] job fails on pattern match in log 2020-03-09 21:48:06 +08:00
Yuxiang Zhang 22cfdfc9c2
[mirror config] extra rsync options (#107) 
* support "rsync_options" array in config

* add test for new options

* fix tests
 2020-03-09 20:48:09 +08:00
z4yx 36010dc33e Merge branch 'master' of github.com:tuna/tunasync 2019-11-30 00:08:59 +08:00
z4yx bc416a6088 add tests for post-hooks 2019-11-30 00:08:46 +08:00
z4yx a065a11b38 change timeout in tests 2019-11-07 12:29:57 +08:00
z4yx b4fe4db82a Merge remote-tracking branch 'origin/dev' 2019-11-04 23:11:34 +08:00
z4yx 839363aaaa reschedule the job if any hook fails 2019-11-04 22:52:03 +08:00
Jeeken Wang 501f77ee41 Merge branch 'master' into feature/btrfs-snapshot 2019-08-15 01:26:28 +08:00
z4yx 9e91fd706e Merge branch 'dev' 2019-08-13 23:10:43 +08:00
z4yx 94cf0b4bdb fix possible null dereferencing, reported by #96 2019-08-13 23:07:01 +08:00
WANG Ziqin 6b56c4254c feat(btrfs_snapshot_hook): reimplemented Btrfs snapshots 
TODO: test coverage
 2019-08-02 13:31:33 +08:00
WANG Ziqin 30259da0f0 fix nil pointer dereference: check err first 2019-08-02 02:15:22 +08:00
WANG Ziqin 4854d9b981 Fix test: initialize dockerHook with embedded provider 2019-07-31 17:29:28 +08:00
WANG Ziqin 06fce98c00 Eliminate duplicate mirrorProvider in Hooks 2019-07-31 16:11:56 +08:00
z4yx a6fc97889d [bug fix] stalled scheduler if post-sync hook runs for a time which is longer than the sync interval 2019-07-05 16:29:00 +08:00
zyx 9f91d90fc5 check Retry configuration in providers 2019-04-13 11:01:56 +08:00
zyx 1aa4ae9cc1 Merge remote-tracking branch 'kinosang/master' into wip-test-pr 2019-04-13 02:07:41 +08:00
zyx d0deeb19a9 extract mirror size from rsync provider automatically 2019-04-13 01:27:35 +08:00
zyx a283328dc4 increase test converage of worker 2019-04-12 09:43:57 +08:00
zyx 1890bbed3c add tests for last commit 2019-04-11 12:36:43 +08:00
zyx ddc9efd155 report next scheduled sync time 2019-04-11 12:36:18 +08:00
zyx 7eb119b892 singleton of worker is not used, so remove it 2019-04-11 10:07:42 +08:00
zyx 90d419ca66 add tests for last commit 2019-03-31 12:16:45 +08:00
zyx 96cb975412 Let user create ZFS dataset manually due to security considerations 2019-03-31 12:09:42 +08:00
王邈 ff3e690497
Revert "change owner of folder to current user after creating zfs dataset (close #89)" 
This reverts commit a58e6d37ae and
re-opens #89.

Signed-off-by: 王邈 <shankerwangmiao@gmail.com>
 2019-03-26 00:30:06 +08:00
zyx a58e6d37ae change owner of folder to current user after creating zfs dataset (close #89) 2019-03-25 23:40:04 +08:00
z4yx c750aa1871 new feature: run "tunasynctl start" with "-f" to override concurrent job limit 2018-05-30 18:59:24 +08:00
Yuxiang Zhang 6cbe91b4f1 new command: jobForceStart 2018-05-30 16:07:07 +08:00
Yuxiang Zhang 89a792986d increase test coverage rate of job & provider 2018-05-30 14:00:10 +08:00
Yuxiang Zhang 0fdb07d061 bug fix: log over-written in twoStageRsyncProvider 
solve more DATA RACE problem
 2018-05-30 12:28:09 +08:00
Yuxiang Zhang c5bb172f99 increase test coverage rate of job.go 2018-05-30 11:45:05 +08:00
Yuxiang Zhang 79e6167028 fix race condition on logFile of baseProvider 2018-05-30 01:46:16 +08:00
bigeagle a475b044c6 feat(worker): add 'use_ipv4' option for rsync provider 2017-09-08 00:15:48 +08:00
bigeagle a50a360a91 Revert "feat(worker): add '-4' option to rsync when 'use_ipv6' is false" 
This reverts commit d536aca2ac.
 2017-09-08 00:12:40 +08:00
bigeagle d536aca2ac feat(worker): add '-4' option to rsync when 'use_ipv6' is false 2017-09-06 23:22:55 +08:00
7IN0SAN9 563860d424
fix #63 2017-03-27 13:09:56 +08:00
bigeagle 939abaef9b
feat(worker): TUNASYNC_LOG_DIR environment variable 2016-12-18 20:41:26 +08:00
bigeagle d5a438462f
feat(worker): map current uid and gid to docker 2016-12-18 14:28:48 +08:00
bigeagle d4e07a7b29
fix(worker): keep the same working dir inside and outside of docker 2016-12-18 14:28:32 +08:00
bigeagle fd277388d5
fix(worker): fixed multi-manager configuration 
the worker must be registerred on the manager

`extra_status_manager` option is replaced by `api_base_list`, which overrides the `api_base` option
 2016-12-12 23:17:50 +08:00
bigeagle 7601e5793f
fix(worker): improved cgroup creation 2016-12-10 04:14:39 +08:00
bigeagle 9645fd44ec ci(travis): Enabled docker on travis 2016-12-10 03:48:03 +08:00
bigeagle ebd462be36
feat(worker): Implemented docker executor, close #55 
if docker is enabled in configure file and `docker_image` is set on mirror config, the command would

be executed via `docker run ...`
 2016-12-10 02:44:45 +08:00
bigeagle 21c832c8fb fix(worker): disabled memory limit 
rsync memory is nolonger limited
 2016-12-09 23:07:05 +08:00
bigeagle a0b8ef08ab feat(worker): implemented `extra_status_manager` option to enable a worker reporting status to multi 2016-12-06 23:59:15 +08:00
bigeagle 86153c59e3
feat(worker): ZFS support: isolate mirrors in zfs datasets 2016-12-05 00:44:55 +08:00
bigeagle 96f9db8bb8
fix(worker): extended rsync memory limit to 512MB 2016-12-04 22:56:48 +08:00
bigeagle add7b936e3 refactor(cmd): pinning cli.v1 2016-11-20 00:01:47 +08:00
bigeagle 2b83ea4b45 feat(worker): added global `exec_on_success` and `exec_on_failure` options 
if `exec_on_success` is set on mirror-level config, it overrides the global option; if on mirror

level, extra hook cmd is needed, use `exec_on_success_extra` option
 2016-10-21 01:20:31 +08:00
bigeagle 437acd3f01
fix(worker): cgroup hook now ensure child progresses are killed 2016-08-02 21:55:41 +08:00
bigeagle c8af09f129
fix(worker): fix log file leakage of two-stage-sync provider. closing #36 2016-08-02 21:39:16 +08:00
bigeagle d292f9b0e3
feat(tunasync): add `tunasynctl reload` subcommand 2016-06-15 14:01:10 +08:00
bigeagle 613d46e4e3
fix(worker): fix resource leakage caused by time.Tick(), closing #33 2016-06-15 00:31:53 +08:00
bigeagle ebeee6bc34
feat(worker): added username option for rsync providers 2016-06-14 21:37:03 +08:00
bigeagle f6d53c16d6 fix(worker): do not init cgroup if it's not enabled 2016-05-19 11:42:22 +08:00
bigeagle 28c8145137
feature(worker): limit rsync memory using cgroup 2016-05-08 17:24:41 +08:00
bigeagle ccc31d9289
refactor(worker): added Type() method to provider 2016-05-08 16:26:58 +08:00
bigeagle 5c8d90608c
feature(worker): implemented multi-file configrations. closing #23 2016-05-03 16:52:38 +08:00
bigeagle 5b5f020075
style(worker): put exported methods together 2016-05-02 22:19:02 +08:00
bigeagle 76ad3d40d6
feature(worker): when SIGINT/SIGTERM received, stop all the jobs and update their status before quit. Close #19. 2016-05-02 22:16:49 +08:00
bigeagle 65984053eb
fix(worker): fixed scheduling bugs 2016-05-02 18:56:45 +08:00
bigeagle 51fa12900d
feature(worker): ability to hot reload mirror job configrations, close #18 2016-05-02 17:45:21 +08:00
bigeagle fecfc8f3b1
fix(worker): log file link should use relative path 2016-05-02 13:38:25 +08:00
bigeagle 56459f2ce0
feature(worker): implemented mirror role (master/slave) option 2016-04-30 16:44:39 +08:00
bigeagle 8399417880
feature(worker): fix exec_post_hook_test, manager channel was blocked in the previous version 2016-04-30 16:44:35 +08:00
bigeagle bd423eec4e
feature(worker): added exec_on_success and exec_on_failure option and hooks 2016-04-30 16:44:15 +08:00
bigeagle f8fd1ae460
style: better logging 2016-04-30 16:43:47 +08:00
bigeagle 924fda6dd8
feature(worker): use cgroup track job process, so that they can be all-killed 2016-04-30 16:43:30 +08:00
bigeagle 41e1f263a5
refactor(worker): use atomic state to simplify job control 2016-04-30 16:43:24 +08:00
bigeagle 2268eb3b0f
fix(tunasync): connection leakage caused by http keep-alive 2016-04-30 16:43:17 +08:00
bigeagle 292a24ba20
fix(worker): fixed job status and control logic 2016-04-30 16:42:51 +08:00
bigeagle 9fbb8ab155
refactor(tunasync): 
1. refactored manager and worker to support TLS transport
2. if mirror_dir is specified from a mirror config, don't add the mirror name
 2016-04-30 16:42:15 +08:00
bigeagle f8151e689f
refactor(worker): export worker's LoadConfig 2016-04-30 16:41:51 +08:00
bigeagle 0dcd89da31
refactor(manager): refactored structure names in manager 2016-04-30 16:41:44 +08:00
bigeagle ce3471e30d
feature(worker): implemented Worker object, worker side code is almost done 2016-04-30 16:41:39 +08:00
bigeagle 6062aa4b9d
refactor(worker): rename provider.setLogFile to provider.prepareLogFile 2016-04-30 16:40:45 +08:00
bigeagle 9afd47ddcb
feature(worker): LogLimiter hook 2016-04-30 16:40:40 +08:00
bigeagle 23c3125cbf
tests(worker): added test for initProfile 2016-04-30 16:40:34 +08:00
bigeagle 8b56fda1e9
feature(worker): added worker http server config 2016-04-30 16:40:27 +08:00
bigeagle 731fba842f
feature(worker): job need to be started by jobStart signal 2016-04-30 16:40:16 +08:00
bigeagle 13161d77cf
feature(worker): two-stage-rsync provider 2016-04-30 16:40:12 +08:00