From 0ec7a552f2b9e2f9881d5b6fcc712d743edf11fd Mon Sep 17 00:00:00 2001 From: Xiangyue Cai Date: Tue, 20 Jul 2021 14:55:07 +0800 Subject: [PATCH 1/3] add use_block_cache and seaf-gc --rm-fs (#37) * add use_block_cache and seaf-gc --rm-fs * improve doc * improve doc --- manual/config/seafile-conf.md | 10 ++++++++++ manual/maintain/seafile_gc.md | 9 +++++++++ 2 files changed, 19 insertions(+) diff --git a/manual/config/seafile-conf.md b/manual/config/seafile-conf.md index 1004252c..bbcf5609 100644 --- a/manual/config/seafile-conf.md +++ b/manual/config/seafile-conf.md @@ -142,6 +142,16 @@ max_sync_file_count = 100000 fs_id_list_request_timeout = 300 ``` +New in Seafile Pro 8.0.5: If you use cloud storage backend, you can enable the `use_block_cache` configuration to speed up file access. `use_block_cache` the default value is false. Note that this configuration is only effective for downloading files through web page or API, not for synchronized files. The `block_cache_size_limit` configuration is used to limit the size of the cache. `block_cache_size_limit` the default value is 10GB. the `block_cache_file_types` configuration is used to limit the file types that are cached. `block_cache_file_types` the default value is mp4;mov. + +``` +[fileserver] +use_block_cache = true +# Set block cache size limit to 100MB +block_cache_size_limit = 100 +block_cache_file_types = mp4;mov +``` + ## Database configuration The whole database configuration is stored in the `[database]` section of the configuration file, whether you use SQLite, MySQL or PostgreSQL. diff --git a/manual/maintain/seafile_gc.md b/manual/maintain/seafile_gc.md index 4613a44d..822b6cb7 100644 --- a/manual/maintain/seafile_gc.md +++ b/manual/maintain/seafile_gc.md @@ -92,6 +92,15 @@ seaf-gc.sh -r **In Seafile version 4.1.1 and later, libraries deleted by the users are not immediately removed from the system. Instead, they're moved into a "trash" in the system admin page. Before they're cleared from the trash, their blocks won't be garbage collected.** +### Removing FS objects + +Since Pro server 8.0.6, you can remove garbage fs objects. It should be run without the --dry-run option: + +``` +seaf-gc.sh --rm-fs + +``` + ### Using Multiple Threads in GC Since Pro server 5.1.0, you can specify the thread number in GC. By default, From c587f0b0a4e5351035e0cb833641a0860c7bde35 Mon Sep 17 00:00:00 2001 From: Jiaqiang Xu Date: Wed, 21 Jul 2021 10:29:24 +0800 Subject: [PATCH 2/3] Update seafile-conf.md Improve description for block cache options. --- manual/config/seafile-conf.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/manual/config/seafile-conf.md b/manual/config/seafile-conf.md index bbcf5609..df93b2e7 100644 --- a/manual/config/seafile-conf.md +++ b/manual/config/seafile-conf.md @@ -142,7 +142,11 @@ max_sync_file_count = 100000 fs_id_list_request_timeout = 300 ``` -New in Seafile Pro 8.0.5: If you use cloud storage backend, you can enable the `use_block_cache` configuration to speed up file access. `use_block_cache` the default value is false. Note that this configuration is only effective for downloading files through web page or API, not for synchronized files. The `block_cache_size_limit` configuration is used to limit the size of the cache. `block_cache_size_limit` the default value is 10GB. the `block_cache_file_types` configuration is used to limit the file types that are cached. `block_cache_file_types` the default value is mp4;mov. +If you use object storage as storage backend, when a large file is frequently downloaded, the same blocks need to be fetched from the storage backend to Seafile server. This may waste bandwith and cause high load on the internal network. Since Seafile Pro 8.0.5 version, we add block caching to improve the situation. Note that this configuration is only effective for downloading files through web page or API, but not for syncing files. + +* To enable this feature, set `use_block_cache` option in the `[fileserver]` group. It's not enabled by default. +* The `block_cache_size_limit` option is used to limit the size of the cache. Its default value is 10GB. The blocks are cached in `seafile-data/block-cache` directory. When the total size of cached files exceeds the limit, seaf-server will clean up older files until the size reduces to 70% of the limit. The cleanup interval is 5 minutes. You have to have a good estimate on how much space you need for the cache directory. Otherwise on frequent downloads this directory can be quickly filled up. +* The `block_cache_file_types` configuration is used to choose the file types that are cached. `block_cache_file_types` the default value is mp4;mov. ``` [fileserver] From e903aaf3286f78f14d3b09676fb10f3c4de09197 Mon Sep 17 00:00:00 2001 From: Xiangyue Cai Date: Wed, 21 Jul 2021 16:42:01 +0800 Subject: [PATCH 3/3] add migrate to local file system for single storage (#40) * add migrate to local file system for single storage * improve doc --- manual/deploy_pro/migrate.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/manual/deploy_pro/migrate.md b/manual/deploy_pro/migrate.md index b1dacf1c..b7079f92 100644 --- a/manual/deploy_pro/migrate.md +++ b/manual/deploy_pro/migrate.md @@ -39,6 +39,31 @@ mv seafile.conf /opt ``` +If you want to migrate to a local file system, the seafile.conf temporary configuration example is as follows: + +``` +cat > seafile.conf << EOF +[commit_object_backend] +name = fs +# the dir configuration is the new seafile-data path +dir = /var/data_backup + +[fs_object_backend] +name = fs +# the dir configuration is the new seafile-data path +dir = /var/data_backup + +[block_backend] +name = fs +# the dir configuration is the new seafile-data path +dir = /var/data_backup + +EOF + +mv seafile.conf /opt + +``` + Repalce the configurations with your own choice. ## Migrating large number of objects