15308555518/ seafile-admin-docs
1
0
Fork 0
mirror of https://github.com/haiwen/seafile-admin-docs.git synced 2025-12-26 02:32:50 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Update configurations for S3 (#529)
Some checks failed
Deploy CI - 13.0 / deploy (push) Has been cancelled
Details

This commit is contained in:
Huang Junxiang 2025-06-03 16:37:58 +08:00 committed by GitHub
parent 5ed38a4837
commit 29609deffc
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
18 changed files with 272 additions and 155 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
manual/config/env.md
View File

@ -71,9 +71,9 @@ This part of configurations is only valid in `CACHE_PROVIDER=memcached`:
- `ENABLE_SEADOC`: Enable the SeaDoc server or not, default is `false`.
- `SEADOC_SERVER_URL`: Only valid in `ENABLE_SEADOC=true`. Url of Seadoc server (e.g., http://seafile.example.com/sdoc-server).
## S3 storage backend configurations
## S3 storage backend configurations (pro)
- `USE_S3_STORAGE`: Use S3 backend storage in Seafile server
- `SEAF_SERVER_STORAGE_TYPE`: What kind of the Seafile data for storage. Available options are `disk` (i.e., local disk), `s3` and `multiple` (see the details of [multiple storage backends](../setup/setup_with_multiple_storage_backends.md))
- `S3_COMMIT_BUCKET`: S3 storage backend fs objects bucket
- `S3_FS_BUCKET`: S3 storage backend block objects bucket
- `S3_BLOCK_BUCKET`: S3 storage backend block objects bucket
@ -89,9 +89,18 @@ This part of configurations is only valid in `CACHE_PROVIDER=memcached`:
- `S3_SSE_C_KEY`: A string of 32 characters can be generated by openssl rand -base64 24. It can be any 32-character long random string. It's required to use V4 authentication protocol and https if you enable SSE-C.
!!! success "Easier to configure S3 for Seafile and its components"
Since Seafile Pro 13.0, in order to facilitate users to deploy Seafile's related extension components and other services in the future, a section will be provided in `.env` to store the **S3 Configurations** for Seafile and some extension components (such as *SeaSearch*, *Metadata server*). You can locate it with the title bar **\#\#S3**.
However, please note that Seafile only support configuring S3 in `.env` for **single storage backend mode** (i.e., when `USE_S3_STORAGE=true`), if you would like to use multiple backends, please set `USE_S3_STORAGE=false` in `.env` and refer [here](../setup/setup_with_multiple_storage_backends.md) for the details.
Since Seafile Pro 13.0, in order to facilitate users to deploy Seafile's related extension components and other services in the future, a section will be provided in `.env` to store the **S3 Configurations** for Seafile and some extension components (such as *SeaSearch*, *Metadata server*). You can locate it with the title bar **Storage configurations for S3**.
!!! warning "S3 configurations in `.env` only support single S3 storage backend mode"
The Seafile server only support configuring S3 in `.env` for **single S3 storage backend mode** (i.e., when `SEAF_SERVER_STORAGE_TYPE=s3`). If you would like to use other storage backend (e.g., [Ceph](./setup_with_ceph.md), [Swift](./setup_with_swift.md)) or other settings that can only be set in `seafile.conf` (like [multiple storage backends](./setup_with_multiple_storage_backends.md)), please set `SEAF_SERVER_STORAGE_TYPE` to `multiple`, and set `MD_STORAGE_TYPE` and `SS_STORAGE_TYPE` according to your configurations.
!!! note "The S3 configurations only valid with at least one `STORAGE_TYPE` has specified to `s3`"
Now there are three (pro) and one (cluster) ***STORAGE_TYPE*** we provided in `.env`:
- SEAF_SERVER_STORAGE_TYPE (pro & cluster)
- MD_STORAGE_TYPE (pro, see the [Metadata server](#metadata-server) section for the details)
- SS_STORAGE_TYPE (pro, see the [SeaSearch](#seasearch) section for the details)
You have to specify at least one of them as s3 for the above configuration to take effect.
## SeaSearch

33
manual/extension/metadata-server.md
View File

@ -54,29 +54,25 @@ MD_MAX_CACHE_SIZE=1GB
#### Example `.env` for Seafile data is stored in the storage backend (e.g., S3)
First you need to create a bucket for Metadata on your S3 storage backend provider. Then add or modify the following information to `.env`:
!!! success "Easier to configure S3 for Seafile and its components"
Since Seafile Pro 13.0, in order to facilitate users to deploy Seafile's related extension components and other services in the future, a section will be provided in `.env` to store the **S3 Configurations** for Seafile and some extension components (such as *SeaSearch*, *Metadata server*). You can locate it with the title bar **\#\#S3**.
In other words, if you deploy SeaSearch and Seafile together, and if you have deployed Seafile Pro following [here](../setup/setup_pro_by_docker.md#downloading-and-modifying-env) (and using the latest `.env`), you only need to specify the following variables in `.env` to make it work (that is, the `USE_S3_STORAGE` is set to `true`):
```sh
S3_COMMIT_BUCKET=<your s3 bucket name for Seafile commit>
S3_FS_BUCKET=<your s3 bucket name for Seafile file storage>
S3_BLOCK_BUCKET=<your s3 bucket name for Seafile block>
S3_MD_BUCKET=<your s3 bucket name for Metadata>
```
First you need to create a bucket for metadata on your S3 storage backend provider. Then add or modify the following information to `.env`:
```sh
MD_IMAGE=seafileltd/seafile-md-server:latest
USE_S3_STORAGE=true
S3_COMMIT_BUCKET=...
S3_FS_BUCKET=...
S3_BLOCK_BUCKET=...
MD_STORAGE_TYPE=s3
S3_MD_BUCKET=...
S3_KEY_ID=<your-key-id>
S3_SECRET_KEY=<your-secret-key>
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=false
S3_AWS_REGION=us-east-1
S3_HOST=
S3_USE_HTTPS=true
S3_SSE_C_KEY=
```
!!! note "Data for Seafile server should be accessible for Metadata server"
In order to correctly obtain metadata information, you must ensure that the data of your Seafile server can be correctly accessed. In the case of deploying Metadata server and Seafile server together, Metadata server will be able to automatically obtain the configuration information of Seafile server, so you don't need to worry about this. But if your Metadata server is deployed in Standalone (usually in a cluster environment), then you need to ensure that the description of the Seafile server storage part in the `.env` deployed by Metadata server needs to be consistent with the `.env` deployed by Seafile server (e.g., `SEAF_SERVER_STORAGE_TYPE`), and can access the configuration file information of Seafile server (e.g., `seafile.conf`) to ensure that Metadata server can correctly obtain data from Seafile server.
#### List of environment variables for Metadata server
The following table is all the related environment variables with Metadata server:
@ -87,7 +83,8 @@ The following table is all the related environment variables with Metadata serve
| `REDIS_HOST` | Your *Redis* service host. | Optional, default `redis` |
| `REDIS_PORT` | Your *Redis* service port. | Optional, default `6379` |
| `REDIS_PASSWORD` | Your *Redis* access password. | Optional |
| `S3_MD_BUCKET` | Your S3 bucket name for the bucket storing metadata | Required when using S3 (`USE_S3_STORAGE` = `true`) |
| `MD_STORAGE_TYPE` | Where the metadata storage in. Available options are `disk` (local storage) and `s3` | `disk` |
| `S3_MD_BUCKET` | Your S3 bucket name for the bucket storing metadata | Required when using S3 (`MD_STORAGE_TYPE=s3`) |
In addition, there are some environment variables **related to S3 authorization**, please refer to the part with `S3_` prefix in this [table](../setup/setup_pro_by_docker.md#downloading-and-modifying-env) (**the buckets name for Seafile are also needed**).

2
manual/repo/docker/cluster/env
View File

@ -32,7 +32,7 @@ MEMCACHED_HOST=<your memcached host>
MEMCACHED_PORT=11211
## S3
USE_S3_STORAGE=false
SEAF_SERVER_STORAGE_TYPE=disk
S3_KEY_ID=<your-key-id>
S3_SECRET_KEY=<your-secret-key>
S3_USE_V4_SIGNATURE=true

2
manual/repo/docker/cluster/seafile-server.yml
View File

@ -28,7 +28,7 @@ services:
- TIME_ZONE=${TIME_ZONE:-UTC}
- INIT_SEAFILE_ADMIN_EMAIL=${INIT_SEAFILE_ADMIN_EMAIL:-me@example.com}
- INIT_SEAFILE_ADMIN_PASSWORD=${INIT_SEAFILE_ADMIN_PASSWORD:-asecret}
- USE_S3_STORAGE=${USE_S3_STORAGE:-false}
- SEAF_SERVER_STORAGE_TYPE=${SEAF_SERVER_STORAGE_TYPE:-disk}
- S3_COMMIT_BUCKET=${S3_COMMIT_BUCKET:-}
- S3_FS_BUCKET=${S3_FS_BUCKET:-}
- S3_BLOCK_BUCKET=${S3_BLOCK_BUCKET:-}

3
manual/repo/docker/md-server.yml
View File

@ -17,7 +17,8 @@ services:
- MD_PORT=${MD_PORT:-8084}
- MD_LOG_LEVEL=${MD_LOG_LEVEL:-info}
- MD_MAX_CACHE_SIZE=${MD_MAX_CACHE_SIZE:-1GB}
- USE_S3_STORAGE=${USE_S3_STORAGE:-false}
- SEAF_SERVER_STORAGE_TYPE=${SEAF_SERVER_STORAGE_TYPE:-disk}
- MD_STORAGE_TYPE=${MD_STORAGE_TYPE:-disk}
- S3_COMMIT_BUCKET=${S3_COMMIT_BUCKET:-}
- S3_FS_BUCKET=${S3_FS_BUCKET:-}
- S3_BLOCK_BUCKET=${S3_BLOCK_BUCKET:-}

3
manual/repo/docker/metadata-server/env
View File

@ -15,7 +15,8 @@ SEAFILE_MYSQL_DB_SEAFILE_DB_NAME=seafile_db
SEAFILE_LOG_TO_STDOUT=false
# for s3 configurations
USE_S3_STORAGE=false
SEAF_SERVER_STORAGE_TYPE=disk
MD_STORAGE_TYPE=$SEAF_SERVER_STORAGE_TYPE
S3_COMMIT_BUCKET=
S3_FS_BUCKET=
S3_BLOCK_BUCKET=

3
manual/repo/docker/metadata-server/md-server.yml
View File

@ -17,7 +17,8 @@ services:
- MD_PORT=${MD_PORT:-8084}
- MD_LOG_LEVEL=${MD_LOG_LEVEL:-info}
- MD_MAX_CACHE_SIZE=${MD_MAX_CACHE_SIZE:-1GB}
- USE_S3_STORAGE=${USE_S3_STORAGE:-false}
- SEAF_SERVER_STORAGE_TYPE=${SEAF_SERVER_STORAGE_TYPE:-disk}
- MD_STORAGE_TYPE=${MD_STORAGE_TYPE:-disk}
- S3_COMMIT_BUCKET=${S3_COMMIT_BUCKET:-}
- S3_FS_BUCKET=${S3_FS_BUCKET:-}
- S3_BLOCK_BUCKET=${S3_BLOCK_BUCKET:-}

40
manual/repo/docker/pro/env
View File

@ -16,7 +16,14 @@ NOTIFICATION_SERVER_IMAGE=seafileltd/notification-server:12.0-latest
SEASEARCH_IMAGE=seafileltd/seasearch:1.0-latest
MD_IMAGE=seafileltd/seafile-md-server:latest
## Persistent Storage
## Storage
### Storage type
SEAF_SERVER_STORAGE_TYPE=disk # disk, s3, multiple
MD_STORAGE_TYPE=$SEAF_SERVER_STORAGE_TYPE # disk, s3
SS_STORAGE_TYPE=$SEAF_SERVER_STORAGE_TYPE # disk, s3
### Local storage path
BASIC_STORAGE_PATH=/opt
SEAFILE_VOLUME=$BASIC_STORAGE_PATH/seafile-data
SEAFILE_MYSQL_VOLUME=$BASIC_STORAGE_PATH/seafile-mysql/db
@ -26,6 +33,21 @@ NOTIFICATION_SERVER_VOLUME=$BASIC_STORAGE_PATH/notification-data
SS_DATA_PATH=$BASIC_STORAGE_PATH/seasearch-data
SEADOC_VOLUME=$BASIC_STORAGE_PATH/seadoc-data
### Storage configurations for S3 (only valid in XXX_STORAGE_TYPE=s3)
S3_COMMIT_BUCKET=<your commit bucket name>
S3_FS_BUCKET=<your fs bucket name>
S3_BLOCK_BUCKET=<your block bucket name>
S3_SS_BUCKET=<your seasearch bucket name> # for seasearch
S3_MD_BUCKET=<your metadata bucket name> # for metadata-server
S3_KEY_ID=<your-key-id>
S3_SECRET_KEY=<your-secret-key>
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=false
S3_AWS_REGION=us-east-1
S3_HOST=
S3_USE_HTTPS=true
S3_SSE_C_KEY=
#################################
# Startup parameters #
#################################
@ -58,22 +80,6 @@ REDIS_PASSWORD=
MEMCACHED_HOST=memcached
MEMCACHED_PORT=11211
## S3
USE_S3_STORAGE=false
S3_COMMIT_BUCKET=<your commit bucket name>
S3_FS_BUCKET=<your fs bucket name>
S3_BLOCK_BUCKET=<your block bucket name>
S3_SS_BUCKET=<your seasearch bucket name> # for seasearch
S3_MD_BUCKET=<your metadata bucket name> # for metadata-server
S3_KEY_ID=<your-key-id>
S3_SECRET_KEY=<your-secret-key>
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=false
S3_AWS_REGION=us-east-1
S3_HOST=
S3_USE_HTTPS=true
S3_SSE_C_KEY=
######################################
# Initial variables #
# (Only valid in first-time startup) #

2
manual/repo/docker/pro/seafile-server.yml
View File

@ -69,7 +69,7 @@ services:
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
- MEMCACHED_HOST=${MEMCACHED_HOST:-memcached}
- MEMCACHED_PORT=${MEMCACHED_PORT:-11211}
- USE_S3_STORAGE=${USE_S3_STORAGE:-false}
- SEAF_SERVER_STORAGE_TYPE=${SEAF_SERVER_STORAGE_TYPE:-disk}
- S3_COMMIT_BUCKET=${S3_COMMIT_BUCKET:-}
- S3_FS_BUCKET=${S3_FS_BUCKET:-}
- S3_BLOCK_BUCKET=${S3_BLOCK_BUCKET:-}

2
manual/repo/docker/pro/seasearch.yml
View File

@ -8,7 +8,7 @@ services:
- SS_FIRST_ADMIN_USER=${INIT_SS_ADMIN_USER:-}
- SS_FIRST_ADMIN_PASSWORD=${INIT_SS_ADMIN_PASSWORD:-}
- SS_MAX_OBJ_CACHE_SIZE=${SS_MAX_OBJ_CACHE_SIZE:-10GB}
- USE_S3_STORAGE=${USE_S3_STORAGE:-false}
- SS_STORAGE_TYPE=${SS_STORAGE_TYPE:-disk}
- S3_SS_BUCKET=${S3_SS_BUCKET:-}
- S3_KEY_ID=${S3_KEY_ID:-}
- S3_USE_V4_SIGNATURE=${S3_USE_V4_SIGNATURE:-true}

4
manual/repo/k8s/cluster/seafile-backend-deployment.yaml
View File

@ -119,11 +119,11 @@ spec:
configMapKeyRef:
name: seafile-env
key: CLUSTER_INIT_ES_PORT
- name: USE_S3_STORAGE
- name: SEAF_SERVER_STORAGE_TYPE
valueFrom:
configMapKeyRef:
name: seafile-env
key: USE_S3_STORAGE
key: SEAF_SERVER_STORAGE_TYPE
- name: S3_COMMIT_BUCKET
valueFrom:
configMapKeyRef:

3
manual/repo/k8s/cluster/seafile-env.yaml
View File

@ -23,8 +23,7 @@ data:
MEMCACHED_PORT: "11211"
# for s3
USE_S3_STORAGE: "false"
S3_STORAGE_BACKEND_CONFIG: "false"
SEAF_SERVER_STORAGE_TYPE: "disk"
S3_COMMIT_BUCKET: ""
S3_FS_BUCKET: ""
S3_BLOCK_BUCKET: ""

4
manual/repo/k8s/cluster/seafile-frontend-deployment.yaml
View File

@ -104,11 +104,11 @@ spec:
configMapKeyRef:
name: seafile-env
key: MEMCACHED_PORT
- name: USE_S3_STORAGE
- name: SEAF_SERVER_STORAGE_TYPE
valueFrom:
configMapKeyRef:
name: seafile-env
key: USE_S3_STORAGE
key: SEAF_SERVER_STORAGE_TYPE
- name: S3_COMMIT_BUCKET
valueFrom:
configMapKeyRef:

9
manual/repo/k8s/pro/seafile-deployment.yaml
View File

@ -81,16 +81,11 @@ spec:
configMapKeyRef:
name: seafile-env
key: SEAFILE_MYSQL_DB_SEAHUB_DB_NAME
- name: USE_S3_STORAGE
- name: SEAF_SERVER_STORAGE_TYPE
valueFrom:
configMapKeyRef:
name: seafile-env
key: USE_S3_STORAGE
- name: S3_STORAGE_BACKEND_CONFIG
valueFrom:
configMapKeyRef:
name: seafile-env
key: S3_STORAGE_BACKEND_CONFIG
key: SEAF_SERVER_STORAGE_TYPE
- name: S3_COMMIT_BUCKET
valueFrom:
configMapKeyRef:

2
manual/repo/k8s/pro/seafile-env.yaml
View File

@ -31,7 +31,7 @@ data:
MEMCACHED_PORT: "11211"
# for S3
USE_S3_STORAGE: "false"
SEAF_SERVER_STORAGE_TYPE: "disk"
S3_COMMIT_BUCKET: ""
S3_FS_BUCKET: ""
S3_BLOCK_BUCKET: ""

19
manual/setup/setup_pro_by_docker.md
View File

@ -75,12 +75,12 @@ The following fields merit particular attention:
| `TIME_ZONE` | Time zone | `UTC` |
| `INIT_SEAFILE_ADMIN_EMAIL` | Synchronously set admin username during initialization | me@example.com |
| `INIT_SEAFILE_ADMIN_PASSWORD` | Synchronously set admin password during initialization | asecret |
| `USE_S3_STORAGE` | Use S3 backend storage in Seafile server | `false` |
| `S3_COMMIT_BUCKET` | S3 storage backend commit objects bucket | (required when `USE_S3_STORAGE=true`) |
| `S3_FS_BUCKET` | S3 storage backend fs objects bucket | (required when you would like to initialize with S3) |
| `S3_BLOCK_BUCKET` | S3 storage backend block objects bucket | (required when `USE_S3_STORAGE=true`) |
| `S3_KEY_ID` | S3 storage backend key ID | (required when `USE_S3_STORAGE=true`) |
| `S3_SECRET_KEY` | S3 storage backend secret key | (required when `USE_S3_STORAGE=true`) |
| `SEAF_SERVER_STORAGE_TYPE` | What kind of the Seafile data for storage. Available options are `disk` (i.e., local disk), `s3` and `multiple` (see the details of [multiple storage backends](./setup_with_multiple_storage_backends.md)) | `disk` |
| `S3_COMMIT_BUCKET` | S3 storage backend commit objects bucket | (required when `SEAF_SERVER_STORAGE_TYPE=s3`) |
| `S3_FS_BUCKET` | S3 storage backend fs objects bucket | (required when `SEAF_SERVER_STORAGE_TYPE=s3`) |
| `S3_BLOCK_BUCKET` | S3 storage backend block objects bucket | (required when `SEAF_SERVER_STORAGE_TYPE=s3`) |
| `S3_KEY_ID` | S3 storage backend key ID | (required when `SEAF_SERVER_STORAGE_TYPE=s3`) |
| `S3_SECRET_KEY` | S3 storage backend secret key | (required when `SEAF_SERVER_STORAGE_TYPE=s3`) |
| `S3_AWS_REGION` | Region of your buckets | `us-east-1` |
| `S3_HOST` | Host of your buckets | (required when not use AWS) |
| `S3_USE_HTTPS` | Use HTTPS connections to S3 if enabled | `true` |
@ -91,9 +91,10 @@ The following fields merit particular attention:
| `NON_ROOT` | Run Seafile container without a root user | `false` |
!!! success "Easier to configure S3 for Seafile and its components"
Since Seafile Pro 13.0, in order to facilitate users to deploy Seafile's related extension components and other services in the future, a section will be provided in `.env` to store the **S3 Configurations** for Seafile and some extension components (such as *SeaSearch*, *Metadata server*). You can locate it with the title bar **\#\#S3**.
However, please note that Seafile only support configuring S3 in `.env` for **single storage backend mode** (i.e., when `USE_S3_STORAGE=true`), if you would like to use multiple backends, please set `USE_S3_STORAGE=false` in `.env` and refer [here](../setup/setup_with_multiple_storage_backends.md) for the details.
Since Seafile Pro 13.0, in order to facilitate users to deploy Seafile's related extension components and other services in the future, a section will be provided in `.env` to store the **S3 Configurations** for Seafile and some extension components (such as *SeaSearch*, *Metadata server*). You can locate it with the title bar **Storage configurations for S3**.
!!! warning "S3 configurations in `.env` only support single S3 storage backend mode"
The Seafile server only support configuring S3 in `.env` for **single S3 storage backend mode** (i.e., when `SEAF_SERVER_STORAGE_TYPE=s3`). If you would like to use other storage backend (e.g., [Ceph](./setup_with_ceph.md), [Swift](./setup_with_swift.md)) or other settings that can only be set in `seafile.conf` (like [multiple storage backends](./setup_with_multiple_storage_backends.md)), please set `SEAF_SERVER_STORAGE_TYPE` to `multiple`, and set `MD_STORAGE_TYPE` and `SS_STORAGE_TYPE` according to your configurations.
To conclude, set the directory permissions of the Elasticsearch volumne:

254
manual/setup/setup_with_s3.md
View File

@ -1,60 +1,46 @@
---
status: new
---
# Setup With S3 Storage
!!! note "Deployment notes"
- This feature is only for ***Pro Edition***
- If your Seafile server is deployed from [binary packages](../setup_binary/installation_pro.md), you have to do the following steps before deploying:
From Seafile 13, there are two ways to configure S3 storage (**single S3 storage backend**) for Seafile server:
- Environment variables (**recommend** since Seafile 13)
- Config file (`seafile.conf`)
!!! note "Setup note for binary packages deployment (Pro)"
If your Seafile server is deployed from [binary packages](../setup_binary/installation_pro.md), you have to do the following steps before deploying:
1. **install `boto3` to your machine**
1. **install `boto3` to your machine**
```sh
sudo pip install boto3
```
```sh
sudo pip install boto3
```
2. **Install and configure [memcached or Redis](../setup_binary/installation_pro.md#setup-memory-cache)**.
For best performance, Seafile requires enable memory cache for objects. We recommend to **at least allocate 128MB** memory for memcached or Redis.
The configuration options differ for different S3 storage. We'll describe the configurations in separate sections. You also need to add [memory cache configurations](../config/seafile-conf.md#cache-pro-edition-only)
2. **Install and configure [memcached or Redis](../setup_binary/installation_pro.md#setup-memory-cache)**.
For best performance, Seafile requires enable memory cache for objects. We recommend to **at least allocate 128MB** memory for memcached or Redis.
!!! tip "New feature from 13.0 pro edition"
If your will deploy Seafile server in Docker, you can specify your S3 configurations in `S3` title bar in `.env` to deploy with S3 together. This operation will generate the same configuration file as this manual.
However, please note that Seafile only support configuring S3 in `.env` for **single storage backend mode** (i.e., when `USE_S3_STORAGE=true`), if you would like to use multiple backends, please set `USE_S3_STORAGE=false` in `.env` and refer [here](../setup/setup_with_multiple_storage_backends.md) for the details.
The configuration options differ for different S3 storage. We'll describe the configurations in separate sections. You also need to add [memory cache configurations](../config/seafile-conf.md#cache-pro-edition-only)
## How to configure S3 in Seafile
Seafile configures S3 storage by adding or modifying the following section in `seafile.conf`:
## Setup swith environment variables (recommend)
```conf
[xxx_object_backend]
name = s3
bucket = my-xxx-objects
key_id = your-key-id
key = your-secret-key
use_v4_signature = true
use_https = true
... ; other optional configurations
```
From Seafile 13, configuring S3 from environment variables will be supported and will provide a more convenient way. You can refer to the detailed description of this part in the introduction of `.env` file. Generally,
You have to create at least **3** buckets for Seafile, corresponding to the sections: `commit_object_backend`, `fs_object_backend` and `block_backend`. For the configurations for each backend section, please refer to the following table:
1. Prepare at least **3** buckets for Seafile (`S3_COMMIT_BUCKET`, `S3_FS_BUCKET` and `S3_BLOCK_BUCKET`).
2. Set `SEAF_SERVER_STORAGE_TYPE` to `true`
3. Fill in the corresponding variable values in `.env` ​​according to the following table:
| Variable | Description |
| --- | --- |
| `bucket` | Bucket name for commit, fs, and block objects. Make sure it follows [S3 naming rules](https://docs.aws.amazon.com/AmazonS3/latest/userguide/BucketRestrictions.html#bucketnamingrules) (you can refer the notes below the table). |
| `key_id` | The `key_id` is required to authenticate you to S3. You can find the `key_id` in the "security credentials" section on your AWS account page or from your storage provider. |
| `key` | The `key` is required to authenticate you to S3. You can find the `key` in the "security credentials" section on your AWS account page or from your storage provider. |
| `use_v4_signature` | There are two versions of authentication protocols that can be used with S3 storage: Version 2 (older, may still be supported by some regions) and Version 4 (current, used by most regions). If you don't set this option, Seafile will use the v2 protocol. It's suggested to use the v4 protocol. |
| `use_https` | Use https to connect to S3. It's recommended to use https. |
| `aws_region` | (**Optional**) If you use the v4 protocol and AWS S3, set this option to the region you chose when you create the buckets. If it's not set and you're using the v4 protocol, Seafile will use `us-east-1` as the default. This option will be ignored if you use the v2 protocol. |
| `host` | (**Optional**) The endpoint by which you access the storage service. Usually it starts with the region name. It's required to provide the host address if you use storage provider other than AWS, *otherwise Seafile will use AWS's address* (i.e., `s3.us-east-1.amazonaws.com`).|
| `sse_c_key` | (**Optional**) A string of 32 characters can be generated by `openssl rand -base64 24`. It can be any 32-character long random string. It's required to use V4 authentication protocol and https if you enable SSE-C. |
| `path_style_request` | (**Optional**) This option asks Seafile to use URLs like `https://192.168.1.123:8080/bucketname/object` to access objects. In Amazon S3, the default URL format is in virtual host style, such as `https://bucketname.s3.amazonaws.com/object`. But this style relies on advanced DNS server setup. So most self-hosted storage systems only implement the path style format. So we recommend to set this option to true for self-hosted storage. |
[1]: <https://docs.aws.amazon.com/AmazonS3/latest/userguide/BucketRestrictions.html#bucketnamingrules> (Replace this placeholder with the actual link to the S3 bucket naming rules documentation if necessary)
| Variable | Description | Default Value |
| ------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------- |
| `S3_COMMIT_BUCKET` | S3 storage backend commit objects bucket | (required) |
| `S3_FS_BUCKET` | S3 storage backend fs objects bucket | (required) |
| `S3_BLOCK_BUCKET` | S3 storage backend block objects bucket | (required) |
| `S3_KEY_ID` | S3 storage backend key ID | (required) |
| `S3_SECRET_KEY` | S3 storage backend secret key | (required) |
| `S3_AWS_REGION` | Region of your buckets | `us-east-1` |
| `S3_HOST` | Host of your buckets | (required when not use AWS) |
| `S3_USE_HTTPS` | Use HTTPS connections to S3 if enabled | `true` |
| `S3_USE_V4_SIGNATURE` | Use the v4 protocol of S3 if enabled | `true` |
| `S3_PATH_STYLE_REQUEST` | This option asks Seafile to use URLs like `https://192.168.1.123:8080/bucketname/object` to access objects. In *Amazon S3*, the default URL format is in virtual host style, such as `https://bucketname.s3.amazonaws.com/object`. But this style relies on advanced DNS server setup. So most self-hosted storage systems only implement the path style format. | `false` |
| `S3_SSE_C_KEY` | A string of 32 characters can be generated by openssl rand -base64 24. It can be any 32-character long random string. It's required to use V4 authentication protocol and https if you enable SSE-C. | (none) |
!!! warning "Bucket naming conventions"
@ -72,47 +58,138 @@ You have to create at least **3** buckets for Seafile, corresponding to the sect
- seafileFSObject
- seafile block object
!!! note "About S3_SSE_C_KEY"
`S3_SSE_C_KEY` is a string of 32 characters.
### Use server-side encryption with customer-provided keys (SSE-C) in Seafile
You can generate sse_c_key with the following command. Note that the key doesn't have to be base64 encoded. It can be any 32-character long random string. The example just show one possible way to generate such a key.
Since Pro 11.0, you can use SSE-C to S3. Add the following `sse_c_key` to seafile.conf (as shown in the above variables table):
```
openssl rand -base64 24
```
Howevery, if you have existing data in your S3 storage bucket, **turning on the above configuration will make your data inaccessible**. That's because Seafile server doesn't support encrypted and non-encrypted objects mixed in the same bucket. You have to create a new bucket, and migrate your data to it by following [storage backend migration documentation](./migrate_backends_data.md#migrating-to-sse-c-encrypted-s3-storage).
!!! tip "For other S3 support extensions"
In addition to Seafile server, the following extensions (if already installed) will share the same S3 authorization information in `.env` with Seafile server:
- [SeaSearch](./use_seasearch.md): Enable the feature by specifying `SS_STORAGE_TYPE=s3` and `S3_SS_BUCKET`
- [Metadata server](../extension/metadata-server.md): Enable the feature by specifying `MD_STORAGE_TYPE=s3` and `S3_MD_BUCKET`
### Example configurations
=== "AWS"
```sh
SEAF_SERVER_STORAGE_TYPE=s3
S3_COMMIT_BUCKET=my-commit-objects
S3_FS_BUCKET=my-fs-objects
S3_BLOCK_BUCKET=my-block-objects
S3_KEY_ID=your-key-id
S3_SECRET_KEY=your-secret-key
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=false
S3_AWS_REGION=eu-central-1
S3_HOST=
S3_USE_HTTPS=true
```
=== "Exoscale"
```conf
SEAF_SERVER_STORAGE_TYPE=s3
S3_COMMIT_BUCKET=my-commit-objects
S3_FS_BUCKET=my-fs-objects
S3_BLOCK_BUCKET=my-block-objects
S3_KEY_ID=your-key-id
S3_SECRET_KEY=your-secret-key
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=true
S3_AWS_REGION=eu-central-1 # will be ignored when S3_HOST is specified
S3_HOST=sos-de-fra-1.exo.io
S3_USE_HTTPS=true
```
=== "Hetzner"
```conf
SEAF_SERVER_STORAGE_TYPE=s3
S3_COMMIT_BUCKET=my-commit-objects
S3_FS_BUCKET=my-fs-objects
S3_BLOCK_BUCKET=my-block-objects
S3_KEY_ID=your-key-id
S3_SECRET_KEY=your-secret-key
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=true
S3_AWS_REGION=eu-central-1 # will be ignored when S3_HOST is specified
S3_HOST=fsn1.your-objectstorage.com
S3_USE_HTTPS=true
```
=== "Other Public Hosted S3 Storage"
There are other S3-compatible cloud storage providers in the market, such as ***Blackblaze*** and ***Wasabi***. Configuration for those providers are just a bit different from AWS. We don't assure the following configuration works for all providers. If you have problems please contact our support
```
SEAF_SERVER_STORAGE_TYPE=s3
S3_COMMIT_BUCKET=my-commit-objects
S3_FS_BUCKET=my-fs-objects
S3_BLOCK_BUCKET=my-block-objects
S3_KEY_ID=your-key-id
S3_SECRET_KEY=your-secret-key
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=true
S3_AWS_REGION=eu-central-1 # will be ignored when S3_HOST is specified
S3_HOST=<access endpoint for storage provider>
S3_USE_HTTPS=true
```
=== "Self-hosted S3 Storage"
Many self-hosted object storage systems are now compatible with the S3 API, such as ***OpenStack Swift***, ***Ceph's RADOS Gateway*** and ***Minio***. You can use these S3-compatible storage systems as backend for Seafile. Here is an example config:
```
SEAF_SERVER_STORAGE_TYPE=s3
S3_COMMIT_BUCKET=my-commit-objects
S3_FS_BUCKET=my-fs-objects
S3_BLOCK_BUCKET=my-block-objects
S3_KEY_ID=your-key-id
S3_SECRET_KEY=your-secret-key
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=true
S3_AWS_REGION=eu-central-1 # will be ignored when S3_HOST is specified
S3_HOST=<your s3 api endpoint host>:<your s3 api endpoint port>
S3_USE_HTTPS=true # according to your S3 configuration
```
## Setup with config file
Seafile configures S3 storage by adding or modifying the following section in `seafile.conf`:
```conf
[commit_object_backend]
[xxx_object_backend]
name = s3
......
bucket = my-xxx-objects
key_id = your-key-id
key = your-secret-key
use_v4_signature = true
use_https = true
sse_c_key = XiqMSf3x5ja4LRibBbV0sVntVpdHXl3P
[fs_object_backend]
name = s3
......
use_v4_signature = true
use_https = true
sse_c_key = XiqMSf3x5ja4LRibBbV0sVntVpdHXl3P
[block_backend]
name = s3
......
use_v4_signature = true
use_https = true
sse_c_key = XiqMSf3x5ja4LRibBbV0sVntVpdHXl3P
... ; other optional configurations
```
`sse_c_key` is a string of 32 characters.
Similar to configure in `.env`, you have to create at least **3** buckets for Seafile too, corresponding to the sections: `commit_object_backend`, `fs_object_backend` and `block_backend`. For the configurations for each backend section, please refer to the following table:
You can generate sse_c_key with the following command. Note that the key doesn't have to be base64 encoded. It can be any 32-character long random string. The example just show one possible way to generate such a key.
| Variable | Description |
| --- | --- |
| `bucket` | Bucket name for commit, fs, and block objects. Make sure it follows [S3 naming rules](https://docs.aws.amazon.com/AmazonS3/latest/userguide/BucketRestrictions.html#bucketnamingrules) (you can refer the notes below the table). |
| `key_id` | The `key_id` is required to authenticate you to S3. You can find the `key_id` in the "security credentials" section on your AWS account page or from your storage provider. |
| `key` | The `key` is required to authenticate you to S3. You can find the `key` in the "security credentials" section on your AWS account page or from your storage provider. |
| `use_v4_signature` | There are two versions of authentication protocols that can be used with S3 storage: Version 2 (older, may still be supported by some regions) and Version 4 (current, used by most regions). If you don't set this option, Seafile will use the v2 protocol. It's suggested to use the v4 protocol. |
| `use_https` | Use https to connect to S3. It's recommended to use https. |
| `aws_region` | (**Optional**) If you use the v4 protocol and AWS S3, set this option to the region you chose when you create the buckets. If it's not set and you're using the v4 protocol, Seafile will use `us-east-1` as the default. This option will be ignored if you use the v2 protocol. |
| `host` | (**Optional**) The endpoint by which you access the storage service. Usually it starts with the region name. It's required to provide the host address if you use storage provider other than AWS, *otherwise Seafile will use AWS's address* (i.e., `s3.us-east-1.amazonaws.com`).|
| `sse_c_key` | (**Optional**) A string of 32 characters can be generated by `openssl rand -base64 24`. It can be any 32-character long random string. It's required to use V4 authentication protocol and https if you enable SSE-C. |
| `path_style_request` | (**Optional**) This option asks Seafile to use URLs like `https://192.168.1.123:8080/bucketname/object` to access objects. In Amazon S3, the default URL format is in virtual host style, such as `https://bucketname.s3.amazonaws.com/object`. But this style relies on advanced DNS server setup. So most self-hosted storage systems only implement the path style format. So we recommend to set this option to true for self-hosted storage. |
```
openssl rand -base64 24
```
[1]: <https://docs.aws.amazon.com/AmazonS3/latest/userguide/BucketRestrictions.html#bucketnamingrules> (Replace this placeholder with the actual link to the S3 bucket naming rules documentation if necessary)
!!! warning
If you have existing data in your S3 storage bucket, turning on the above configuration will make your data inaccessible. That's because Seafile server doesn't support encrypted and non-encrypted objects mixed in the same bucket. You have to create a new bucket, and migrate your data to it by following [storage backend migration documentation](./migrate_backends_data.md#migrating-to-sse-c-encrypted-s3-storage).
## Example
### Example configurations
=== "AWS"
@ -278,6 +355,33 @@ openssl rand -base64 24
use_https = true
```
!!! tip "Use server-side encryption with customer-provided keys (SSE-C) in Seafile"
Since Pro 11.0, you can use SSE-C to S3. Add the following `sse_c_key` to seafile.conf (as shown in the above variables table):
```conf
[commit_object_backend]
name = s3
......
use_v4_signature = true
use_https = true
sse_c_key = XiqMSf3x5ja4LRibBbV0sVntVpdHXl3P
[fs_object_backend]
name = s3
......
use_v4_signature = true
use_https = true
sse_c_key = XiqMSf3x5ja4LRibBbV0sVntVpdHXl3P
[block_backend]
name = s3
......
use_v4_signature = true
use_https = true
sse_c_key = XiqMSf3x5ja4LRibBbV0sVntVpdHXl3P
```
## Run and Test ##
Now you can start Seafile and test

23
manual/setup/use_seasearch.md
View File

@ -38,18 +38,21 @@ SEASEARCH_IMAGE=seafileltd/seasearch:1.0-latest
SS_DATA_PATH=/opt/seasearch-data
INIT_SS_ADMIN_USER=<admin-username>
INIT_SS_ADMIN_PASSWORD=<admin-password>
# if you would like to use S3 for saving seasearch data
SS_STORAGE_TYPE=s3
S3_SS_BUCKET=...
S3_KEY_ID=<your-key-id>
S3_SECRET_KEY=<your-secret-key>
S3_USE_V4_SIGNATURE=true
S3_PATH_STYLE_REQUEST=false
S3_AWS_REGION=us-east-1
S3_HOST=
S3_USE_HTTPS=true
S3_SSE_C_KEY=
```
!!! success "Easier to configure S3 for Seafile and its components"
Since Seafile Pro 13.0, in order to facilitate users to deploy Seafile's related extension components and other services in the future, a section will be provided in `.env` to store the **S3 Configurations** for Seafile and some extension components (such as *SeaSearch*, *Metadata server*). You can locate it with the title bar **\#\#S3**.
In other words, if you deploy SeaSearch and Seafile together, and if you have deployed Seafile Pro following [here](../setup/setup_pro_by_docker.md#downloading-and-modifying-env) (and using the latest `.env`), you only need to specify the following variables in `.env` to make it work (that is, the `USE_S3_STORAGE` is set to `true`):
```sh
USE_S3_STORAGE=true
S3_SS_BUCKET=<your s3 bucket name for SeaSearch>
```
## Modify `seafile-server.yml` to disable `elasticSearch` service
If you would like to use *SeaSearch* as the search engine, the `elasticSearch` service can be removed, which is no longer used: remove `elasticsearch.yml` in the list variable `COMPOSE_FILE` on the file `.env`.