mirror of
https://github.com/haiwen/seafile-admin-docs.git
synced 2025-12-25 18:22:48 +00:00
opt documents & k8s
This commit is contained in:
Junxiang Huang
parent
a6e239a454
commit
26b0dc988f
|
|
@ -34,7 +34,7 @@ Let's assume you have three nodes in your cluster: A, B, and C.
|
|||
|
||||
## Cluster requirements
|
||||
|
||||
Please refer [here](./system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster.
|
||||
Please refer [here](./system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster. In general, we recommend that each node should have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
## Deploy Seafile service
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ This manual explains how to deploy and run Seafile cluster on a Linux server usi
|
|||
|
||||
## Cluster requirements
|
||||
|
||||
Please refer [here](./system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster.
|
||||
Please refer [here](./system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster. In general, we recommend that each node should have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
### K8S tools
|
||||
|
||||
|
|
@ -68,10 +68,10 @@ nano /opt/seafile-k8s-yaml/seafile-secret.yaml
|
|||
For the fields marked with `<...>` are **required**, please make sure these items are filled in, otherwise Seafile server may not run properly.
|
||||
|
||||
## Initialize Seafile cluster
|
||||
You can use following command to initialize Seafile cluster now:
|
||||
You can use following command to initialize Seafile cluster now (the Seafile's K8S resources will be specified in namespace `seafile` for easier management):
|
||||
|
||||
```shell
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/ -n seafile
|
||||
```
|
||||
|
||||
!!! note "About Seafile cluster initialization"
|
||||
|
|
@ -81,7 +81,7 @@ kubectl apply -f /opt/seafile-k8s-yaml/
|
|||
- `CLUSTER_INIT_MODE=true`
|
||||
|
||||
!!! success
|
||||
You can get the following information through `kubectl logs seafile-xxxx` (for details about this opeartions, please refer to [here](#container-management)) to check the initialization process is done or not:
|
||||
You can get the following information through `kubectl logs seafile-xxxx -n seafile` to check the initialization process is done or not:
|
||||
|
||||
```
|
||||
---------------------------------
|
||||
|
|
@ -146,8 +146,7 @@ Finally you can use the `tar -zcvf` and `tar -zxvf` commands to package the enti
|
|||
If you modify some configurations in `/opt/seafile/shared/seafile/conf` or YAML files in `/opt/seafile-k8s-yaml/`, you still need to restart services to make modifications.
|
||||
|
||||
```shell
|
||||
kubectl delete -f /opt/seafile-k8s-yaml/
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/
|
||||
kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
|
||||
```
|
||||
|
||||
!!! sucess
|
||||
|
|
@ -180,258 +179,14 @@ Finally you can use the `tar -zcvf` and `tar -zxvf` commands to package the enti
|
|||
Done.
|
||||
```
|
||||
|
||||
## Container management
|
||||
## Uninstall Seafile K8S
|
||||
|
||||
Similar to docker installation, you can also manage containers through [some kubectl commands](https://kubernetes.io/docs/reference/kubectl/#operations). For example, you can use the following command to check whether the relevant resources are started successfully and whether the relevant services can be accessed normally. First, execute the following command and remember the pod name with `seafile-<node type>-` as the prefix (such as `seafile-frontend-748b695648-d6l4g`)
|
||||
You can uninstall the Seafile K8S by the following command:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```sh
|
||||
kubectl delete -f /opt/seafile-k8s-yaml/ -n seafile
|
||||
```
|
||||
|
||||
You can check a status of a pod by
|
||||
## Advanced operations
|
||||
|
||||
```shell
|
||||
kubectl logs seafile-frontend-748b695648-d6l4g
|
||||
```
|
||||
|
||||
and enter a container by
|
||||
|
||||
```shell
|
||||
kubectl exec -it seafile-frontend-748b695648-d6l4g -- bash
|
||||
```
|
||||
|
||||
## Load balance and HTTPS
|
||||
|
||||
When deploying a Seafile cluster using K8S, you can enable HTTPS and use load balance in the following two ways:
|
||||
|
||||
- External load balancing server, such as *Nginx*. Typically you will need to reverse proxy `http://<your control plane>/`
|
||||
- K8S Gateway API, e.g., [Nginx-gateway](https://docs.nginx.com/nginx-gateway-fabric/installation/installing-ngf/manifests/) and [Istio-gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/)
|
||||
|
||||
## Log routing and aggregation system
|
||||
|
||||
Similar to [Single-pod Seafile](./k8s_single_node.md), you can browse the log files of Seafile running directly in the persistent volume directory. The difference is that when using K8S to deploy a Seafile cluster (especially in a cloud environment), the persistent volume created is usually shared and synchronized for all nodes. However, ***the logs generated by the Seafile service do not record the specific node information where these logs are located***, so browsing the files in the above folder may make it difficult to identify which node these logs are generated from. Therefore, one solution proposed here is:
|
||||
|
||||
1. Record the generated logs to the standard output. In this way, the logs can be distinguished under each node by `kubectl logs` (but all types of logs will be output together now). You can enable this feature (**it should be enabled by default in K8S Seafile cluster but not in K8S single-pod Seafile**) by modifing `SEAFILE_LOG_TO_STDOUT` to `true` in `seafile-env.yaml`:
|
||||
|
||||
```yaml
|
||||
...
|
||||
data:
|
||||
...
|
||||
SEAFILE_LOG_TO_STDOUT: "true"
|
||||
...
|
||||
```
|
||||
|
||||
Then restart the Seafile server:
|
||||
|
||||
```sh
|
||||
kubectl delete -f /opt/seafile-k8s-yaml/
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/
|
||||
```
|
||||
|
||||
2. Since the logs in step 1 can be distinguished between nodes, but they are aggregated and output together, it is not convenient for log retrieval. So you have to route the standard output logs (i.e., distinguish logs by corresponding components name) and re-record them in a new file or upload them to a log aggregation system (e.g., [*Loki*](https://grafana.com/oss/loki/)).
|
||||
|
||||
Currently in the K8S environment, the commonly used log routing plugins are:
|
||||
|
||||
- [*Fluent Bit*](https://fluentbit.io/)
|
||||
- [*Fluentd*](https://www.fluentd.org/)
|
||||
- [*Logstash*](https://www.elastic.co/logstash/)
|
||||
- [*Promtail*](https://grafana.com/loki/docs/sources/promtail/) (also a part of Loki)
|
||||
|
||||
***Fluent Bit*** and ***Promtail*** are more lightweight (i.e., consume less system resources), while *Promtail* only supports transferring logs to *Loki*. Therefore, this document will mainly introduce log routing through ***Fluent Bit*** which is a fast, lightweight logs and metrics agent. It is also a CNCF graduated sub-project under the umbrella of *Fluentd*. *Fluent Bit* is licensed under the terms of the Apache License v2.0. You should deploy the *Fluent Bit* in your K8S cluster by following [offical document](https://docs.fluentbit.io/manual/installation/kubernetes) firstly. Then modify Fluent-Bit pod settings to mount a new directory to load the configuration files:
|
||||
|
||||
```yaml
|
||||
#kubectl edit ds fluent-bit
|
||||
|
||||
...
|
||||
spec:
|
||||
...
|
||||
spec:
|
||||
...
|
||||
containers:
|
||||
- name: fluent-bit
|
||||
volumeMounts:
|
||||
...
|
||||
- mountPath: /fluent-bit/etc/seafile
|
||||
name: fluent-bit-seafile
|
||||
- mountPath: /
|
||||
...
|
||||
...
|
||||
volumes:
|
||||
...
|
||||
- hostPath:
|
||||
path: /opt/fluent-bit
|
||||
name: fluent-bit-seafile
|
||||
```
|
||||
|
||||
and
|
||||
|
||||
```yaml
|
||||
#kubectl edit cm fluent-bit
|
||||
|
||||
data:
|
||||
...
|
||||
fluent-bit.conf: |
|
||||
[SERVICE]
|
||||
...
|
||||
Parsers_File /fluent-bit/etc/seafile/confs/parsers.conf
|
||||
...
|
||||
@INCLUDE /fluent-bit/etc/seafile/confs/*-log.conf
|
||||
```
|
||||
|
||||
For example in here, we use `/opt/fluent-bit/confs` (**it has to be non-shared**). What's more, the parsers will be defined in `/opt/fluent-bit/confs/parsers.conf`, and for each type log (e.g., *seahub*'s log, *seafevent*'s log) will be defined in `/opt/fluent-bit/confs/*-log.conf`. Each `.conf` file defines several Fluent-Bit data pipeline components:
|
||||
|
||||
| **Pipeline** | **Description** | **Required/Optional** |
|
||||
| ------------- | --------------- | --------------------- |
|
||||
| **INPUT** | Specifies where and how Fluent-Bit can get the original log information, and assigns a tag for each log record after read. | Required |
|
||||
| **PARSER** | Parse the read log records. For K8S Docker runtime logs, they are usually in Json format. | Required |
|
||||
| **FILTER** | Filters and selects log records with a specified tag, and assigns a new tag to new records. | Optional |
|
||||
| **OUTPUT** | tells Fluent-Bit what format the log records for the specified tag will be in and where to output them (such as file, *Elasticsearch*, *Loki*, etc.). | Required |
|
||||
|
||||
!!! warning
|
||||
For ***PARSER***, it can only be stored in `/opt/fluent-bit/confs/parsers.conf`, otherwise the Fluent-Bit cannot startup normally.
|
||||
|
||||
### Inputer
|
||||
|
||||
According to the above, a container will generate a log file (usually in `/var/log/containers/<container-name>-xxxxxx.log`), so you need to prepare an importer and add the following information (for more details, please refer to offical document about [*TAIL inputer*](https://docs.fluentbit.io/manual/pipeline/inputs/tail)) in `/opt/fluent-bit/confs/seafile-log.conf`:
|
||||
|
||||
```conf
|
||||
[INPUT]
|
||||
Name tail
|
||||
Path /var/log/containers/seafile-frontend-*.log
|
||||
Buffer_Chunk_Size 2MB
|
||||
Buffer_Max_Size 10MB
|
||||
Docker_Mode On
|
||||
Docker_Mode_Flush 5
|
||||
Tag seafile.*
|
||||
Parser Docker # for definition, please see the next section as well
|
||||
|
||||
[INPUT]
|
||||
Name tail
|
||||
Path /var/log/containers/seafile-backend-*.log
|
||||
Buffer_Chunk_Size 2MB
|
||||
Buffer_Max_Size 10MB
|
||||
Docker_Mode On
|
||||
Docker_Mode_Flush 5
|
||||
Tag seafile.*
|
||||
Parser Docker
|
||||
```
|
||||
|
||||
The above defines two importers, which are used to monitor seafile-frontend and seafile-backend services respectively. The reason why they are written together here is that for a node, you may not know when it will run the frontend service and when it will run the backend service, but they have the same tag prefix `seafile.`.
|
||||
|
||||
### Parser
|
||||
|
||||
Each input has to use a parser to parse the logs and pass them to the filter. Here, a parser named `Docker` is created to parse the logs generated by the *K8S-docker-runtime container*. The parser is placed in `/opt/fluent-bit/confs/parser.conf` (for more details, please refer to offical document about [JSON parser](https://docs.fluentbit.io/manual/pipeline/parsers/json)):
|
||||
|
||||
```conf
|
||||
[PARSER]
|
||||
Name Docker
|
||||
Format json
|
||||
Time_Key time
|
||||
Time_Format %Y-%m-%dT%H:%M:%S.%LZ
|
||||
|
||||
```
|
||||
|
||||
!!! tip "Log records after parsing"
|
||||
The logs of the Docker container are saved in /var/log/containers in **Json** format (see the sample below), which is why we use the `Json` format in the above parser.
|
||||
|
||||
```json
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(86): fileserver: web_token_expire_time = 3600\n","stream":"stdout","time":"2025-01-17T07:43:48.294638442Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(98): fileserver: max_index_processing_threads= 3\n","stream":"stdout","time":"2025-01-17T07:43:48.294810145Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(111): fileserver: fixed_block_size = 8388608\n","stream":"stdout","time":"2025-01-17T07:43:48.294879777Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(123): fileserver: max_indexing_threads = 1\n","stream":"stdout","time":"2025-01-17T07:43:48.295002479Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(138): fileserver: put_head_commit_request_timeout = 10\n","stream":"stdout","time":"2025-01-17T07:43:48.295082733Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(150): fileserver: skip_block_hash = 0\n","stream":"stdout","time":"2025-01-17T07:43:48.295195843Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] ../common/seaf-utils.c(553): Use database Mysql\n","stream":"stdout","time":"2025-01-17T07:43:48.29704895Z"}
|
||||
```
|
||||
|
||||
When these logs are obtained by the importer and parsed by the parser, they will become independent log records with the following fields:
|
||||
|
||||
- `log`: The original log content (i.e., same as you seen in `kubectl logs seafile-xxx`) and an extra line break at the end (i.e., `\n`). **This is also the field we need to save or upload to the log aggregation system in the end**.
|
||||
- `stream`: The original log come from. `stdout` means the *standard output*.
|
||||
- `time`: The time when the log is recorded in the corresponding stream (ISO 8601 format).
|
||||
|
||||
|
||||
### Filter
|
||||
|
||||
Add two filters in `/opt/fluent-bit/confs/seafile-log.conf` for records filtering and routing. Here, the [*record_modifier* filter](https://docs.fluentbit.io/manual/pipeline/filters/record-modifier) is to select useful keys (see the contents in above *tip* label, only the `log` field is what we need) in the log records and [*rewrite_tag* filter](https://docs.fluentbit.io/manual/pipeline/filters/rewrite-tag) is used to route logs according to specific rules:
|
||||
|
||||
```conf
|
||||
[FILTER]
|
||||
Name record_modifier
|
||||
Match seafile.*
|
||||
Allowlist_key log
|
||||
|
||||
|
||||
[FILTER]
|
||||
Name rewrite_tag
|
||||
Match seafile.*
|
||||
Rule $log ^.*\[seaf-server\].*$ seaf-server false # for seafile's logs
|
||||
Rule $log ^.*\[seahub\].*$ seahub false # for seahub's logs
|
||||
Rule $log ^.*\[seafevents\].*$ seafevents false # for seafevents' lgos
|
||||
Rule $log ^.*\[seafile-slow-rpc\].*$ seafile-slow-rpc false # for slow-rpc's logs
|
||||
```
|
||||
|
||||
### Output log's to *Loki*
|
||||
|
||||
Loki is multi-tenant log aggregation system inspired by *Prometheus*. It is designed to be very cost effective and easy to operate. The Fluent-Bit *loki* built-in output plugin allows you to send your log or events to a *Loki service*. It supports data enrichment with Kubernetes labels, custom label keys and Tenant ID within others.
|
||||
|
||||
!!! tip "Alternative Fluent-Bit Loki plugin by *Grafana*"
|
||||
For sending logs to Loki, there are [two plugins](https://grafana.com/docs/loki/latest/send-data/fluentbit/) for Fluent-Bit:
|
||||
|
||||
- The [built-in *Loki* plugin](https://docs.fluentbit.io/manual/pipeline/outputs/loki) maintained by the Fluent-Bit officially, and we will use it in this part because it provides the most complete features.
|
||||
- [*Grafana-loki* plugin](https://grafana.com/docs/loki/latest/send-data/fluentbit/community-plugin/) maintained by *Grafana Labs*.
|
||||
|
||||
|
||||
Due to each outputer dose not have a distinguishing marks in the configuration files (because Fluent-Bit takes each plugin as a tag workflow):
|
||||
|
||||
- ***Seaf-server log***: Add an outputer to `/opt/fluent-bit/confs/seaf-server-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seaf-server
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
- ***seahub log***: Add an outputer to `/opt/fluent-bit/confs/seahub-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seahub
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
- ***seafevents log***: Add an outputer to `/opt/fluent-bit/confs/seafevents-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seafevents
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
- ***seafile-slow-rpc log***: Add an outputer to `/opt/fluent-bit/confs/seafile-slow-rpc-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seafile-slow-rpc
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
!!! tip "Cloud Loki instance"
|
||||
If you are using a cloud Loki instance, you can follow the [Fluent-Bit Loki plugin document](https://docs.fluentbit.io/manual/pipeline/outputs/loki) to fill up all necessary fields. Usually, the following fields are **additional needs** in cloud Loki service:
|
||||
|
||||
- `tls`
|
||||
- `tls.verify`
|
||||
- `http_user`
|
||||
- `http_passwd`
|
||||
Please refer from [here](./k8s_advanced_management) for futher advanced operations.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ This manual explains how to deploy and run Seafile cluster on a Linux server usi
|
|||
|
||||
## Cluster requirements
|
||||
|
||||
Please refer [here](./system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster.
|
||||
Please refer [here](./system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster. In general, we recommend that each node should have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
### K8S tools
|
||||
|
||||
|
|
@ -39,6 +39,8 @@ After installation, you need to start the k8s control plane service on each node
|
|||
--from-literal=INIT_S3_SECRET_KEY=''
|
||||
```
|
||||
|
||||
where the `JWT_PRIVATE_KEY` can be generate by `pwgen -s 40 1`
|
||||
|
||||
3. Download and modify the `my-values.yaml` according to your configurations. By the way, you can follow [here](./setup_pro_by_docker.md#downloading-and-modifying-env) for the details:
|
||||
|
||||
```sh
|
||||
|
|
@ -132,7 +134,7 @@ After installation, you need to start the k8s control plane service on each node
|
|||
|
||||
```
|
||||
|
||||
5. After the first-time startup, you have to turn off (i.e., set `CLUSTER_INIT_MODE` to `false`) in your `my-values.yaml`, then upgrade the chart:
|
||||
5. After the first-time startup, you have to turn off (i.e., set `initMode` to `false`) in your `my-values.yaml`, then upgrade the chart:
|
||||
|
||||
```sh
|
||||
helm upgrade --install seafile seafile/cluster --namespace seafile --create-namespace --values my-values.yaml
|
||||
|
|
@ -234,18 +236,51 @@ After installation, you need to start the k8s control plane service on each node
|
|||
```bash
|
||||
kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
|
||||
```
|
||||
|
||||
## Uninstall chart
|
||||
|
||||
You can uninstall chart by the following command:
|
||||
!!! tip "A safer way to use your Seafile license file"
|
||||
You can also create a *secret* resource to encrypt your license file in your K8S cluster, which is a safer way:
|
||||
|
||||
```sh
|
||||
helm delete seafile --namespace seafile
|
||||
```
|
||||
```sh
|
||||
kubectl create secret generic seafile-license --from-file=seafile-license.txt=$PATH_TO_YOUR_LICENSE_FILE --namespace seafile
|
||||
```
|
||||
|
||||
## Advanced operations
|
||||
Then modify `my-values.yaml` to add the information extra volumes:
|
||||
|
||||
### Version control
|
||||
```yaml
|
||||
seafile:
|
||||
...
|
||||
extraVolumes:
|
||||
backend:
|
||||
- name: seafileLicense
|
||||
volumeInfo:
|
||||
secret:
|
||||
secretName: seafile-license
|
||||
items:
|
||||
- key: seafile-license.txt
|
||||
path: seafile-license.txt
|
||||
subPath: seafile-license.txt
|
||||
mountPath: /shared/seafile/seafile-license.txt
|
||||
readOnly: true
|
||||
frontend:
|
||||
- name: seafileLicense
|
||||
volumeInfo:
|
||||
secret:
|
||||
secretName: seafile-license
|
||||
items:
|
||||
- key: seafile-license.txt
|
||||
path: seafile-license.txt
|
||||
subPath: seafile-license.txt
|
||||
mountPath: /shared/seafile/seafile-license.txt
|
||||
readOnly: true
|
||||
```
|
||||
|
||||
Finally you can upgrade your chart by:
|
||||
|
||||
```sh
|
||||
helm upgrade --install seafile seafile/cluster --namespace seafile --create-namespace --values my-values.yaml
|
||||
```
|
||||
|
||||
## Version control
|
||||
|
||||
Seafile Helm Chart is designed to provide fast deployment and version control. You can update and rollback versions using the following setps:
|
||||
|
||||
|
|
@ -286,53 +321,14 @@ Seafile Helm Chart is designed to provide fast deployment and version control. Y
|
|||
helm rollback seafile -n seafile <revision>
|
||||
```
|
||||
|
||||
### Extra volumes
|
||||
## Uninstall chart
|
||||
|
||||
In this part, we take encrypted seafile-license file for example. You can also create a *secret* resource to encrypt your license file in your K8S cluster:
|
||||
You can uninstall chart by the following command:
|
||||
|
||||
```sh
|
||||
kubectl create secret generic seafile-license --from-file=seafile-license.txt=$PATH_TO_YOUR_LICENSE_FILE --namespace seafile
|
||||
helm delete seafile --namespace seafile
|
||||
```
|
||||
|
||||
Then modify `my-values.yaml` to add the information extra volumes:
|
||||
## Advanced operations
|
||||
|
||||
```yaml
|
||||
seafile:
|
||||
...
|
||||
extraVolumes:
|
||||
backend:
|
||||
- name: seafileLicense
|
||||
volumeInfo:
|
||||
secret:
|
||||
secretName: seafile-license
|
||||
items:
|
||||
- key: seafile-license.txt
|
||||
path: seafile-license.txt
|
||||
subPath: seafile-license.txt
|
||||
mountPath: /shared/seafile/seafile-license.txt
|
||||
readOnly: true
|
||||
frontend:
|
||||
- name: seafileLicense
|
||||
volumeInfo:
|
||||
secret:
|
||||
secretName: seafile-license
|
||||
items:
|
||||
- key: seafile-license.txt
|
||||
path: seafile-license.txt
|
||||
subPath: seafile-license.txt
|
||||
mountPath: /shared/seafile/seafile-license.txt
|
||||
readOnly: true
|
||||
```
|
||||
|
||||
Finally you can upgrade your chart by:
|
||||
|
||||
```sh
|
||||
helm upgrade --install seafile seafile/cluster --namespace seafile --create-namespace --values my-values.yaml
|
||||
```
|
||||
|
||||
!!! success "Flexible in Seafile Helm Chart"
|
||||
No only the extra volumes, you can also define the other environment variables objects (i.e., `extraEnv`) and resources objects (i.e., `extraResources`) in `my-values.yaml`
|
||||
|
||||
### Other advanced operations
|
||||
|
||||
Please refer from [here](./cluster_deploy_with_k8s.md#load-balance-and-https) for futher advanced operations.
|
||||
Please refer from [here](./k8s_advanced_management) for futher advanced operations.
|
||||
|
|
@ -12,7 +12,7 @@ By the way, we don't provide the deployment methods of basic services (e.g., **M
|
|||
|
||||
### System requirements
|
||||
|
||||
Please refer [here](./system_requirements.md) for system requirements about Seafile service. By the way, this will apply to all nodes where Seafile pods may appear in your K8S cluster.
|
||||
Please refer [here](./system_requirements.md) for the details of system requirements about Seafile service. By the way, this will apply to all nodes where Seafile pods may appear in your K8S cluster. In general, we recommend that each node should have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
## Install Seafile helm chart
|
||||
|
||||
|
|
@ -44,6 +44,8 @@ Please refer [here](./system_requirements.md) for system requirements about Seaf
|
|||
--from-literal=INIT_SEAFILE_MYSQL_ROOT_PASSWORD='<required>'
|
||||
```
|
||||
|
||||
where the `JWT_PRIVATE_KEY` can be generate by `pwgen -s 40 1`
|
||||
|
||||
3. Download and modify the `my-values.yaml` according to your configurations. By the way, you can follow [here](../config/env.md) for the details:
|
||||
|
||||
=== "Seafile Pro"
|
||||
|
|
@ -116,37 +118,46 @@ Then restart Seafile:
|
|||
kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
|
||||
```
|
||||
|
||||
## Container management
|
||||
!!! tip "A safer way to use your Seafile license file"
|
||||
You can also create a *secret* resource to encrypt your license file in your K8S cluster, which is a safer way:
|
||||
|
||||
Similar to docker installation, you can also manage containers through [some kubectl commands](https://kubernetes.io/docs/reference/kubectl/#operations). For example, you can use the following command to check whether the relevant resources are started successfully and whether the relevant services can be accessed normally. First, execute the following command and remember the pod name with `seafile-` as the prefix (such as `seafile-748b695648-d6l4g`)
|
||||
```sh
|
||||
kubectl create secret generic seafile-license --from-file=seafile-license.txt=$PATH_TO_YOUR_LICENSE_FILE --namespace seafile
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl get pods -n seafile
|
||||
```
|
||||
Then modify `my-values.yaml` to add the information extra volumes:
|
||||
|
||||
You can check a status of a pod by
|
||||
```yaml
|
||||
seafile:
|
||||
...
|
||||
extraVolumes:
|
||||
- name: seafileLicense
|
||||
volumeInfo:
|
||||
secret:
|
||||
secretName: seafile-license
|
||||
items:
|
||||
- key: seafile-license.txt
|
||||
path: seafile-license.txt
|
||||
subPath: seafile-license.txt
|
||||
mountPath: /shared/seafile/seafile-license.txt
|
||||
readOnly: true
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl logs seafile-748b695648-d6l4g -n seafile
|
||||
```
|
||||
Finally you can upgrade your chart by:
|
||||
|
||||
and enter a container by
|
||||
=== "Seafile Pro"
|
||||
|
||||
```shell
|
||||
kubectl exec -it seafile-748b695648-d6l4g -n seafile -- bash
|
||||
```
|
||||
```sh
|
||||
helm upgrade --install seafile seafile/pro --namespace seafile --create-namespace --values my-values.yaml
|
||||
```
|
||||
|
||||
## Uninstall chart
|
||||
=== "Seafile CE"
|
||||
|
||||
You can uninstall chart by the following command:
|
||||
```sh
|
||||
helm upgrade --install seafile seafile/ce --namespace seafile --create-namespace --values my-values.yaml
|
||||
```
|
||||
|
||||
```sh
|
||||
helm delete seafile --namespace seafile
|
||||
```
|
||||
|
||||
## Advanced operations
|
||||
|
||||
### Version control
|
||||
## Version control
|
||||
|
||||
Seafile Helm Chart is designed to provide fast deployment and version control. You can update and rollback versions using the following setps:
|
||||
|
||||
|
|
@ -202,49 +213,14 @@ Seafile Helm Chart is designed to provide fast deployment and version control. Y
|
|||
helm rollback seafile -n seafile <revision>
|
||||
```
|
||||
|
||||
### Extra volumes
|
||||
## Uninstall chart
|
||||
|
||||
In this part, we take encrypted seafile-license file for example. You can also create a *secret* resource to encrypt your license file in your K8S cluster:
|
||||
You can uninstall chart by the following command:
|
||||
|
||||
```sh
|
||||
kubectl create secret generic seafile-license --from-file=seafile-license.txt=$PATH_TO_YOUR_LICENSE_FILE --namespace seafile
|
||||
helm delete seafile --namespace seafile
|
||||
```
|
||||
|
||||
Then modify `my-values.yaml` to add the information extra volumes:
|
||||
## Advanced operations
|
||||
|
||||
```yaml
|
||||
seafile:
|
||||
...
|
||||
extraVolumes:
|
||||
- name: seafileLicense
|
||||
volumeInfo:
|
||||
secret:
|
||||
secretName: seafile-license
|
||||
items:
|
||||
- key: seafile-license.txt
|
||||
path: seafile-license.txt
|
||||
subPath: seafile-license.txt
|
||||
mountPath: /shared/seafile/seafile-license.txt
|
||||
readOnly: true
|
||||
```
|
||||
|
||||
Finally you can upgrade your chart by:
|
||||
|
||||
=== "Seafile Pro"
|
||||
|
||||
```sh
|
||||
helm upgrade --install seafile seafile/pro --namespace seafile --create-namespace --values my-values.yaml
|
||||
```
|
||||
|
||||
=== "Seafile CE"
|
||||
|
||||
```sh
|
||||
helm upgrade --install seafile seafile/ce --namespace seafile --create-namespace --values my-values.yaml
|
||||
```
|
||||
|
||||
!!! success "Flexible in Seafile Helm Chart"
|
||||
No only the extra volumes, you can also define the other environment variables objects (i.e., `extraEnv`) and resources objects (i.e., `extraResources`) in `my-values.yaml`
|
||||
|
||||
### Other advanced operations
|
||||
|
||||
Please refer from [here](./k8s_single_node.md#https) for futher advanced operations.
|
||||
Please refer from [here](./k8s_advanced_management.md) for futher advanced operations.
|
||||
|
|
@ -0,0 +1,412 @@
|
|||
# Seafile K8S advanced management
|
||||
|
||||
This document mainly describes how to manage and maintain Seafile deployed through our K8S deployment document. At the same time, if you are already proficient in using kubectl commands to manage K8S resources, you can also customize the deployment solutions we provide.
|
||||
|
||||
!!! note "Namespaces for Seafile K8S deployment"
|
||||
Our documentation provides two deployment solutions for both single-node and cluster deployment (via Seafile Helm Chart and K8S resource files), both of which can be highly customized.
|
||||
|
||||
Regardless of which deployment method you use, in our newer manuals (usually in versions after Seafile 12.0.9), Seafile-related K8S resources (including related Pods, services, and persistent volumes, etc.) are defined in the `seafile` namespace. In previous versions, you may deploy Seafile in the `default` namespace, so in this case, when referring to this document for Seafile K8S resource management, be sure to remove `-n seafile` in the command.
|
||||
|
||||
## Seafile K8S Container management
|
||||
|
||||
Similar to docker installation, you can also manage containers through [some kubectl commands](https://kubernetes.io/docs/reference/kubectl/#operations). For example, you can use the following command to check whether the relevant resources are started successfully and whether the relevant services can be accessed normally. First, execute the following command and remember the pod name with `seafile-` as the prefix (such as `seafile-748b695648-d6l4g`)
|
||||
|
||||
```shell
|
||||
kubectl get pods -n seafile
|
||||
```
|
||||
|
||||
You can check a status of a pod by
|
||||
|
||||
```shell
|
||||
kubectl logs seafile-748b695648-d6l4g -n seafile
|
||||
```
|
||||
|
||||
and enter a container by
|
||||
|
||||
```shell
|
||||
kubectl exec -it seafile-748b695648-d6l4g -n seafile -- bash
|
||||
```
|
||||
|
||||
Also, you can restart the services by the following commands:
|
||||
|
||||
```sh
|
||||
kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
|
||||
```
|
||||
|
||||
## K8S Gateway and HTTPS
|
||||
|
||||
Since the Ingress feature [is no longer supported](https://kubernetes.io/docs/concepts/services-networking/ingress/) in the new version of K8S (even the commonly used *Nginx-Ingress* will not be deployed after 1.24), this article will introduce how to use the new version of K8S feature [*K8S Gateway*](https://kubernetes.io/docs/concepts/services-networking/gateway/) to implement Seafile service exposure and load balancing.
|
||||
|
||||
!!! tip "Still use *Nginx-Ingress*"
|
||||
If your K8S is still running with an old version, and still using *Nginx-Ingress*, you can follow [here](https://artifacthub.io/packages/helm/datamate/seafile#deploy-an-ingress-controller-ingress-nginx) to setup ingress controller and HTTPS. We sincerely thanks *Datamate* to give an example to this configuration.
|
||||
|
||||
For the details and features about ***K8S Gateway***, please refer to the K8S [official document](https://kubernetes.io/docs/concepts/services-networking/gateway/#design-principles), you can simpily install it by
|
||||
|
||||
```sh
|
||||
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml
|
||||
```
|
||||
|
||||
The *Gateway API* requires configuration of three API categories in its [resource model](https://kubernetes.io/docs/concepts/services-networking/gateway/#resource-model):
|
||||
- `GatewayClass`: Defines a group of gateways with the same configuration, managed by the controller that implements the class.
|
||||
- `Gateway`: Defines an instance of traffic handling infrastructure, which can be thought of as a load balancer.
|
||||
- `HTTPRoute`: Defines HTTP-specific rules for mapping traffic from gateway listeners to representations of backend network endpoints. These endpoints are typically represented as Services.
|
||||
|
||||
### GatewayClass
|
||||
|
||||
The ***GatewayClass*** resource serves the same purpose as the `IngressClass` in the old-ingress API, similar to the *StorageClass* in the *Storage API.* It defines the categories of Gateways that can be created. Typically, this resource is provided by your infrastructure platform, such as EKS or GKE. It can also be provided by a third-party Ingress Controller, such as [Nginx-gateway](https://docs.nginx.com/nginx-gateway-fabric/overview/gateway-architecture/) or [Istio-gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/).
|
||||
|
||||
Here, we take the *Nginx-gateway* for the example, and you can install it with the [official document](https://docs.nginx.com/nginx-gateway-fabric/installation/installing-ngf/manifests/). After installation, you can view the installation status with the following command:
|
||||
|
||||
```sh
|
||||
# `gc` means the `gatewayclass`, and its same as `kubectl get gatewayclass`
|
||||
kubectl get gc
|
||||
|
||||
#NAME CONTROLLER ACCEPTED AGE
|
||||
#nginx gateway.nginx.org/nginx-gateway-controller True 22s
|
||||
```
|
||||
|
||||
Typically, after you install GatewayClass, your cloud provider will provide you with a load balancing IP, which is visible in GatewayClass. If this IP is not assigned, you can manually bind it to a IP that can be accessed from exteranl network.
|
||||
|
||||
```sh
|
||||
kubectl edit svc nginx-gateway -n nginx-gateway
|
||||
```
|
||||
|
||||
and modify the following section:
|
||||
|
||||
```yaml
|
||||
...
|
||||
spec:
|
||||
...
|
||||
externalIPs:
|
||||
- <your external IP>
|
||||
externalTrafficPolicy: Cluster
|
||||
...
|
||||
...
|
||||
```
|
||||
|
||||
### Gateway
|
||||
|
||||
***Gateway*** is used to describe an instance of traffic processing infrastructure. Usually, *Gateway* defines a network endpoint that can be used to process traffic, that is, to filter, balance, split, etc. Service and other backends. For example, it can represent a cloud load balancer, or a cluster proxy server configured to accept HTTP traffic. As above, please refer to the official documentation for a detailed description of [Gateway](https://kubernetes.io/docs/concepts/services-networking/gateway/#api-kind-gateway). Here is only a simple reference configuration for Seafile:
|
||||
|
||||
```yaml
|
||||
# nano seafile-gateway/gateway.yaml
|
||||
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: Gateway
|
||||
metadata:
|
||||
name: seafile-gateway
|
||||
spec:
|
||||
gatewayClassName: nginx
|
||||
listeners:
|
||||
- name: seafile-http
|
||||
protocol: HTTP
|
||||
port: 80
|
||||
```
|
||||
|
||||
### HTTPRoute
|
||||
|
||||
The ***HTTPRoute*** category specifies the routing behavior of HTTP requests from the *Gateway* listener to the backend network endpoints. For service backends, the implementation can represent the backend network endpoint as a service IP or a supporting endpoint of the service. it represents the configuration that will be applied to the underlying *Gateway* implementation. For example, defining a new *HTTPRoute* may result in configuring additional traffic routes in a cloud load balancer or in-cluster proxy server. As above, please refer to the official documentation for a detailed description of the [HTTPRoute](https://kubernetes.io/docs/concepts/services-networking/gateway/#api-kind-httproute) resource. Here is only a reference configuration solution that is only applicable to this document.
|
||||
|
||||
```yaml
|
||||
# nano seafile-gateway/httproute.yaml
|
||||
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: seafile-httproute
|
||||
spec:
|
||||
parentRefs:
|
||||
- group: gateway.networking.k8s.io
|
||||
kind: Gateway
|
||||
name: seafile-gateway
|
||||
hostnames:
|
||||
- "<your domain>"
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
backendRefs:
|
||||
- name: seafile
|
||||
port: 80
|
||||
|
||||
```
|
||||
|
||||
After installing or defining ***GatewayClass***, ***Gateway*** and ***HTTPRoute***, you can now enable this feature by following command and view your Seafile server by the URL `http://seafile.example.com/`:
|
||||
|
||||
```sh
|
||||
kubectl apply -f seafile-gateway -n seafile
|
||||
```
|
||||
|
||||
### Enable HTTPS (Optional)
|
||||
|
||||
When using *K8S Gateway*, a common way to enable HTTPS is to add relevant information about the TLS listener in *Gateway* resource. You can [refer here](https://gateway-api.sigs.k8s.io/guides/tls/#examples) for futher details. We will provide a simple way here so that you can quickly enable HTTPS for your Seafile K8S.
|
||||
|
||||
1. Create a *secret* resource (`seafile-tls-cert`) for your TLS certificates:
|
||||
|
||||
```sh
|
||||
kubectl create secret tls seafile-tls-cert \
|
||||
--cert=<your path to fullchain.pem> \
|
||||
--key=<your path to privkey.pem>
|
||||
```
|
||||
2. Use the TLS in your *Gateway* resource and enable HTTPS:
|
||||
|
||||
```yaml
|
||||
# nano seafile-gateway/gateway.yaml
|
||||
|
||||
...
|
||||
spec:
|
||||
...
|
||||
listeners:
|
||||
- name: seafile-http
|
||||
...
|
||||
tls:
|
||||
certificateRefs:
|
||||
- kind: Secret
|
||||
group: ""
|
||||
name: seafile-tls-cert
|
||||
...
|
||||
```
|
||||
|
||||
3. Modify `seahub_settings.py`:
|
||||
|
||||
```py
|
||||
SERVICE_URL = "https://<your domain>/"
|
||||
```
|
||||
|
||||
4. Restart Seafile K8S Gateway:
|
||||
|
||||
```sh
|
||||
kubectl delete -f seafile-gateway -n seafile
|
||||
kubectl apply -f seafile-gateway -n seafile
|
||||
```
|
||||
|
||||
Now you can access your Seafile service in `https://<your domain>/`
|
||||
|
||||
## Log routing and aggregating system
|
||||
|
||||
Similar to single-node deployment, you can browse the log files of Seafile running directly in the persistent volume directory (i.e., `<path>/seafile/logs`). The difference is that when using K8S to deploy a Seafile cluster (especially in a cloud environment), the persistent volume created is usually shared and synchronized for all nodes. However, ***the logs generated by the Seafile service do not record the specific node information where these logs are located***, so browsing the files in the above folder may make it difficult to identify which node these logs are generated from. Therefore, one solution proposed here is:
|
||||
|
||||
1. Record the generated logs to the standard output. In this way, the logs can be distinguished under each node by `kubectl logs` (but all types of logs will be output together now). You can enable this feature (**it should be enabled by default in K8S Seafile cluster but not in K8S single-pod Seafile**) by modifing `SEAFILE_LOG_TO_STDOUT` to `true` in `seafile-env.yaml`:
|
||||
|
||||
```yaml
|
||||
...
|
||||
data:
|
||||
...
|
||||
SEAFILE_LOG_TO_STDOUT: "true"
|
||||
...
|
||||
```
|
||||
|
||||
Then restart the Seafile server:
|
||||
|
||||
```sh
|
||||
kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
|
||||
```
|
||||
|
||||
2. Since the logs in step 1 can be distinguished between nodes, but they are aggregated and output together, it is not convenient for log retrieval. So you have to route the standard output logs (i.e., distinguish logs by corresponding components name) and re-record them in a new file or upload them to a log aggregation system (e.g., [*Loki*](https://grafana.com/oss/loki/)).
|
||||
|
||||
Currently in the K8S environment, the commonly used log routing plugins are:
|
||||
|
||||
- [*Fluent Bit*](https://fluentbit.io/)
|
||||
- [*Fluentd*](https://www.fluentd.org/)
|
||||
- [*Logstash*](https://www.elastic.co/logstash/)
|
||||
- [*Promtail*](https://grafana.com/loki/docs/sources/promtail/) (also a part of Loki)
|
||||
|
||||
***Fluent Bit*** and ***Promtail*** are more lightweight (i.e., consume less system resources), while *Promtail* only supports transferring logs to *Loki*. Therefore, this document will mainly introduce log routing through ***Fluent Bit*** which is a fast, lightweight logs and metrics agent. It is also a CNCF graduated sub-project under the umbrella of *Fluentd*. *Fluent Bit* is licensed under the terms of the Apache License v2.0. You should deploy the *Fluent Bit* in your K8S cluster by following [offical document](https://docs.fluentbit.io/manual/installation/kubernetes) firstly. Then modify Fluent-Bit pod settings to mount a new directory to load the configuration files:
|
||||
|
||||
```yaml
|
||||
#kubectl edit ds fluent-bit
|
||||
|
||||
...
|
||||
spec:
|
||||
...
|
||||
spec:
|
||||
...
|
||||
containers:
|
||||
- name: fluent-bit
|
||||
volumeMounts:
|
||||
...
|
||||
- mountPath: /fluent-bit/etc/seafile
|
||||
name: fluent-bit-seafile
|
||||
- mountPath: /
|
||||
...
|
||||
...
|
||||
volumes:
|
||||
...
|
||||
- hostPath:
|
||||
path: /opt/fluent-bit
|
||||
name: fluent-bit-seafile
|
||||
```
|
||||
|
||||
and
|
||||
|
||||
```yaml
|
||||
#kubectl edit cm fluent-bit
|
||||
|
||||
data:
|
||||
...
|
||||
fluent-bit.conf: |
|
||||
[SERVICE]
|
||||
...
|
||||
Parsers_File /fluent-bit/etc/seafile/confs/parsers.conf
|
||||
...
|
||||
@INCLUDE /fluent-bit/etc/seafile/confs/*-log.conf
|
||||
```
|
||||
|
||||
For example in here, we use `/opt/fluent-bit/confs` (**it has to be non-shared**). What's more, the parsers will be defined in `/opt/fluent-bit/confs/parsers.conf`, and for each type log (e.g., *seahub*'s log, *seafevent*'s log) will be defined in `/opt/fluent-bit/confs/*-log.conf`. Each `.conf` file defines several Fluent-Bit data pipeline components:
|
||||
|
||||
| **Pipeline** | **Description** | **Required/Optional** |
|
||||
| ------------- | --------------- | --------------------- |
|
||||
| **INPUT** | Specifies where and how Fluent-Bit can get the original log information, and assigns a tag for each log record after read. | Required |
|
||||
| **PARSER** | Parse the read log records. For K8S Docker runtime logs, they are usually in Json format. | Required |
|
||||
| **FILTER** | Filters and selects log records with a specified tag, and assigns a new tag to new records. | Optional |
|
||||
| **OUTPUT** | tells Fluent-Bit what format the log records for the specified tag will be in and where to output them (such as file, *Elasticsearch*, *Loki*, etc.). | Required |
|
||||
|
||||
!!! warning
|
||||
For ***PARSER***, it can only be stored in `/opt/fluent-bit/confs/parsers.conf`, otherwise the Fluent-Bit cannot startup normally.
|
||||
|
||||
### Inputer
|
||||
|
||||
According to the above, a container will generate a log file (usually in `/var/log/containers/<container-name>-xxxxxx.log`), so you need to prepare an importer and add the following information (for more details, please refer to offical document about [*TAIL inputer*](https://docs.fluentbit.io/manual/pipeline/inputs/tail)) in `/opt/fluent-bit/confs/seafile-log.conf`:
|
||||
|
||||
```conf
|
||||
[INPUT]
|
||||
Name tail
|
||||
Path /var/log/containers/seafile-frontend-*.log
|
||||
Buffer_Chunk_Size 2MB
|
||||
Buffer_Max_Size 10MB
|
||||
Docker_Mode On
|
||||
Docker_Mode_Flush 5
|
||||
Tag seafile.*
|
||||
Parser Docker # for definition, please see the next section as well
|
||||
|
||||
[INPUT]
|
||||
Name tail
|
||||
Path /var/log/containers/seafile-backend-*.log
|
||||
Buffer_Chunk_Size 2MB
|
||||
Buffer_Max_Size 10MB
|
||||
Docker_Mode On
|
||||
Docker_Mode_Flush 5
|
||||
Tag seafile.*
|
||||
Parser Docker
|
||||
```
|
||||
|
||||
The above defines two importers, which are used to monitor seafile-frontend and seafile-backend services respectively. The reason why they are written together here is that for a node, you may not know when it will run the frontend service and when it will run the backend service, but they have the same tag prefix `seafile.`.
|
||||
|
||||
### Parser
|
||||
|
||||
Each input has to use a parser to parse the logs and pass them to the filter. Here, a parser named `Docker` is created to parse the logs generated by the *K8S-docker-runtime container*. The parser is placed in `/opt/fluent-bit/confs/parser.conf` (for more details, please refer to offical document about [JSON parser](https://docs.fluentbit.io/manual/pipeline/parsers/json)):
|
||||
|
||||
```conf
|
||||
[PARSER]
|
||||
Name Docker
|
||||
Format json
|
||||
Time_Key time
|
||||
Time_Format %Y-%m-%dT%H:%M:%S.%LZ
|
||||
|
||||
```
|
||||
|
||||
!!! tip "Log records after parsing"
|
||||
The logs of the Docker container are saved in /var/log/containers in **Json** format (see the sample below), which is why we use the `Json` format in the above parser.
|
||||
|
||||
```json
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(86): fileserver: web_token_expire_time = 3600\n","stream":"stdout","time":"2025-01-17T07:43:48.294638442Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(98): fileserver: max_index_processing_threads= 3\n","stream":"stdout","time":"2025-01-17T07:43:48.294810145Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(111): fileserver: fixed_block_size = 8388608\n","stream":"stdout","time":"2025-01-17T07:43:48.294879777Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(123): fileserver: max_indexing_threads = 1\n","stream":"stdout","time":"2025-01-17T07:43:48.295002479Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(138): fileserver: put_head_commit_request_timeout = 10\n","stream":"stdout","time":"2025-01-17T07:43:48.295082733Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] seafile-session.c(150): fileserver: skip_block_hash = 0\n","stream":"stdout","time":"2025-01-17T07:43:48.295195843Z"}
|
||||
{"log":"[seaf-server] [2025-01-17 07:43:48] [INFO] ../common/seaf-utils.c(553): Use database Mysql\n","stream":"stdout","time":"2025-01-17T07:43:48.29704895Z"}
|
||||
```
|
||||
|
||||
When these logs are obtained by the importer and parsed by the parser, they will become independent log records with the following fields:
|
||||
|
||||
- `log`: The original log content (i.e., same as you seen in `kubectl logs seafile-xxx -n seafile`) and an extra line break at the end (i.e., `\n`). **This is also the field we need to save or upload to the log aggregation system in the end**.
|
||||
- `stream`: The original log come from. `stdout` means the *standard output*.
|
||||
- `time`: The time when the log is recorded in the corresponding stream (ISO 8601 format).
|
||||
|
||||
|
||||
### Filter
|
||||
|
||||
Add two filters in `/opt/fluent-bit/confs/seafile-log.conf` for records filtering and routing. Here, the [*record_modifier* filter](https://docs.fluentbit.io/manual/pipeline/filters/record-modifier) is to select useful keys (see the contents in above *tip* label, only the `log` field is what we need) in the log records and [*rewrite_tag* filter](https://docs.fluentbit.io/manual/pipeline/filters/rewrite-tag) is used to route logs according to specific rules:
|
||||
|
||||
```conf
|
||||
[FILTER]
|
||||
Name record_modifier
|
||||
Match seafile.*
|
||||
Allowlist_key log
|
||||
|
||||
|
||||
[FILTER]
|
||||
Name rewrite_tag
|
||||
Match seafile.*
|
||||
Rule $log ^.*\[seaf-server\].*$ seaf-server false # for seafile's logs
|
||||
Rule $log ^.*\[seahub\].*$ seahub false # for seahub's logs
|
||||
Rule $log ^.*\[seafevents\].*$ seafevents false # for seafevents' lgos
|
||||
Rule $log ^.*\[seafile-slow-rpc\].*$ seafile-slow-rpc false # for slow-rpc's logs
|
||||
```
|
||||
|
||||
### Output log's to *Loki*
|
||||
|
||||
Loki is multi-tenant log aggregation system inspired by *Prometheus*. It is designed to be very cost effective and easy to operate. The Fluent-Bit *loki* built-in output plugin allows you to send your log or events to a *Loki service*. It supports data enrichment with Kubernetes labels, custom label keys and Tenant ID within others.
|
||||
|
||||
!!! tip "Alternative Fluent-Bit Loki plugin by *Grafana*"
|
||||
For sending logs to Loki, there are [two plugins](https://grafana.com/docs/loki/latest/send-data/fluentbit/) for Fluent-Bit:
|
||||
|
||||
- The [built-in *Loki* plugin](https://docs.fluentbit.io/manual/pipeline/outputs/loki) maintained by the Fluent-Bit officially, and we will use it in this part because it provides the most complete features.
|
||||
- [*Grafana-loki* plugin](https://grafana.com/docs/loki/latest/send-data/fluentbit/community-plugin/) maintained by *Grafana Labs*.
|
||||
|
||||
|
||||
Due to each outputer dose not have a distinguishing marks in the configuration files (because Fluent-Bit takes each plugin as a tag workflow):
|
||||
|
||||
- ***Seaf-server log***: Add an outputer to `/opt/fluent-bit/confs/seaf-server-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seaf-server
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
- ***seahub log***: Add an outputer to `/opt/fluent-bit/confs/seahub-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seahub
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
- ***seafevents log***: Add an outputer to `/opt/fluent-bit/confs/seafevents-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seafevents
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
- ***seafile-slow-rpc log***: Add an outputer to `/opt/fluent-bit/confs/seafile-slow-rpc-log.conf`:
|
||||
|
||||
```conf
|
||||
[OUTPUT]
|
||||
Name loki
|
||||
Match seafile-slow-rpc
|
||||
Host <your Loki's host>
|
||||
port <your Loki's port>
|
||||
labels job=fluentbit, node_name=<your-node-name>, node_id=<your-node-id> # node_name and node_id is optional, but recommended for identifying the source node
|
||||
```
|
||||
|
||||
!!! tip "Cloud Loki instance"
|
||||
If you are using a cloud Loki instance, you can follow the [Fluent-Bit Loki plugin document](https://docs.fluentbit.io/manual/pipeline/outputs/loki) to fill up all necessary fields. Usually, the following fields are **additional needs** in cloud Loki service:
|
||||
|
||||
- `tls`
|
||||
- `tls.verify`
|
||||
- `http_user`
|
||||
- `http_passwd`
|
||||
|
|
@ -6,7 +6,7 @@ For specific environment and configuration requirements, please refer to the des
|
|||
|
||||
## System requirements
|
||||
|
||||
Please refer [here](./system_requirements.md) for system requirements about Seafile service. By the way, this will apply to all nodes where Seafile pods may appear in your K8S cluster.
|
||||
Please refer [here](./system_requirements.md) for the details of system requirements about Seafile service. By the way, this will apply to all nodes where Seafile pods may appear in your K8S cluster. In general, we recommend that each node should have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
## Gettings started
|
||||
|
||||
|
|
@ -77,10 +77,10 @@ nano /opt/seafile-k8s-yaml/seafile-secret.yaml
|
|||
|
||||
## Start Seafile server
|
||||
|
||||
You can start Seafile server simply by
|
||||
You can start Seafile server and specify the resources into the namespace `seafile` for easier management by
|
||||
|
||||
```sh
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/ -n seafile
|
||||
```
|
||||
|
||||
!!! warning "Important for Pro edition"
|
||||
|
|
@ -98,8 +98,7 @@ kubectl apply -f /opt/seafile-k8s-yaml/
|
|||
Please modfiy the files in `/opt/seafile-data/seafile/conf` (especially the `seafevents.conf`, `seafile.conf` and `seahub_settings.py`) to make correct the configurations for above services, otherwise the Seafile server cannot start normally. Then restart Seafile server:
|
||||
|
||||
```sh
|
||||
kubectl delete -f /opt/seafile-k8s-yaml/
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/
|
||||
kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
|
||||
```
|
||||
|
||||
## Activating the Seafile License (Pro)
|
||||
|
|
@ -111,37 +110,17 @@ If you have a `seafile-license.txt` license file, simply put it in the volume of
|
|||
Then restart Seafile:
|
||||
|
||||
```bash
|
||||
kubectl delete -f /opt/seafile-k8s-yaml/
|
||||
kubectl apply -f /opt/seafile-k8s-yaml/
|
||||
kubectl delete pods -n seafile $(kubectl get pods -n seafile -o jsonpath='{.items[*].metadata.name}' | grep seafile)
|
||||
```
|
||||
|
||||
## Container management
|
||||
## Uninstall Seafile K8S
|
||||
|
||||
Similar to docker installation, you can also manage containers through [some kubectl commands](https://kubernetes.io/docs/reference/kubectl/#operations). For example, you can use the following command to check whether the relevant resources are started successfully and whether the relevant services can be accessed normally. First, execute the following command and remember the pod name with `seafile-` as the prefix (such as `seafile-748b695648-d6l4g`)
|
||||
You can uninstall the Seafile K8S by the following command:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```sh
|
||||
kubectl delete -f /opt/seafile-k8s-yaml/ -n seafile
|
||||
```
|
||||
|
||||
You can check a status of a pod by
|
||||
## Advanced operations
|
||||
|
||||
```shell
|
||||
kubectl logs seafile-748b695648-d6l4g
|
||||
```
|
||||
|
||||
and enter a container by
|
||||
|
||||
```shell
|
||||
kubectl exec -it seafile-748b695648-d6l4g -- bash
|
||||
```
|
||||
|
||||
## HTTPS
|
||||
|
||||
Please refer to [here](./cluster_deploy_with_k8s.md#load-balance-and-https) about suggestions of enabling HTTPS in K8S.
|
||||
|
||||
## Seafile directory structure
|
||||
|
||||
Please refer to [here](./setup_pro_by_docker.md#seafile-directory-structure) for the details.
|
||||
|
||||
!!! tip "Send logs to Loki"
|
||||
You can directly view the log files of single-pod Seafile in the persistent volume directory, as the log files are distinguishable even the node of pod has changed (because there will only be one node running Seafile), so by default single-pod Seafile logs are not output to standard output. If you need to record these log files to a log server (e.g., [*Loki*](https://grafana.com/oss/loki/)), you can refer to [here](./cluster_deploy_with_k8s.md#log-routing-and-aggregation-system) for more informations.
|
||||
Please refer from [here](./k8s_advanced_management) for futher advanced operations.
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## System requirements
|
||||
|
||||
Please refer [here](./system_requirements.md#seafile-ce) for system requirements about Seafile CE.
|
||||
Please refer [here](./system_requirements.md#seafile-ce) for system requirements about Seafile CE. In general, we recommend that you have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
## Getting started
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ This manual explains how to deploy and run Seafile Server Professional Edition (
|
|||
|
||||
## System requirements
|
||||
|
||||
Please refer [here](./system_requirements.md#seafile-pro) for system requirements about Seafile PE.
|
||||
Please refer [here](./system_requirements.md#seafile-pro) for system requirements about Seafile PE. In general, we recommend that you have at least 4G RAM and a 4-core CPU (> 2GHz).
|
||||
|
||||
!!! tip "About license"
|
||||
Seafile PE can be used without a paid license with up to three users. Licenses for more user can be purchased in the [Seafile Customer Center](https://customer.seafile.com) or contact Seafile Sales at [sales@seafile.com](mailto:sales@seafile.com). For futher details, please refer the [license page](../setup_binary/seafile_professional_sdition_software_license_agreement.md) of Seafile PE.
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
|
||||
## Cluster requirements
|
||||
|
||||
Please refer [here](../setup/system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster.
|
||||
Please refer [here](../setup/system_requirements.md#seafile-cluster) for the details about the cluster requirements for **all nodes** in Seafile cluster. In general, we recommend that each node should have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
## Preparation (all nodes)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ This manual explains how to deploy and run Seafile Server Community Edition (Sea
|
|||
|
||||
## Requirements
|
||||
|
||||
Please refer [here](../setup/system_requirements.md#seafile-ce) for system requirements about Seafile CE.
|
||||
Please refer [here](../setup/system_requirements.md#seafile-ce) for system requirements about Seafile CE. In general, we recommend that you should have at least 2G RAM and a 2-core CPU (> 2GHz).
|
||||
|
||||
|
||||
## Setup
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ This manual explains how to deploy and run Seafile Server Professional Edition (
|
|||
|
||||
## Requirements
|
||||
|
||||
Please refer [here](../setup/system_requirements.md#seafile-pro) for system requirements about Seafile PE.
|
||||
Please refer [here](../setup/system_requirements.md#seafile-pro) for system requirements about Seafile PE. In general, we recommend that you should have at least 4G RAM and a 4-core CPU (> 2GHz).
|
||||
|
||||
Seafile PE can be used without a paid license with up to three users. Licenses for more user can be purchased in the [Seafile Customer Center](https://customer.seafile.com) or contact Seafile Sales at [sales@seafile.com](mailto:sales@seafile.com) or one of [our partners](https://www.seafile.com/en/partner/).
|
||||
|
||||
|
|
|
|||
|
|
@ -55,5 +55,9 @@
|
|||
}
|
||||
|
||||
.md-nav__item--section {
|
||||
margin-top: 2em;
|
||||
margin-top: 3em;
|
||||
}
|
||||
|
||||
:root [data-md-color-scheme=default]{
|
||||
--md-default-fg-color--light: #333333;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -114,6 +114,7 @@ nav:
|
|||
- By K8S resource files:
|
||||
- Single node: setup/k8s_single_node.md
|
||||
- Cluster (Pro): setup/cluster_deploy_with_k8s.md
|
||||
- Seafile K8S advanced management: setup/k8s_advanced_management.md
|
||||
- Migration:
|
||||
- Migration from Seafile Community: setup/migrate_ce_to_pro_with_docker.md
|
||||
- Migrate from non-docker deployment: setup/migrate_non_docker_to_docker.md
|
||||
|
|
|
|||
Loading…
Reference in New Issue