V4.14.0 features (#5850)
Some checks are pending
Document deploy / sync-images (push) Waiting to run
Document deploy / generate-timestamp (push) Blocked by required conditions
Document deploy / build-images (map[domain:https://fastgpt.cn suffix:cn]) (push) Blocked by required conditions
Document deploy / build-images (map[domain:https://fastgpt.io suffix:io]) (push) Blocked by required conditions
Document deploy / update-images (map[deployment:fastgpt-docs domain:https://fastgpt.cn kube_config:KUBE_CONFIG_CN suffix:cn]) (push) Blocked by required conditions
Document deploy / update-images (map[deployment:fastgpt-docs domain:https://fastgpt.io kube_config:KUBE_CONFIG_IO suffix:io]) (push) Blocked by required conditions
Build FastGPT images in Personal warehouse / get-vars (push) Waiting to run
Build FastGPT images in Personal warehouse / build-fastgpt-images (map[arch:amd64 runs-on:ubuntu-24.04]) (push) Blocked by required conditions
Build FastGPT images in Personal warehouse / build-fastgpt-images (map[arch:arm64 runs-on:ubuntu-24.04-arm]) (push) Blocked by required conditions
Build FastGPT images in Personal warehouse / release-fastgpt-images (push) Blocked by required conditions

* feat: migrate chat files to s3 (#5802)

* feat: migrate chat files to s3

* feat: add delete jobs for deleting s3 files

* chore: improvements

* fix: lockfile

* fix: imports

* feat: add ttl for those uploaded files but not send yet

* feat: init bullmq worker

* fix: s3 key

* perf: s3 internal url

* remove env

* fix: re-sign a new url

* fix: re-sign a new url

* perf: s3 code

---------

Co-authored-by: archer <545436317@qq.com>

* update pacakge

* feat: add more file type for uploading (#5807)

* fix: re-sign a new url

* wip: file selector

* feat: add more file type for uploading

* feat: migrate chat files to s3 (#5802)

* feat: migrate chat files to s3

* feat: add delete jobs for deleting s3 files

* chore: improvements

* fix: lockfile

* fix: imports

* feat: add ttl for those uploaded files but not send yet

* feat: init bullmq worker

* fix: s3 key

* perf: s3 internal url

* remove env

* fix: re-sign a new url

* fix: re-sign a new url

* perf: s3 code

---------

Co-authored-by: archer <545436317@qq.com>

* fix: limit minmax available file upload number

* perf: file select modal code

* fix: fileselect refresh

* fix: ts

---------

Co-authored-by: archer <545436317@qq.com>

* bugfix: chat page (#5809)

* fix: upload avatar

* fix: chat page username display issue and setting button visibility

* doc

* Markdown match base64 performance

* feat: improve global variables(time, file, dataset) (#5804)

* feat: improve global variables(time, file, dataset)

* feat: optimize code

* perf: time variables code

* fix: model, file

* fix: hide file upload

* fix: ts

* hide dataset select

---------

Co-authored-by: archer <545436317@qq.com>

* perf: insert training queue

* perf: s3 upload error i18n

* fix: share page s3

* fix: timeselector ui error

* var update node

* Timepicker ui

* feat: plugin support password

* fix: password disabled UX

* fix: button size

* fix: no model cache for chat page (#5820)

* rename function

* fix: workflow bug

* fix: interactive loop

* fix test

* perf: common textare no richtext

* move system plugin config (#5803) (#5813)

* move system plugin config (#5803)

* move system plugin config

* extract tag bar

* filter

* tool detail temp

* marketplace

* params

* fix

* type

* search

* tags render

* status

* ui

* code

* connect to backend (#5815)

* feat: marketplace apis & type definitions (#5817)

* chore: marketplace init

* chore: marketplace list api type

* chore: detail api

* marketplace & import

* feat: marketplace ui (#5826)

* temp

* marketplace

* import

* feat: detail return readme

* chore: cache data expire 10 mins

* chore: update docs

* feat: marketplace ui

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* feat: marketplace (#5830)

* temp

* marketplace

* chore: tool list tag filter

* chore: adjust

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* tool detail drawer

* remove tag filter

* fix

* fix

* fix build

* update pnpm-lock

* fix type

* perf code

* marketplace router

* fix build

* navbar icon

* fix ui

* fix init

* docs: marketplace/plugin (#5832)

* temp

* marketplace

* docs(plugin): system tool docs

---------

Co-authored-by: heheer <zhiyu44@qq.com>

* default url

* feat: i18n/ docker build (#5833)

* chore: docker build

* feat: i18n selector

* fix

* fix

* fix: i18n parse

* fix: i18n parse

---------

Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: heheer <zhiyu44@qq.com>

* marketplace url

* update action

* market place code

* market place code

* title

* fix: nextconfig

* fix: copilot review

* Remove bypassable regex-based XSS sanitization from marketplace search (#5835)

* Initial plan

* Remove problematic regex-based XSS sanitization from search inputs

Co-authored-by: c121914yu <50446880+c121914yu@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: c121914yu <50446880+c121914yu@users.noreply.github.com>

* feat: tool tag openapi

* api check

* fix: tsc

* fix: ts

* fix: lock

* sdk version

* ts

* sdk version

* remove invalid tip

* perf: export data add timezone

* perf: admin plugin api move

* perf: tool code

* move tag code

* perf: marketplace and team plugin code

* remove workflow invalid request

* rename global tool code

* rename global tool code

* rename api

* fix some bugs (#5841)

* fix some bugs

* fix

* perf: Tag filter

* fix: ts

* fix: ts

---------

Co-authored-by: archer <545436317@qq.com>

* perf: Concat function

* fix: workflow snapshot push

* fix: ts type

* fix: login to config/*

* fix: ts

* fix: model avatar (#5848)

* fix: model avatar

* fix: ts

* fix: avatar migration to s3

* update lock

* fix: avatar redirect

---------

Co-authored-by: archer <545436317@qq.com>

* fix tool detail (#5847)

* fix tool detail

* init script

* fix build

* perf: plugin detail modal

* change tooltags to tags

* fix icon

---------

Co-authored-by: archer <545436317@qq.com>

* fix tag filter scroll (#5852)

* fix create app plugin & import info (#5853)

* tag size

* rename toolkit

* download url

* import plugin status (#5854)

* init doc

* fix: init shell

---------

Co-authored-by: 伍闲犬 <whoeverimf5@gmail.com>
Co-authored-by: Zeng Qingwen <143274079+fishwww-ww@users.noreply.github.com>
Co-authored-by: heheer <heheer@sealos.io>
Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com>
Co-authored-by: heheer <zhiyu44@qq.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
This commit is contained in:
Archer 2025-11-04 16:58:12 +08:00 committed by GitHub
parent fac170306e
commit a499d05a02
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
364 changed files with 15051 additions and 3514 deletions

View File

@ -118,4 +118,9 @@ FastGPT 是一个 AI Agent 构建平台,通过 Flow 提供开箱即用的数据
## 代码规范
- 尽可能使用 type 进行类型声明,而不是 interface。
- 尽可能使用 type 进行类型声明,而不是 interface。
## Agent 设计规范
1. 对于功能的实习和复杂问题修复,优先进行文档设计,并于让用户确认后,再进行执行修复。
2. 采用"设计文档-测试示例-代码编写-测试运行-修正代码/文档"的工作模式,以测试为核心来确保设计的正确性。

View File

@ -80,7 +80,7 @@ addActiveNode(nodeId: string) {
---
### 🔴 H2. MongoDB 连接池配置缺失
### 🔴 H2. MongoDB 连接池配置缺失(已解决)
**位置**:
- `packages/service/common/mongo/index.ts:12-24`
@ -147,7 +147,7 @@ connectionMongo.connection.on('connectionPoolClosed', () => {
---
### 🔴 H3. SSE 流式响应未处理客户端断开
### 🔴 H3. SSE 流式响应未处理客户端断开(已解决)
**位置**: `packages/service/core/workflow/dispatch/index.ts:105-129`

View File

@ -23,4 +23,6 @@ vitest.config.mts
bin/
scripts/
deploy/
document/
document/
projects/marketplace

139
.github/workflows/marketplace-image.yml vendored Normal file
View File

@ -0,0 +1,139 @@
name: Build Marketplace images
on:
workflow_dispatch:
jobs:
build-marketplace-images:
permissions:
packages: write
contents: read
attestations: write
id-token: write
strategy:
matrix:
archs:
- arch: amd64
- arch: arm64
runs-on: ubuntu-24.04-arm
runs-on: ${{ matrix.archs.runs-on || 'ubuntu-24.04' }}
steps:
# install env
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver-opts: network=host
- name: Cache Docker layers
uses: actions/cache@v4
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-${{ matrix.archs.arch }}-marketplace-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-${{ matrix.archs.arch }}-marketplace-buildx-
# login docker
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Login to Ali Hub
uses: docker/login-action@v3
with:
registry: registry.cn-hangzhou.aliyuncs.com
username: ${{ secrets.ALI_HUB_USERNAME }}
password: ${{ secrets.ALI_HUB_PASSWORD }}
- name: Build for ${{ matrix.archs.arch }}
id: build
uses: docker/build-push-action@v6
with:
context: .
file: projects/marketplace/Dockerfile
platforms: linux/${{ matrix.archs.arch }}
labels: |
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.description=marketplace image
outputs: type=image,"name=ghcr.io/${{ github.repository_owner }}/marketplace,${{ secrets.ALI_IMAGE_NAME }}/marketplace",push-by-digest=true,push=true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
- name: Export digest
run: |
mkdir -p ${{ runner.temp }}/digests/marketplace
digest="${{ steps.build.outputs.digest }}"
touch "${{ runner.temp }}/digests/marketplace/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-marketplace-${{ github.sha }}-${{ matrix.archs.arch }}
path: ${{ runner.temp }}/digests/marketplace/*
if-no-files-found: error
retention-days: 1
release-marketplace-images:
permissions:
packages: write
contents: read
attestations: write
id-token: write
needs: build-marketplace-images
runs-on: ubuntu-24.04
steps:
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Login to Ali Hub
uses: docker/login-action@v3
with:
registry: registry.cn-hangzhou.aliyuncs.com
username: ${{ secrets.ALI_HUB_USERNAME }}
password: ${{ secrets.ALI_HUB_PASSWORD }}
- name: Download digests
uses: actions/download-artifact@v4
with:
path: ${{ runner.temp }}/digests
pattern: digests-marketplace-${{ github.sha }}-*
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Generate random tag
id: tag
run: |
# Generate random hash tag (8 characters)
TAG=$(echo $RANDOM | md5sum | head -c 8)
echo "RANDOM_TAG=$TAG" >> $GITHUB_ENV
echo "Generated tag: $TAG"
- name: Set image name and tag
run: |
echo "Git_IMAGE=ghcr.io/${{ github.repository_owner }}/marketplace:${{ env.RANDOM_TAG }}" >> $GITHUB_ENV
echo "Ali_IMAGE=${{ secrets.ALI_IMAGE_NAME }}/marketplace:${{ env.RANDOM_TAG }}" >> $GITHUB_ENV
- name: Create manifest list and push
working-directory: ${{ runner.temp }}/digests
run: |
echo "Pushing image with tag: ${{ env.RANDOM_TAG }}"
TAGS="$(echo -e "${Git_Tag}\n${Ali_Tag}")"
for TAG in $TAGS; do
docker buildx imagetools create -t $TAG \
$(printf 'ghcr.io/${{ github.repository_owner }}/marketplace@sha256:%s ' *)
sleep 5
done
echo "✅ Successfully pushed images:"
echo " - ${{ env.Git_IMAGE }}"
echo " - ${{ env.Ali_IMAGE }}"

View File

@ -32,7 +32,7 @@ description: FastGPT 系统插件设计方案
1. 使用 ts-rest 作为 RPC 框架进行交互,提供 sdk 供 FastGPT 主项目调用
2. 使用 zod 进行类型验证
3. 用 bun 进行编译,每个工具编译为单一的 `.js` 文件,支持热插拔。
3. 用 bun 进行编译,每个工具编译为单一的 `.pkg` 文件,支持热插拔。
## 项目结构
@ -48,7 +48,8 @@ description: FastGPT 系统插件设计方案
- **model** 模型预设
- **scripts** 脚本(编译、创建新工具)
- **sdk**: SDK 定义,供外部调用,发布到了 npm
- **src**: 运行时express 服务
- **runtime**: 运行时express 服务
- **lib**: 库文件,提供工具函数和类库
- **test**: 测试相关
系统工具的结构可以参考 [如何开发系统工具](/docs/introduction/guide/plugins/dev_system_tool)。
@ -78,7 +79,7 @@ zod 可以实现在运行时的类型校验,也可以提供更高级的功能
### 使用 bun 进行打包
将插件 bundle 为一个单一的 `.js` 文件是一个重要的设计。这样可以将插件发布出来直接通过网络挂载等的形式使用。
将插件 bundle 为一个单一的 `.pkg` 文件是一个重要的设计。这样可以将插件发布出来直接通过网络挂载等的形式使用。
## 未来规划

View File

@ -5,7 +5,9 @@ description: FastGPT 系统工具开发指南
## 介绍
FastGPT 系统工具项目从 4.10.0 版本后移动到独立的`fastgpt-plugin`项目中,采用纯代码的模式进行工具编写。你可以在`fastgpt-plugin`项目中进行独立开发和调试好插件后,直接向 FastGPT 官方提交 PR 即可,无需运行 FastGPT 主服务。
FastGPT 系统工具项目从 4.10.0 版本后移动到独立的`fastgpt-plugin`项目中,采用纯代码的模式进行工具编写。
在 4.14.0 版本插件市场更新后,系统工具开发流程有所改变,请依照最新文档贡献代码。
你可以在`fastgpt-plugin`项目中进行独立开发和调试好插件后,直接向 FastGPT 官方提交 PR 即可,无需运行 FastGPT 主服务。
## 概念
@ -14,29 +16,74 @@ FastGPT 系统工具项目从 4.10.0 版本后移动到独立的`fastgpt-plugin`
在`fastgpt-plugin`中,你可以每次创建一个工具/工具集,每次提交时,仅接收一个工具/工具集。如需开发多个,可以创建多个 PR 进行提交。
## 1. 准备工作
## 1. 准备开发环境
- Fork [fastgpt-plugin 项目](https://github.com/labring/fastgpt-plugin)
- 安装 [Bun](https://bun.sh/)
- 部署一套 Minio也可以直接使用 FastGPT 的 `docker-compose.yml` 中的 Minio。
- 本地 clone 项目 `git clone git@github.com:[your-github-username]/fastgpt-plugin.git`
- 拷贝示例环境变量文件,并修改连接到开发环境的 Minio `cp .env.example .env.local`
- 安装依赖 `bun install`
- 运行开发环境 `bun run dev`
### 1.1 安装 Bun
在 dev 环境下Bun 将监听修改并热更新。
- 安装 [Bun](https://bun.sh/), FastGPT-plugin 使用 Bun 作为包管理器
## 2. 初始化一个新的工具/工具集
### 1.2 Fork FastGPT-plugin 仓库
### 2.1 执行创建命令
Fork 本仓库 `https://github.com/labring/fastgpt-plugin`
### 1.3 搭建开发脚手架
<Tabs items={['通过 Bunx 一键搭建','手动搭建']}>
<Tab value="通过 Bunx 一键搭建">
注意:由于使用了 bun 特有的 API必须使用 bunx 进行安装,使用 npx/npx 等会报错
创建一个新的目录,在该目录下执行:
```bash
bunx @fastgpt-sdk/plugin-cli
```
上述命令会在当前目录下创建 fastgpt-plugin 目录,并且添加两个 remote:
- upstream 指向官方仓库
- origin 指向你自己的仓库
默认使用 sparse-checkout 避免拉取所有的官方插件代码
</Tab>
<Tab value="手动搭建">
- 本地在一个新建目录下初始化一个 `git`:
```bash
git init
```
如果配置了 Git SSH Key, 则可以:
```bash
git remote add origin git@github.com:[your-name]/fastgpt-plugin.git
git remote add upstream git@github.com:labring/fastgpt-plugin.git
```
否则使用 https:
```bash
git remote add origin https://github.com/[your-name]/fastgpt-plugin.git
git remote add upstream https://github.com/labring/fastgpt-plugin.git
```
- (可选)使用稀疏检出 (Sparse-checkout) 以避免拉取所有插件代码,如果不进行稀疏检出,则会拉取所有官方插件
```bash
git sparse-checkout init --no-cone
git sparse-checkout add "/*" "!/modules/tool/packages/*"
git pull
```
使用命令创建新工具
```bash
bun i
bun run new:tool
```
依据提示分别选择创建工具/工具集,以及目录名(使用 camelCase 小驼峰法命名)。
</Tab>
</Tabs>
执行完后,系统会在 `modules/tool/packages/[your-tool-name]` 下生成一个工具/工具集的目录。
## 2. 编写工具代码
### 2.1 工具代码结构
依据提示分别选择创建工具/工具集,以及目录名(使用 camelCase 小驼峰法命名)。
系统工具 (Tool) 文件结构如下:
@ -48,6 +95,8 @@ test // 测试样例
config.ts // 配置,配置工具的名称、描述、类型、图标等
index.ts // 入口,不要改这个文件
logo.svg // Logo替换成你的工具的 Logo
README.md // 可选README 文件,用于展示工具的使用说明和示例
assets/ // 可选assets 目录,用于存放工具的资源文件,如图片、音频等
package.json // npm 包
```
@ -55,19 +104,21 @@ package.json // npm 包
```plaintext
children
└── tool // 这个里面的结构就和上面的 tool 基本一致
└── tool // 这个里面的结构就和上面的 tool 一致,但是没有 README 和 assets 目录
config.ts
index.ts
logo.svg
README.md
assets/
package.json
```
### 2.2 修改 config.ts
- **name** 和 **description** 字段为中文和英文两种语言
- **courseUrl** 密钥获取链接,或官网链接,教程链接等
- **courseUrl**(可选) 密钥获取链接,或官网链接,教程链接等,如果提供 README.md则可以写到 README 里面
- **author** 开发者名
- **type** 为枚举类型,目前有:
- **tags** 工具默认的标签,有如下可选标签(枚举类型)
- tools: 工具
- search: 搜索
- multimodal: 多模态
@ -204,7 +255,7 @@ dalle3 的 outputs 参数格式如下:
}
```
## 2. 编写处理逻辑
### 2.3 编写处理逻辑
在 `[your-tool-name]/src/index.ts` 为入口编写处理逻辑,需要注意:
@ -234,15 +285,51 @@ export async function tool(props: z.infer<typeof InputType>): Promise<z.infer<ty
上述例子给出了一个传入 formatStr (格式化字符串)并且返回当前时间的简单样例,如需安装包,可以在`/modules/tools/packages/[your-tool-name]`路径下,使用`bun install PACKAGE` 进行安装。
## 3. 调试
## 4. 构建/打包
### 单测
FastGPT v4.14.0 后,打包方式变为系统插件打包为一个 `.pkg` 文件,使用命令:
```bash
bun run build:pkg
```
将本地所有插件构建打包为 `.pkg` 文件,构建目录为 `dist/pkgs`
在 `test/index.test.ts` 中编写测试样例,使用 `bun run test index.test.ts完整路径` 即可运行测试。
## 5. 单元测试
### 从 Scalar 进行测试
FastGPT-plugin 使用 Vitest 作为单测框架。
浏览器打开`localhost:3000/openapi`可进入`fastgpt-plugin`的 OpenAPI 页面,进行 API 调试。
### 5.1 编写单测样例
在 `test/index.test.ts` 中编写测试样例,使用 `bun run test index.test.ts 完整路径` 即可运行测试。
> 注意:不要把你的 secret 密钥等写到测试样例中
>
> 使用 Agent 工具编写测试样例时,可能 Agent 工具会修改您的处理逻辑甚至修改整个测试框架的逻辑。
### 5.2 查看测试样例覆盖率coverage)
浏览器打开 coverage/index.html 可以插件各个模块的覆盖率
提交插件给官方仓库,必须编写单元测试样例,并且达到:
- 90% 以上代码覆盖率
- 100% 函数覆盖率
- 100% 分支条件覆盖率
## 6. E2E (端到端)测试
对于简单的工具,可能并不需要进行 E2E 测试,而如果工具过于复杂,官方人员可能会要求您完成 E2E 测试。
### 6.1 部署 E2E 测试环境
1. 参考 [快速开始本地开发](/docs/introduction/development/intro),在本地部署一套 FastGPT 开发环境
2. `cd runtime && cp .env.template .env.local` 复制环境变量样例文件,连接到上一步部署的 Minio, Mongo, Redis 中
3. `bun run dev` 运行开发环境,修改 FastGPT 的环境变量,连接到你刚刚启动的 fastgpt-plugin
### 6.2 从 Scalar 进行测试
运行 fastgpt-plugin 开发环境
浏览器打开`http://localhost:PORT/openapi`可进入`fastgpt-plugin`的 OpenAPI 页面,进行 API 调试。
PORT 为你的 fastgpt-plugin 的端口
![](/imgs/plugin-openapi.png)
@ -250,14 +337,18 @@ export async function tool(props: z.infer<typeof InputType>): Promise<z.infer<ty
![](/imgs/plugin-openapi2.png)
### 从 FastGPT 主服务进行测试
### 6.3 在开发环境下 e2e 测试(有热更新)
如果本地运行有`FastGPT`主服务,则可以直接添加对应的工具进行测试。
默认情况下fastgpt-plugin 会自动加载在 modules/tool/packages/ 下的所有工具,并自动监听文件修改并进行热更新。
可以在 FastGPT 中使用这些工具
### 可视化调试TODO
### 6.4 在开发环境下上传工具进行 e2e 测试(没有热更新
## 4. 提交工具至官方目录
设置 FastGPT-plugin 的环境变量 `DISABLE_DEV_TOOLS=true` 会禁用自动加载开发环境下的工具,此时可以测试工具的上传。
完毕上述所有内容后,向官方仓库 `https://github.com/labring/fastgpt-plugin` 提交 PR。官方人员审核通过后即可收录为 FastGPT 的官方插件。
## 7. 提交工具至官方目录
如无需官方收录,可自行对该项目进行 Docker 打包,并替换官方镜像即可。
完毕上述所有内容后,向官方仓库 `https://github.com/labring/fastgpt-plugin` 提交 PR。
官方人员审核通过后即可收录为 FastGPT 的官方插件。
如无需官方收录,则可以参考 [上传系统工具](upload_system_tool) 在自己部署的 FastGPT 中使用。

View File

@ -3,42 +3,38 @@ title: 如何在线上传系统工具
description: FastGPT 系统工具在线上传指南
---
> 从 FastGPT 4.14.0 版本开始,系统管理员可以通过 Web 界面直接上传和更新系统工具,无需重新部署服务
> 从 FastGPT 4.14.0 版本开始,系统管理员可以通过 Web 界面直接上传和更新系统工具进行热更新
## 权限要求
⚠️ **重要提示**:只有 **root 用户** 才能使用在线上传系统工具功能。
- 确保您已使用 `root` 账户登录 FastGPT
- 普通用户无法看到"导入/更新"按钮和删除功能
## 支持的文件格式
- **文件类型**`.js` 文件
- **文件大小**:最大 10MB
- **文件数量**:每次只能上传一个文件
- **文件类型**`.pkg` 文件
- **文件大小**:最大 100 MB
- **文件数量**:每次最多上传 15 个文件
## 上传步骤
### 1. 进入系统工具页面
### 1. 进入配置页面
1. 登录 FastGPT 管理后台
2. 导航到:**工作台** → **系统工具**
3. 确认页面右上角显示"导入/更新"按钮(只有 root 用户可见)
![](/imgs/plugins/entry.png)
### 2. 准备工具文件
在上传之前,请确保您的 `.js` 文件是从 fastgpt-plugin 项目中通过 `bun run build` 命令打包后的 dist/tools/built-in 文件夹下得到的
在上传之前,请确保您的 `.pkg` 文件是从 fastgpt-plugin 项目中通过 `bun run build:pkg` 命令打包后的 `dist/pkgs` 文件夹下得到的
![](/imgs/plugins/file.png)
![](/imgs/plugins/files.png)
### 3. 执行上传
1. 点击 **"导入/更新"** 按钮
2. 在弹出的对话框中,点击文件选择区域
3. 选择您准备好的 `.js` 工具文件
3. 选择您准备好的 `.pkg` 工具文件
4. 确认文件信息无误后,点击 **"确认导入"**
### 4. 上传过程
@ -54,70 +50,10 @@ description: FastGPT 系统工具在线上传指南
- **上传工具**:仅 root 用户可以上传新工具或更新现有工具
- **删除工具**:仅 root 用户可以删除已上传的工具
### 工具类型识别
系统会根据工具的配置自动识别工具类型:
- 🔧 **工具 (tools)**
- 🔍 **搜索 (search)**
- 🎨 **多模态 (multimodal)**
- 💬 **通讯 (communication)**
- 📦 **其他 (other)**
## 常见问题
### Q: 上传失败,提示"文件内容存在错误"
**可能原因:**
- fastgpt-plugin 项目不是最新的,导致打包的 `.js` 文件缺少正确的内容
- 工具配置格式不正确
**解决方案:**
1. 拉取最新的 fastgpt-plugin 项目重新进行 `bun run build` 获得打包后的 `.js` 文件
2. 检查本地插件运行是否成功
### Q: 无法看到"导入/更新"按钮
**原因:** 当前用户不是 root 用户
**解决方案:** 使用 root 账户重新登录
### Q: 文件上传超时
**可能原因:**
- 文件过大(超过 10MB
- 网络连接不稳定
**解决方案:**
1. 确认文件大小在限制范围内
2. 检查网络连接
3. 尝试重新上传
## 最佳实践
### 上传前检查
1. **代码测试**:在本地环境测试工具功能
2. **格式验证**:确保符合 FastGPT 工具规范
3. **文件大小**:保持文件在合理大小范围内
### 版本管理
- 建议为工具添加版本号注释
- 更新工具时,先备份原有版本
- 记录更新日志和功能变更
### 安全考虑
- 仅上传来源可信的工具文件
- 避免包含敏感信息或凭据
- 定期审查已安装的工具
### 存储方式
- 工具文件存储在 MinIO 中
- 工具元数据保存在 MongoDB 中
---
通过在线上传功能,您可以快速部署和管理系统工具,提高 FastGPT 的扩展性和灵活性。如遇到问题,请参考上述常见问题或联系技术支持。

View File

@ -110,6 +110,7 @@ description: FastGPT 文档目录
- [/docs/upgrading/4-13/4130](/docs/upgrading/4-13/4130)
- [/docs/upgrading/4-13/4131](/docs/upgrading/4-13/4131)
- [/docs/upgrading/4-13/4132](/docs/upgrading/4-13/4132)
- [/docs/upgrading/4-14/4140](/docs/upgrading/4-14/4140)
- [/docs/upgrading/4-8/40](/docs/upgrading/4-8/40)
- [/docs/upgrading/4-8/41](/docs/upgrading/4-8/41)
- [/docs/upgrading/4-8/42](/docs/upgrading/4-8/42)

View File

@ -0,0 +1,62 @@
---
title: 'V4.14.0(进行中)'
description: 'FastGPT V4.14.0 更新说明'
---
## 更新指南
### 1. 更新镜像:
- 更新 FastGPT 镜像tag: v4.14.0
- 更新 FastGPT 商业版镜像tag: v4.14.0
- 更新 fastgpt-plugin 镜像 tag: v0.3.0
- mcp_server 无需更新
- Sandbox 无需更新
- AIProxy 无需更新
### 2. 执行升级脚本
仅需使用过自定义系统工具的商业版用户操作。
从任意终端,发起 1 个 HTTP 请求。其中 `{{rootkey}}` 替换成环境变量里的 `rootkey``{{host}}` 替换成**FastGPT 域名**。
```bash
curl --location --request POST 'https://{{host}}/api/admin/initv4140' \
--header 'rootkey: {{rootkey}}' \
--header 'Content-Type: application/json'
```
会将原系统工具迁移到最新数据表中。
### 3. 安装系统插件至系统
* 原先手动安装的 js 插件包将会失效,需重新打包安装。
* 目前插件里仅包含工具,后续将增加触发器,文档解析器,数据分块策略,索引增强策略等。
* 系统安装完插件后对于多租户的系统团队管理员可以在插件库中激活对应工具从而在应用中使用。对于开源版root 团队会默认激活所有系统工具。
从 V4.14.0 版本开始fastgpt-plugin 镜像仅提供运行环境,不再预装系统插件,所有 FastGPT 系统需手动安装系统插件。可以通过公开的 FastGPT Marketplace 进行在线安装,或下载 .pkg 文件进行安装。
除了安装外,还可对工具进行排序、默认安装、标签管理等。
![alt text](/imgs/image-121.png)
## 🚀 新增内容
1. 增加插件市场,同时移除自定义工具分类,仅支持自定义标签。本期支持系统工具,可以从 FastGPT Marketplace 统一安装系统工具。后续将支持更多插件类型:工作流触发器,数据源解析方式,数据分块,索引增强策略等。
2. 对话框上传文件移动存储至 S3并且不会自动过期完全跟随对话记录删除。安全性更高签发预览连接仅 1 小时生效,而不是长期。
3. 全局变量支持时间点/时间范围/对话模型选择类型。
4. 插件输入支持密码类型。
## ⚙️ 优化
1. 匹配 Markdown 中 Base64 图片正则性能。
## 🐛 修复
1. Claude 工具调用,如果下标从 1 开始会导致参数异常。
2. S3 删除头像,如果 key 为空时,会抛错,导致流程阻塞。
3. 工作流前置IO 变更时,依赖未及时刷新。
4. 导出对话日志,缺少反馈记录。
5. 工作流欢迎语输入框输入时,光标会偏移到最后一位。
6. 存在交互节点和连续批量执行时,会导致工作流运行逻辑错误。
7. 工作流 Redo 操作后,编辑记录无法再继续推送快照。

View File

@ -4,6 +4,8 @@
"description": "FastGPT 版本更新介绍及升级操作",
"pages": [
"index",
"---4.14.x---",
"...4-14",
"---4.13.x---",
"...4-13",
"---4.12.x---",

View File

@ -19,7 +19,7 @@
"document/content/docs/introduction/development/custom-models/ollama.mdx": "2025-08-05T23:20:39+08:00",
"document/content/docs/introduction/development/custom-models/xinference.mdx": "2025-08-05T23:20:39+08:00",
"document/content/docs/introduction/development/design/dataset.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/development/design/design_plugin.mdx": "2025-08-20T19:00:48+08:00",
"document/content/docs/introduction/development/design/design_plugin.mdx": "2025-10-30T22:14:07+08:00",
"document/content/docs/introduction/development/docker.mdx": "2025-09-29T11:34:11+08:00",
"document/content/docs/introduction/development/faq.mdx": "2025-08-12T22:22:18+08:00",
"document/content/docs/introduction/development/intro.mdx": "2025-09-29T11:34:11+08:00",
@ -38,7 +38,7 @@
"document/content/docs/introduction/development/proxy/cloudflare.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/development/proxy/http_proxy.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/development/proxy/nginx.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/development/quick-start.mdx": "2025-09-29T11:34:11+08:00",
"document/content/docs/introduction/development/quick-start.mdx": "2025-10-21T11:58:25+08:00",
"document/content/docs/introduction/development/sealos.mdx": "2025-09-29T11:52:39+08:00",
"document/content/docs/introduction/development/signoz.mdx": "2025-09-17T22:29:56+08:00",
"document/content/docs/introduction/guide/DialogBoxes/htmlRendering.mdx": "2025-07-23T21:35:03+08:00",
@ -84,11 +84,11 @@
"document/content/docs/introduction/guide/knowledge_base/websync.mdx": "2025-08-05T23:20:39+08:00",
"document/content/docs/introduction/guide/knowledge_base/yuque_dataset.mdx": "2025-09-17T22:29:56+08:00",
"document/content/docs/introduction/guide/plugins/bing_search_plugin.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/guide/plugins/dev_system_tool.mdx": "2025-08-20T19:00:48+08:00",
"document/content/docs/introduction/guide/plugins/dev_system_tool.mdx": "2025-10-30T22:14:07+08:00",
"document/content/docs/introduction/guide/plugins/doc2x_plugin_guide.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/guide/plugins/google_search_plugin_guide.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/guide/plugins/searxng_plugin_guide.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/guide/plugins/upload_system_tool.mdx": "2025-09-24T22:40:31+08:00",
"document/content/docs/introduction/guide/plugins/upload_system_tool.mdx": "2025-10-30T22:14:07+08:00",
"document/content/docs/introduction/guide/team_permissions/invitation_link.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/guide/team_permissions/team_roles_permissions.mdx": "2025-07-23T21:35:03+08:00",
"document/content/docs/introduction/index.en.mdx": "2025-07-23T21:35:03+08:00",
@ -101,7 +101,7 @@
"document/content/docs/protocol/terms.en.mdx": "2025-08-03T22:37:45+08:00",
"document/content/docs/protocol/terms.mdx": "2025-08-03T22:37:45+08:00",
"document/content/docs/toc.en.mdx": "2025-08-04T13:42:36+08:00",
"document/content/docs/toc.mdx": "2025-10-09T15:10:19+08:00",
"document/content/docs/toc.mdx": "2025-10-23T19:11:11+08:00",
"document/content/docs/upgrading/4-10/4100.mdx": "2025-08-02T19:38:37+08:00",
"document/content/docs/upgrading/4-10/4101.mdx": "2025-09-08T20:07:20+08:00",
"document/content/docs/upgrading/4-11/4110.mdx": "2025-08-05T23:20:39+08:00",
@ -113,7 +113,8 @@
"document/content/docs/upgrading/4-12/4124.mdx": "2025-09-17T22:29:56+08:00",
"document/content/docs/upgrading/4-13/4130.mdx": "2025-09-30T16:00:10+08:00",
"document/content/docs/upgrading/4-13/4131.mdx": "2025-09-30T15:47:06+08:00",
"document/content/docs/upgrading/4-13/4132.mdx": "2025-10-21T11:32:22+08:00",
"document/content/docs/upgrading/4-13/4132.mdx": "2025-10-21T11:46:53+08:00",
"document/content/docs/upgrading/4-14/4140.mdx": "2025-11-03T12:13:10+08:00",
"document/content/docs/upgrading/4-8/40.mdx": "2025-08-02T19:38:37+08:00",
"document/content/docs/upgrading/4-8/41.mdx": "2025-08-02T19:38:37+08:00",
"document/content/docs/upgrading/4-8/42.mdx": "2025-08-02T19:38:37+08:00",

Binary file not shown.

After

Width:  |  Height:  |  Size: 550 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 513 KiB

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 155 KiB

View File

@ -0,0 +1,65 @@
import { formatFileSize } from '../file/tools';
/**
* Parse S3 upload error and return user-friendly error message key
* @param error - The error from S3 upload
* @param maxFileSize - Maximum allowed file size in bytes
* @returns i18n error message key and parameters
*/
export function parseS3UploadError({
t,
error,
maxSize
}: {
t: any;
error: any;
maxSize?: number;
}): string {
const maxSizeStr = maxSize ? formatFileSize(maxSize) : '-';
// Handle S3 XML error response
if (typeof error === 'string' && error.includes('EntityTooLarge')) {
return t('common:error:s3_upload_file_too_large', { max: maxSizeStr });
}
// Handle axios error response
if (error?.response?.data) {
const data = error.response.data;
// Try to parse XML error response
if (typeof data === 'string') {
if (data.includes('EntityTooLarge')) {
return t('common:error:s3_upload_file_too_large', { max: maxSizeStr });
}
if (data.includes('AccessDenied')) {
return t('common:error:s3_upload_auth_failed');
}
if (data.includes('InvalidAccessKeyId') || data.includes('SignatureDoesNotMatch')) {
return t('common:error:s3_upload_auth_failed');
}
if (data.includes('NoSuchBucket')) {
return t('common:error:s3_upload_bucket_not_found');
}
if (data.includes('RequestTimeout')) {
return t('common:error:s3_upload_timeout');
}
}
}
// Handle network errors
if (error?.code === 'ECONNREFUSED' || error?.code === 'ETIMEDOUT') {
return t('common:error:s3_upload_network_error');
}
// Handle axios timeout
if (error?.code === 'ECONNABORTED' || error?.message?.includes('timeout')) {
return t('common:error:s3_upload_timeout');
}
// Handle file size validation error (client-side)
if (error?.message?.includes('file size') || error?.message?.includes('too large')) {
return t('common:error:s3_upload_file_too_large', { max: maxSizeStr });
}
// Default error
return t('common:error:s3_upload_network_error');
}

View File

@ -1,19 +1,19 @@
export const fileImgs = [
{ suffix: 'pdf', src: 'file/fill/pdf' },
{ suffix: 'ppt', src: 'file/fill/ppt' },
{ suffix: 'xlsx', src: 'file/fill/xlsx' },
{ suffix: 'csv', src: 'file/fill/csv' },
{ suffix: '(doc|docs)', src: 'file/fill/doc' },
{ suffix: 'txt', src: 'file/fill/txt' },
{ suffix: 'md', src: 'file/fill/markdown' },
{ suffix: 'html', src: 'file/fill/html' },
{ suffix: '(jpg|jpeg|png|gif|bmp|webp|svg|ico|tiff|tif)', src: 'image' }
export const getFileIcon = (name = '', defaultImg = 'file/fill/file') => {
const fileImgs = [
{ suffix: 'pdf', src: 'file/fill/pdf' },
{ suffix: 'ppt', src: 'file/fill/ppt' },
{ suffix: 'xlsx', src: 'file/fill/xlsx' },
{ suffix: 'csv', src: 'file/fill/csv' },
{ suffix: '(doc|docs)', src: 'file/fill/doc' },
{ suffix: 'txt', src: 'file/fill/txt' },
{ suffix: 'md', src: 'file/fill/markdown' },
{ suffix: 'html', src: 'file/fill/html' },
{ suffix: '(jpg|jpeg|png|gif|bmp|webp|svg|ico|tiff|tif)', src: 'image' },
{ suffix: '(mp3|wav|ogg|m4a|amr|mpga)', src: 'file/fill/audio' },
{ suffix: '(mp4|mov|avi|mpeg|webm)', src: 'file/fill/video' }
];
// { suffix: '.', src: '/imgs/files/file.svg' }
];
export function getFileIcon(name = '', defaultImg = 'file/fill/file') {
return (
fileImgs.find((item) => new RegExp(`\.${item.suffix}`, 'gi').test(name))?.src || defaultImg
);
}
};

View File

@ -1,7 +1,7 @@
import { detect } from 'jschardet';
import { documentFileType } from './constants';
import { ChatFileTypeEnum } from '../../core/chat/constants';
import { type UserChatItemValueItemType } from '../../core/chat/type';
import { type UserChatItemFileItemType } from '../../core/chat/type';
import * as fs from 'fs';
export const formatFileSize = (bytes: number): string => {
@ -36,7 +36,7 @@ export const detectFileEncodingByPath = async (path: string) => {
};
// Url => user upload file type
export const parseUrlToFileType = (url: string): UserChatItemValueItemType['file'] | undefined => {
export const parseUrlToFileType = (url: string): UserChatItemFileItemType | undefined => {
if (typeof url !== 'string') return;
// Handle base64 image
@ -74,13 +74,13 @@ export const parseUrlToFileType = (url: string): UserChatItemValueItemType['file
// Default to image type for non-document files
return {
type: ChatFileTypeEnum.image,
name: filename || 'null.png',
name: filename || 'null',
url
};
} catch (error) {
return {
type: ChatFileTypeEnum.image,
name: 'invalid.png',
type: ChatFileTypeEnum.file,
name: url,
url
};
}

View File

@ -2,5 +2,17 @@ import type { I18nStringType, localeType } from './type';
export const parseI18nString = (str: I18nStringType | string = '', lang = 'en') => {
if (!str || typeof str === 'string') return str;
return str[lang as localeType] ?? str['en'];
// 尝试使用当前语言
if (str[lang as localeType]) {
return str[lang as localeType] || '';
}
// 如果当前语言是繁体中文但没有对应翻译,优先回退到简体中文
if (lang === 'zh-Hant' && !str['zh-Hant'] && str['zh-CN']) {
return str['zh-CN'];
}
// 最后回退到英文
return str['en'] || '';
};

View File

@ -1,3 +1,8 @@
import z from 'zod';
export const ParentIdSchema = z.string().nullish();
export type ParentIdType = string | null | undefined;
export type GetPathProps = {
sourceId?: ParentIdType;
type: 'current' | 'parent';
@ -7,7 +12,6 @@ export type ParentTreePathItemType = {
parentId: string;
parentName: string;
};
export type ParentIdType = string | null | undefined;
export type GetResourceFolderListProps = {
parentId: ParentIdType;

View File

@ -173,18 +173,20 @@ export const markdownProcess = async ({
};
export const matchMdImg = (text: string) => {
const base64Regex = /!\[([^\]]*)\]\((data:image\/[^;]+;base64[^)]+)\)/g;
// 优化后的正则:
// 1. 使用 [^\]]* 匹配 alt 文本(更精确)
// 2. 使用 [A-Za-z0-9+/=]+ 匹配 base64 数据(避免回溯)
// 3. 明确匹配 data:image/ 前缀
const base64Regex = /!\[([^\]]*)\]\((data:image\/([^;]+);base64,([A-Za-z0-9+/=]+))\)/g;
const imageList: ImageType[] = [];
text = text.replace(base64Regex, (match, altText, base64Url) => {
text = text.replace(base64Regex, (_match, altText, _fullDataUrl, mime, base64Data) => {
const uuid = `IMAGE_${getNanoid(12)}_IMAGE`;
const mime = base64Url.split(';')[0].split(':')[1];
const base64 = base64Url.split(',')[1];
imageList.push({
uuid,
base64,
mime
base64: base64Data,
mime: `image/${mime}`
});
// 保持原有的 alt 文本,只替换 base64 部分

View File

@ -82,9 +82,10 @@ const markdownTableSplit = (props: SplitProps): SplitResponse => {
.join(' | ')} |`;
const chunks: string[] = [];
let chunk = `${header}
const defaultChunk = `${header}
${mdSplitString}
`;
let chunk = defaultChunk;
for (let i = 2; i < splitText2Lines.length; i++) {
const chunkLength = getTextValidLength(chunk);
@ -93,9 +94,7 @@ ${mdSplitString}
// Over size
if (chunkLength + nextLineLength > chunkSize) {
chunks.push(chunk);
chunk = `${header}
${mdSplitString}
`;
chunk = defaultChunk;
}
chunk += `${splitText2Lines[i]}\n`;
}

View File

@ -79,7 +79,7 @@ export type FastGPTFeConfigsType = {
concatMd?: string;
docUrl?: string;
openAPIDocUrl?: string;
systemPluginCourseUrl?: string;
submitPluginRequestUrl?: string;
appTemplateCourse?: string;
customApiDomain?: string;
customSharePageDomain?: string;

View File

@ -73,8 +73,8 @@ export const getTimeZoneList = () => {
};
export const timeZoneList = getTimeZoneList();
export const getMongoTimezoneCode = (timeString: string) => {
if (!timeString.includes(':')) {
export const getTimezoneCodeFromStr = (timeString: string | Date) => {
if (typeof timeString !== 'string' || !timeString.includes(':')) {
return '+00:00';
}

View File

@ -1,4 +1,4 @@
import type { I18nStringStrictType } from '@fastgpt-sdk/plugin';
import type { I18nStringStrictType } from '../../sdk/fastgpt-plugin';
export type ModelProviderItemType = {
id: string;
@ -23,16 +23,18 @@ export const defaultProvider: ModelProviderItemType = {
order: 999
};
export const formatModelProviders = (data: { provider: string; value: I18nStringStrictType }[]) => {
export const formatModelProviders = (
data: { provider: string; value: I18nStringStrictType; avatar: string }[]
) => {
const getLocalizedName = (translations: I18nStringStrictType, language = 'en'): string => {
return translations[language as langType] || translations.en;
};
const formatModelProviderList = (language?: string): ModelProviderItemType[] => {
return data.map(({ provider, value }, index) => ({
return data.map(({ provider, value, avatar }, index) => ({
id: provider,
name: getLocalizedName(value, language),
avatar: `/api/system/plugin/models/${provider}.svg`,
avatar,
order: index
}));
};
@ -40,11 +42,11 @@ export const formatModelProviders = (data: { provider: string; value: I18nString
const formatModelProviderMap = (language?: string) => {
const provider = {} as Record<string, ModelProviderItemType>;
data.forEach(({ provider: id, value }, index) => {
data.forEach(({ provider: id, value, avatar }, index) => {
provider[id] = {
id,
name: getLocalizedName(value, language),
avatar: `/api/system/plugin/models/${id}.svg`,
avatar,
order: index
};
});

View File

@ -19,10 +19,11 @@ export type ChatCompletionContentPartFile = {
type: 'file_url';
name: string;
url: string;
key?: string;
};
// Rewrite ChatCompletionContentPart, Add file type
export type ChatCompletionContentPart =
| SdkChatCompletionContentPart
| (SdkChatCompletionContentPart & { key?: string })
| ChatCompletionContentPartFile;
type CustomChatCompletionUserMessageParam = Omit<ChatCompletionUserMessageParam, 'content'> & {
role: 'user';

View File

@ -50,7 +50,11 @@ export const defaultChatInputGuideConfig = {
export const defaultAppSelectFileConfig: AppFileSelectConfigType = {
canSelectFile: false,
canSelectImg: false,
maxFiles: 10
maxFiles: 10,
canSelectVideo: false,
canSelectAudio: false,
canSelectCustomFileExtension: false,
customFileExtensionList: []
};
export enum AppTemplateTypeEnum {
@ -64,3 +68,45 @@ export enum AppTemplateTypeEnum {
// special type
contribute = 'contribute'
}
export const defaultFileExtensionTypes = {
canSelectFile: ['.pdf', '.docx', '.pptx', '.xlsx', '.txt', '.md', '.html', '.csv'],
canSelectImg: ['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.webp', '.svg'],
canSelectVideo: ['.mp4', '.mov', '.avi', '.mpeg', '.webm'],
canSelectAudio: ['.mp3', '.wav', '.ogg', '.m4a', '.amr', '.mpga'],
canSelectCustomFileExtension: []
};
export type FileExtensionKeyType = keyof typeof defaultFileExtensionTypes;
export const getUploadFileType = ({
canSelectFile,
canSelectImg,
canSelectVideo,
canSelectAudio,
canSelectCustomFileExtension,
customFileExtensionList
}: {
canSelectFile?: boolean;
canSelectImg?: boolean;
canSelectVideo?: boolean;
canSelectAudio?: boolean;
canSelectCustomFileExtension?: boolean;
customFileExtensionList?: string[];
}) => {
const types: string[] = [];
if (canSelectFile) {
types.push(...defaultFileExtensionTypes.canSelectFile);
}
if (canSelectImg) {
types.push(...defaultFileExtensionTypes.canSelectImg);
}
if (canSelectVideo) {
types.push(...defaultFileExtensionTypes.canSelectVideo);
}
if (canSelectAudio) {
types.push(...defaultFileExtensionTypes.canSelectAudio);
}
if (canSelectCustomFileExtension && customFileExtensionList) {
types.push(...customFileExtensionList);
}
return types.join(', ');
};

View File

@ -4,7 +4,7 @@ import type { FlowNodeInputItemType, FlowNodeOutputItemType } from '../workflow/
import SwaggerParser from '@apidevtools/swagger-parser';
import yaml from 'js-yaml';
import type { OpenAPIV3 } from 'openapi-types';
import type { OpenApiJsonSchema } from './httpTools/type';
import type { OpenApiJsonSchema } from './tool/httpTool/type';
import { i18nT } from '../../../web/i18n/utils';
type SchemaInputValueType = 'string' | 'number' | 'integer' | 'boolean' | 'array' | 'object';

View File

@ -1,12 +0,0 @@
import type { McpToolConfigType } from '../type';
export type McpToolSetDataType = {
url: string;
headerSecret?: StoreSecretValueType;
toolList: McpToolConfigType[];
};
export type McpToolDataType = McpToolConfigType & {
url: string;
headerSecret?: StoreSecretValueType;
};

View File

@ -1,70 +0,0 @@
import { type StoreNodeItemType } from '../../workflow/type/node';
import { type FlowNodeInputItemType } from '../../workflow/type/io';
import { FlowNodeTypeEnum } from '../../workflow/node/constant';
import { PluginSourceEnum } from './constants';
export const getPluginInputsFromStoreNodes = (nodes: StoreNodeItemType[]) => {
return nodes.find((node) => node.flowNodeType === FlowNodeTypeEnum.pluginInput)?.inputs || [];
};
export const getPluginRunContent = ({
pluginInputs,
variables
}: {
pluginInputs: FlowNodeInputItemType[];
variables: Record<string, any>;
}) => {
const pluginInputsWithValue = pluginInputs.map((input) => {
const { key } = input;
const value = variables?.hasOwnProperty(key) ? variables[key] : input.defaultValue;
return {
...input,
value
};
});
return JSON.stringify(pluginInputsWithValue);
};
/**
plugin id rule:
- personal: ObjectId
- commercial: commercial-ObjectId
- systemtool: systemTool-id
- mcp tool: mcp-parentId/toolName
(deprecated) community: community-id
*/
export function splitCombinePluginId(id: string) {
const splitRes = id.split('-');
if (splitRes.length === 1) {
// app id
return {
source: PluginSourceEnum.personal,
pluginId: id
};
}
const [source, ...rest] = id.split('-') as [PluginSourceEnum, string | undefined];
const pluginId = rest.join('-');
if (!source || !pluginId) throw new Error('pluginId not found');
// 兼容4.10.0 之前的插件
if (source === 'community' || id === 'commercial-dalle3') {
return {
source: PluginSourceEnum.systemTool,
pluginId: `${PluginSourceEnum.systemTool}-${pluginId}`
};
}
if (source === 'mcp') {
return {
source: PluginSourceEnum.mcp,
pluginId
};
}
if (source === 'http') {
return {
source: PluginSourceEnum.http,
pluginId
};
}
return { source, pluginId: id };
}

View File

@ -1,18 +0,0 @@
import { i18nT } from '../../../../web/i18n/utils';
export enum SystemToolInputTypeEnum {
system = 'system',
team = 'team',
manual = 'manual'
}
export const SystemToolInputTypeMap = {
[SystemToolInputTypeEnum.system]: {
text: i18nT('common:System')
},
[SystemToolInputTypeEnum.team]: {
text: i18nT('common:Team')
},
[SystemToolInputTypeEnum.manual]: {
text: i18nT('common:Manual')
}
};

View File

@ -1,4 +1,4 @@
export enum PluginSourceEnum {
export enum AppToolSourceEnum {
personal = 'personal', // this is a app.
systemTool = 'systemTool', // FastGPT-plugin tools, pure code.
commercial = 'commercial', // configured in Pro, with associatedPluginId. Specially, commercial-dalle3 is a systemTool

View File

@ -1,14 +1,14 @@
import { getNanoid } from '../../../common/string/tools';
import { getNanoid } from '../../../../common/string/tools';
import type { PathDataType } from './type';
import { type RuntimeNodeItemType } from '../../workflow/runtime/type';
import { FlowNodeOutputTypeEnum, FlowNodeTypeEnum } from '../../workflow/node/constant';
import { type HttpToolConfigType } from '../type';
import { PluginSourceEnum } from '../plugin/constants';
import { jsonSchema2NodeInput, jsonSchema2NodeOutput } from '../jsonschema';
import { type StoreSecretValueType } from '../../../common/secret/type';
import { type JsonSchemaPropertiesItemType } from '../jsonschema';
import { NodeOutputKeyEnum, WorkflowIOValueTypeEnum } from '../../workflow/constants';
import { i18nT } from '../../../../web/i18n/utils';
import { type RuntimeNodeItemType } from '../../../workflow/runtime/type';
import { FlowNodeOutputTypeEnum, FlowNodeTypeEnum } from '../../../workflow/node/constant';
import { type HttpToolConfigType } from '../../type';
import { AppToolSourceEnum } from '../constants';
import { jsonSchema2NodeInput, jsonSchema2NodeOutput } from '../../jsonschema';
import { type StoreSecretValueType } from '../../../../common/secret/type';
import { type JsonSchemaPropertiesItemType } from '../../jsonschema';
import { NodeOutputKeyEnum, WorkflowIOValueTypeEnum } from '../../../workflow/constants';
import { i18nT } from '../../../../../web/i18n/utils';
export const getHTTPToolSetRuntimeNode = ({
name,
@ -66,7 +66,7 @@ export const getHTTPToolRuntimeNode = ({
intro: tool.description,
toolConfig: {
httpTool: {
toolId: `${PluginSourceEnum.http}-${parentId}/${tool.name}`
toolId: `${AppToolSourceEnum.http}-${parentId}/${tool.name}`
}
},
inputs: jsonSchema2NodeInput(tool.inputSchema),

View File

@ -0,0 +1,19 @@
import type { StoreSecretValueType } from '../../../../common/secret/type';
import type { JSONSchemaInputType } from '../../jsonschema';
export type McpToolConfigType = {
name: string;
description: string;
inputSchema: JSONSchemaInputType;
};
export type McpToolSetDataType = {
url: string;
headerSecret?: StoreSecretValueType;
toolList: McpToolConfigType[];
};
export type McpToolDataType = McpToolConfigType & {
url: string;
headerSecret?: StoreSecretValueType;
};

View File

@ -1,12 +1,12 @@
import { NodeOutputKeyEnum, WorkflowIOValueTypeEnum } from '../../workflow/constants';
import { i18nT } from '../../../../web/i18n/utils';
import { FlowNodeOutputTypeEnum, FlowNodeTypeEnum } from '../../workflow/node/constant';
import { type McpToolConfigType } from '../type';
import { type RuntimeNodeItemType } from '../../workflow/runtime/type';
import { type StoreSecretValueType } from '../../../common/secret/type';
import { jsonSchema2NodeInput } from '../jsonschema';
import { getNanoid } from '../../../common/string/tools';
import { PluginSourceEnum } from '../plugin/constants';
import { NodeOutputKeyEnum, WorkflowIOValueTypeEnum } from '../../../workflow/constants';
import { i18nT } from '../../../../../web/i18n/utils';
import { FlowNodeOutputTypeEnum, FlowNodeTypeEnum } from '../../../workflow/node/constant';
import { type McpToolConfigType } from '../../tool/mcpTool/type';
import { type RuntimeNodeItemType } from '../../../workflow/runtime/type';
import { type StoreSecretValueType } from '../../../../common/secret/type';
import { jsonSchema2NodeInput } from '../../jsonschema';
import { getNanoid } from '../../../../common/string/tools';
import { AppToolSourceEnum } from '../constants';
export const getMCPToolSetRuntimeNode = ({
url,
@ -59,7 +59,7 @@ export const getMCPToolRuntimeNode = ({
intro: tool.description,
toolConfig: {
mcpTool: {
toolId: `${PluginSourceEnum.mcp}-${parentId}/${tool.name}`
toolId: `${AppToolSourceEnum.mcp}-${parentId}/${tool.name}`
}
},
inputs: jsonSchema2NodeInput(tool.inputSchema),

View File

@ -0,0 +1,18 @@
import { i18nT } from '../../../../../web/i18n/utils';
export enum SystemToolSecretInputTypeEnum {
system = 'system',
team = 'team',
manual = 'manual'
}
export const SystemToolSecretInputTypeMap = {
[SystemToolSecretInputTypeEnum.system]: {
text: i18nT('common:System')
},
[SystemToolSecretInputTypeEnum.team]: {
text: i18nT('common:Team')
},
[SystemToolSecretInputTypeEnum.manual]: {
text: i18nT('common:Manual')
}
};

View File

@ -6,8 +6,12 @@ import type { FlowNodeTemplateType } from '../../workflow/type/node';
import type { WorkflowTemplateType } from '../../workflow/type';
import type { FlowNodeInputItemType, FlowNodeOutputItemType } from '../../workflow/type/io';
import type { ParentIdType } from 'common/parentFolder/type';
import type { I18nStringStrictType } from '../../../common/i18n/type';
import type { I18nStringType } from '../../../common/i18n/type';
import type { ToolSimpleType, ToolDetailType } from '../../../sdk/fastgpt-plugin';
import type { PluginStatusType, SystemPluginToolTagType } from '../../plugin/type';
export type PluginRuntimeType = {
export type AppToolRuntimeType = {
id: string;
teamId?: string;
tmbId?: string;
@ -23,10 +27,9 @@ export type PluginRuntimeType = {
hasTokenFee?: boolean;
};
// system plugin
export type SystemPluginTemplateItemType = WorkflowTemplateType & {
templateType: string;
// System tool
export type AppToolTemplateItemType = WorkflowTemplateType & {
status?: PluginStatusType;
// FastGPT-plugin tool
inputs?: FlowNodeInputItemType[];
outputs?: FlowNodeOutputItemType[];
@ -49,7 +52,8 @@ export type SystemPluginTemplateItemType = WorkflowTemplateType & {
hasTokenFee?: boolean;
pluginOrder?: number;
isActive?: boolean;
tags?: string[] | null;
defaultInstalled?: boolean;
isOfficial?: boolean;
// Admin config
@ -57,14 +61,16 @@ export type SystemPluginTemplateItemType = WorkflowTemplateType & {
inputListVal?: Record<string, any>;
hasSystemSecret?: boolean;
// Plugin source type
toolSource?: 'uploaded' | 'built-in';
// @deprecated use tags instead
isActive?: boolean;
templateType?: string;
};
export type SystemPluginTemplateListItemType = Omit<
SystemPluginTemplateItemType,
'name' | 'intro'
export type AppToolTemplateListItemType = Omit<
AppToolTemplateItemType,
'name' | 'intro' | 'workflow'
> & {
name: string;
intro: string;
tags?: SystemPluginToolTagType[];
};

View File

@ -0,0 +1,46 @@
import { AppToolSourceEnum } from '../tool/constants';
/**
Tool id rule:
- personal: ObjectId
- commercial: commercial-ObjectId
- systemtool: systemTool-id
- mcp tool: mcp-parentId/toolName
(deprecated) community: community-id
*/
export function splitCombineToolId(id: string) {
const splitRes = id.split('-');
if (splitRes.length === 1) {
// app id
return {
source: AppToolSourceEnum.personal,
pluginId: id
};
}
const [source, ...rest] = id.split('-') as [AppToolSourceEnum, string | undefined];
const pluginId = rest.join('-');
if (!source || !pluginId) throw new Error('pluginId not found');
// 兼容4.10.0 之前的插件
if (source === 'community' || id === 'commercial-dalle3') {
return {
source: AppToolSourceEnum.systemTool,
pluginId: `${AppToolSourceEnum.systemTool}-${pluginId}`
};
}
if (source === 'mcp') {
return {
source: AppToolSourceEnum.mcp,
pluginId
};
}
if (source === 'http') {
return {
source: AppToolSourceEnum.http,
pluginId
};
}
return { source, pluginId: id };
}

View File

@ -0,0 +1,6 @@
import { type StoreNodeItemType } from '../../../workflow/type/node';
import { FlowNodeTypeEnum } from '../../../workflow/node/constant';
export const getWorkflowToolInputsFromStoreNodes = (nodes: StoreNodeItemType[]) => {
return nodes.find((node) => node.flowNodeType === FlowNodeTypeEnum.pluginInput)?.inputs || [];
};

View File

@ -7,7 +7,7 @@ import type {
VariableInputEnum,
WorkflowIOValueTypeEnum
} from '../workflow/constants';
import type { SelectedDatasetType } from '../workflow/type/io';
import type { InputComponentPropsType, SelectedDatasetType } from '../workflow/type/io';
import type { DatasetSearchModeEnum } from '../dataset/constants';
import { TeamTagSchema as TeamTagsSchemaType } from '@fastgpt/global/support/user/team/type.d';
import type { StoreEdgeItemType } from '../workflow/type/edge';
@ -115,12 +115,6 @@ export type AppSimpleEditFormType = {
chatConfig: AppChatConfigType;
};
export type McpToolConfigType = {
name: string;
description: string;
inputSchema: JSONSchemaInputType;
};
export type HttpToolConfigType = {
name: string;
description: string;
@ -170,38 +164,11 @@ export type SettingAIDataType = {
};
// variable
export type VariableItemType = {
// id: string;
key: string;
label: string;
type: VariableInputEnum;
required: boolean;
description: string;
valueType?: WorkflowIOValueTypeEnum;
defaultValue?: any;
// input
maxLength?: number;
// password
minLength?: number;
// numberInput
max?: number;
min?: number;
// select
list?: { label: string; value: string }[];
// file
canSelectFile?: boolean;
canSelectImg?: boolean;
maxFiles?: number;
// timeSelect
timeGranularity?: 'second' | 'minute' | 'hour' | 'day';
timeType?: 'point' | 'range';
timeRangeStart?: string;
timeRangeEnd?: string;
// @deprecated
enums?: { value: string; label: string }[];
};
export type VariableItemType = AppFileSelectConfigType &
InputComponentPropsType & {
type: VariableInputEnum;
description: string;
};
// tts
export type AppTTSConfigType = {
type: 'none' | 'web' | 'model';
@ -241,16 +208,14 @@ export type AppAutoExecuteConfigType = {
};
// File
export type AppFileSelectConfigType = {
canSelectFile: boolean;
maxFiles?: number;
canSelectFile?: boolean;
customPdfParse?: boolean;
canSelectImg: boolean;
maxFiles: number;
};
export type SystemPluginListItemType = {
_id: string;
name: string;
avatar: string;
canSelectImg?: boolean;
canSelectVideo?: boolean;
canSelectAudio?: boolean;
canSelectCustomFileExtension?: boolean;
customFileExtensionList?: string[];
};
export type AppTemplateSchemaType = {

View File

@ -17,6 +17,7 @@ import type {
ChatCompletionToolMessageParam
} from '../../core/ai/type.d';
import { ChatCompletionRequestMessageRoleEnum } from '../../core/ai/constants';
const GPT2Chat = {
[ChatCompletionRequestMessageRoleEnum.System]: ChatRoleEnum.System,
[ChatCompletionRequestMessageRoleEnum.User]: ChatRoleEnum.Human,
@ -71,6 +72,7 @@ export const chats2GPTMessages = ({
if (item.file?.type === ChatFileTypeEnum.image) {
return {
type: 'image_url',
key: item.file.key,
image_url: {
url: item.file.url
}
@ -79,7 +81,8 @@ export const chats2GPTMessages = ({
return {
type: 'file_url',
name: item.file?.name || '',
url: item.file.url
url: item.file.url,
key: item.file.key
};
}
}
@ -171,6 +174,7 @@ export const chats2GPTMessages = ({
return results;
};
export const GPTMessages2Chats = ({
messages,
reserveTool = true,
@ -238,7 +242,8 @@ export const GPTMessages2Chats = ({
file: {
type: ChatFileTypeEnum.image,
name: '',
url: item.image_url.url
url: item.image_url.url,
key: item.key
}
});
} else if (item.type === 'file_url') {
@ -248,7 +253,8 @@ export const GPTMessages2Chats = ({
file: {
type: ChatFileTypeEnum.file,
name: item.name,
url: item.url
url: item.url,
key: item.key
}
});
}

View File

@ -1,9 +0,0 @@
import type { OutLinkChatAuthProps } from '../../support/permission/chat';
export type UpdateChatFeedbackProps = OutLinkChatAuthProps & {
appId: string;
chatId: string;
dataId: string;
userBadFeedback?: string;
userGoodFeedback?: string;
};

View File

@ -0,0 +1,68 @@
import type { OutLinkChatAuthType } from '../../support/permission/chat/type';
import { OutLinkChatAuthSchema } from '../../support/permission/chat/type';
import { ObjectIdSchema } from '../../common/type/mongo';
import z from 'zod';
export const PresignChatFileGetUrlSchema = z
.object({
key: z.string().min(1),
appId: ObjectIdSchema,
outLinkAuthData: OutLinkChatAuthSchema.optional()
})
.meta({
description: '获取对话文件预览链接',
example: {
key: '1234567890',
appId: '1234567890',
outLinkAuthData: {
shareId: '1234567890',
outLinkUid: '1234567890'
}
}
});
export type PresignChatFileGetUrlParams = z.infer<typeof PresignChatFileGetUrlSchema> & {
outLinkAuthData?: OutLinkChatAuthType;
};
export const PresignChatFilePostUrlSchema = z
.object({
filename: z.string().min(1),
appId: ObjectIdSchema,
chatId: ObjectIdSchema,
outLinkAuthData: OutLinkChatAuthSchema.optional()
})
.meta({
description: '获取上传对话文件预签名 URL',
example: {
filename: '1234567890',
appId: '1234567890',
chatId: '1234567890',
outLinkAuthData: {
shareId: '1234567890',
outLinkUid: '1234567890'
}
}
});
export type PresignChatFilePostUrlParams = z.infer<typeof PresignChatFilePostUrlSchema> & {
outLinkAuthData?: OutLinkChatAuthType;
};
export const UpdateChatFeedbackSchema = z
.object({
appId: z.string().min(1),
chatId: z.string().min(1),
dataId: z.string().min(1),
userBadFeedback: z.string().optional(),
userGoodFeedback: z.string().optional()
})
.meta({
description: '更新对话反馈',
example: {
appId: '1234567890',
chatId: '1234567890',
dataId: '1234567890',
userBadFeedback: '1234567890',
userGoodFeedback: '1234567890'
}
});
export type UpdateChatFeedbackProps = z.infer<typeof UpdateChatFeedbackSchema>;

View File

@ -51,16 +51,18 @@ export type ChatWithAppSchema = Omit<ChatSchemaType, 'appId'> & {
};
/* --------- chat item ---------- */
export type UserChatItemFileItemType = {
type: `${ChatFileTypeEnum}`;
name?: string;
key?: string;
url: string;
};
export type UserChatItemValueItemType = {
type: ChatItemValueTypeEnum.text | ChatItemValueTypeEnum.file;
text?: {
content: string;
};
file?: {
type: `${ChatFileTypeEnum}`;
name?: string;
url: string;
};
file?: UserChatItemFileItemType;
};
export type UserChatItemType = {
obj: ChatRoleEnum.Human;

View File

@ -171,7 +171,8 @@ export const removeEmptyUserInput = (input?: UserChatItemValueItemType[]) => {
if (item.type === ChatItemValueTypeEnum.text && !item.text?.content?.trim()) {
return false;
}
if (item.type === ChatItemValueTypeEnum.file && !item.file?.url) {
// type 为 'file' 时 key 和 url 不能同时为空
if (item.type === ChatItemValueTypeEnum.file && !item.file?.key && !item.file?.url) {
return false;
}
return true;
@ -204,7 +205,7 @@ export const getChatSourceByPublishChannel = (publishChannel: PublishChannelEnum
}
};
/*
/*
Merge chat responseData
1. Same tool mergeSignId (Interactive tool node)
2. Recursively merge plugin details with same mergeSignId

View File

@ -0,0 +1,38 @@
import { ParentIdSchema } from '../../../../common/parentFolder/type';
import { SystemToolBasicConfigSchema, ToolSecretInputItemSchema } from '../../tool/type';
import z from 'zod';
export const AdminSystemToolListItemSchema = SystemToolBasicConfigSchema.extend({
id: z.string(),
parentId: ParentIdSchema,
name: z.string(),
intro: z.string().optional(),
author: z.string().optional(),
avatar: z.string().optional(),
tags: z.array(z.string()).nullish(),
hasSystemSecret: z.boolean().optional(),
// App tool
associatedPluginId: z.string().optional(),
isFolder: z.boolean().optional(),
hasSecretInput: z.boolean()
});
export type AdminSystemToolListItemType = z.infer<typeof AdminSystemToolListItemSchema>;
// Child config schema for update
export const ToolsetChildSchema = z.object({
pluginId: z.string(),
name: z.string(),
systemKeyCost: z.number().optional()
});
export const AdminSystemToolDetailSchema = AdminSystemToolListItemSchema.omit({
hasSecretInput: true
}).extend({
userGuide: z.string().nullish(),
inputList: z.array(ToolSecretInputItemSchema).optional(),
inputListVal: z.record(z.string(), z.any()).nullish(),
childTools: z.array(ToolsetChildSchema).optional()
});
export type AdminSystemToolDetailType = z.infer<typeof AdminSystemToolDetailSchema>;

View File

@ -0,0 +1,9 @@
import z from 'zod';
export const TeamInstalledPluginSchema = z.object({
_id: z.string(),
teamId: z.string(),
pluginId: z.string(),
installed: z.boolean()
});
export type TeamInstalledPluginSchemaType = z.infer<typeof TeamInstalledPluginSchema>;

View File

@ -0,0 +1,57 @@
import z from 'zod';
import { PluginStatusEnum, PluginStatusSchema } from '../type';
// 无论哪种 Tool都会有这一层配置
export const SystemToolBasicConfigSchema = z.object({
defaultInstalled: z.boolean().optional(),
status: PluginStatusSchema.optional().default(PluginStatusEnum.Normal),
originCost: z.number().optional(),
currentCost: z.number().optional(),
hasTokenFee: z.boolean().optional(),
systemKeyCost: z.number().optional(),
pluginOrder: z.number().optional()
});
export const SystemPluginToolCollectionSchema = SystemToolBasicConfigSchema.extend({
pluginId: z.string(),
customConfig: z
.object({
name: z.string(),
avatar: z.string().optional(),
intro: z.string().optional(),
toolDescription: z.string().optional(),
version: z.string(),
tags: z.array(z.string()).nullish(),
associatedPluginId: z.string().optional(),
userGuide: z.string().optional(),
author: z.string().optional()
})
.optional(),
inputListVal: z.record(z.string(), z.any()).optional(),
// @deprecated
isActive: z.boolean().optional(),
inputConfig: z
.array(
z.object({
key: z.string(),
label: z.string(),
description: z.string().optional(),
value: z.any().optional()
})
)
.optional()
});
export type SystemPluginToolCollectionType = z.infer<typeof SystemPluginToolCollectionSchema>;
// TODO: 移动到 plugin sdk 里
export const ToolSecretInputItemSchema = z.object({
key: z.string(),
label: z.string(),
description: z.string().optional(),
required: z.boolean().optional(),
inputType: z.enum(['input', 'numberInput', 'secret', 'switch', 'select']),
value: z.any().optional(),
list: z.array(z.object({ label: z.string(), value: z.string() })).optional()
});
export type ToolSecretInputItemType = z.infer<typeof ToolSecretInputItemSchema>;

View File

@ -0,0 +1,43 @@
import { z } from 'zod';
import { i18nT } from '../../../web/i18n/utils';
export const I18nStringSchema = z.object({
en: z.string(),
'zh-CN': z.string().optional(),
'zh-Hant': z.string().optional()
});
// I18nStringType can be either an object with language keys or a plain string
export const I18nUnioStringSchema = z.union([I18nStringSchema, z.string()]);
export const PluginToolTagSchema = z.object({
tagId: z.string(),
tagName: I18nUnioStringSchema,
tagOrder: z.number(),
isSystem: z.boolean()
});
export type SystemPluginToolTagType = z.infer<typeof PluginToolTagSchema>;
export const PluginStatusSchema = z.union([z.literal(1), z.literal(2), z.literal(3)]);
export type PluginStatusType = z.infer<typeof PluginStatusSchema>;
export enum PluginStatusEnum {
Normal = 1,
SoonOffline = 2,
Offline = 3
}
export const PluginStatusMap = {
[PluginStatusEnum.Normal]: {
label: i18nT('app:toolkit_status_normal'),
tooltip: '',
tagColor: 'blue' as const
},
[PluginStatusEnum.SoonOffline]: {
label: i18nT('app:toolkit_status_soon_offline'),
tooltip: i18nT('app:tool_soon_offset_tips'),
tagColor: 'yellow' as const
},
[PluginStatusEnum.Offline]: {
label: i18nT('app:toolkit_status_offline'),
tooltip: i18nT('app:tool_offset_tips'),
tagColor: 'red' as const
}
};

View File

@ -328,7 +328,7 @@ export enum VariableInputEnum {
password = 'password',
file = 'file',
modelSelect = 'modelSelect',
llmSelect = 'llmSelect',
datasetSelect = 'datasetSelect',
custom = 'custom',
@ -386,20 +386,26 @@ export const variableConfigs: VariableConfigType[][] = [
label: i18nT('common:core.workflow.inputType.switch'),
value: VariableInputEnum.switch,
defaultValueType: WorkflowIOValueTypeEnum.boolean
},
{
icon: 'core/workflow/inputType/timePointSelect',
label: i18nT('common:core.workflow.inputType.timePointSelect'),
value: VariableInputEnum.timePointSelect,
defaultValueType: WorkflowIOValueTypeEnum.string
},
{
icon: 'core/workflow/inputType/timeRangeSelect',
label: i18nT('common:core.workflow.inputType.timeRangeSelect'),
value: VariableInputEnum.timeRangeSelect,
defaultValueType: WorkflowIOValueTypeEnum.arrayString
},
{
icon: 'core/workflow/inputType/model',
label: i18nT('common:core.workflow.inputType.modelSelect'),
value: VariableInputEnum.llmSelect,
defaultValueType: WorkflowIOValueTypeEnum.string
}
// {
// icon: 'core/workflow/inputType/timePointSelect',
// label: i18nT('common:core.workflow.inputType.timePointSelect'),
// value: VariableInputEnum.timePointSelect,
// defaultValueType: WorkflowIOValueTypeEnum.string
// },
// {
// icon: 'core/workflow/inputType/timeRangeSelect',
// label: i18nT('common:core.workflow.inputType.timeRangeSelect'),
// value: VariableInputEnum.timeRangeSelect,
// defaultValueType: WorkflowIOValueTypeEnum.arrayString
// }
// {
// icon: 'core/workflow/inputType/file',
// label: i18nT('common:core.workflow.inputType.file'),
// value: VariableInputEnum.file,
@ -410,14 +416,14 @@ export const variableConfigs: VariableConfigType[][] = [
// {
// icon: 'core/workflow/inputType/model',
// label: i18nT('common:core.workflow.inputType.modelSelect'),
// value: VariableInputEnum.modelSelect,
// value: VariableInputEnum.llmSelect,
// defaultValueType: WorkflowIOValueTypeEnum.string
// },
// {
// icon: 'core/workflow/inputType/dataset',
// label: i18nT('common:core.workflow.inputType.datasetSelect'),
// value: VariableInputEnum.datasetSelect,
// defaultValueType: WorkflowIOValueTypeEnum.arrayString
// defaultValueType: WorkflowIOValueTypeEnum.selectDataset
// }
// ],
[

View File

@ -13,11 +13,9 @@ export enum FlowNodeInputTypeEnum { // render ui
JSONEditor = 'JSONEditor',
addInputParam = 'addInputParam', // params input
customVariable = 'customVariable', // 外部变量
// special input
selectApp = 'selectApp',
customVariable = 'customVariable',
// ai model select
selectLLMModel = 'selectLLMModel',
settingLLMModel = 'settingLLMModel',
@ -28,7 +26,7 @@ export enum FlowNodeInputTypeEnum { // render ui
settingDatasetQuotePrompt = 'settingDatasetQuotePrompt',
hidden = 'hidden',
custom = 'custom',
custom = 'custom', // 自定义渲染
fileSelect = 'fileSelect',
timePointSelect = 'timePointSelect',

View File

@ -30,6 +30,8 @@ import type {
} from '../template/system/interactive/type';
import type { SearchDataResponseItemType } from '../../dataset/type';
import type { localeType } from '../../../common/i18n/type';
import { type UserChatItemValueItemType } from '../../chat/type';
export type ExternalProviderType = {
openaiAccount?: OpenaiAccountType;
externalWorkflowVariables?: Record<string, string>;
@ -102,7 +104,7 @@ export type SystemVariablesType = {
export type RuntimeNodeItemType = {
nodeId: StoreNodeItemType['nodeId'];
name: StoreNodeItemType['name'];
avatar: StoreNodeItemType['avatar'];
avatar?: StoreNodeItemType['avatar'];
intro?: StoreNodeItemType['intro'];
toolDescription?: StoreNodeItemType['toolDescription'];
flowNodeType: StoreNodeItemType['flowNodeType'];

View File

@ -52,7 +52,8 @@ export const Input_Template_SettingAiModel: FlowNodeInputItemType = {
export const Input_Template_System_Prompt: FlowNodeInputItemType = {
key: NodeInputKeyEnum.aiSystemPrompt,
renderTypeList: [FlowNodeInputTypeEnum.textarea, FlowNodeInputTypeEnum.reference],
max: 3000,
maxLength: 100000,
isRichText: true,
valueType: WorkflowIOValueTypeEnum.string,
label: i18nT('common:core.ai.Prompt'),
description: systemPromptTip,

View File

@ -24,6 +24,8 @@ export const AssignedAnswerModule: FlowNodeTemplateType = {
renderTypeList: [FlowNodeInputTypeEnum.textarea, FlowNodeInputTypeEnum.reference],
valueType: WorkflowIOValueTypeEnum.any,
required: true,
isRichText: false,
maxLength: 100000,
label: i18nT('common:core.module.input.label.Response content'),
description: i18nT('common:core.module.input.description.Response content'),
placeholder: i18nT('common:core.module.input.description.Response content')

View File

@ -34,7 +34,7 @@ export type WorkflowTemplateType = {
parentId?: ParentIdType;
isFolder?: boolean;
avatar: string;
avatar?: string;
name: I18nStringType | string;
intro?: I18nStringType | string;
toolDescription?: string;

View File

@ -15,9 +15,18 @@ export type CustomFieldConfigType = {
showDescription?: boolean;
};
export type InputComponentPropsType = {
key: `${NodeInputKeyEnum}` | string;
label: string;
valueType?: WorkflowIOValueTypeEnum; // data type
required?: boolean;
defaultValue?: any;
referencePlaceholder?: string;
isRichText?: boolean;
placeholder?: string; // input,textarea
maxLength?: number; // input,textarea
minLength?: number; // password
list?: { label: string; value: string }[]; // select
@ -27,12 +36,29 @@ export type InputComponentPropsType = {
min?: number; // slider, number input
precision?: number; // number input
defaultValue?: string;
llmModelType?: `${LLMModelTypeEnum}`;
// file
canSelectFile?: boolean;
canSelectImg?: boolean;
canSelectVideo?: boolean;
canSelectAudio?: boolean;
canSelectCustomFileExtension?: boolean;
customFileExtensionList?: string[];
canLocalUpload?: boolean;
canUrlUpload?: boolean;
maxFiles?: number;
// Time
timeGranularity?: 'day' | 'hour' | 'minute' | 'second';
timeRangeStart?: string;
timeRangeEnd?: string;
// dynamic input
customInputConfig?: CustomFieldConfigType;
// @deprecated
enums?: { value: string; label: string }[];
};
export type InputConfigType = {
key: string;
@ -50,30 +76,20 @@ export type FlowNodeInputItemType = InputComponentPropsType & {
selectedTypeIndex?: number;
renderTypeList: FlowNodeInputTypeEnum[]; // Node Type. Decide on a render style
key: `${NodeInputKeyEnum}` | string;
valueType?: WorkflowIOValueTypeEnum; // data type
valueDesc?: string; // data desc
value?: any;
label: string;
debugLabel?: string;
description?: string; // field desc
required?: boolean;
enum?: string;
inputList?: InputConfigType[]; // when key === 'system_input_config', this field is used
toolDescription?: string; // If this field is not empty, it is entered as a tool
enum?: string;
inputList?: InputConfigType[]; // when key === 'system_input_config', this field is used
// render components params
canEdit?: boolean; // dynamic inputs
isPro?: boolean; // Pro version field
isToolOutput?: boolean;
// file
canSelectFile?: boolean;
canSelectImg?: boolean;
maxFiles?: number;
deprecated?: boolean;
};

View File

@ -19,16 +19,13 @@ import { ChatNodeUsageType } from '../../../support/wallet/bill/type';
import { RuntimeNodeItemType } from '../runtime/type';
import { RuntimeEdgeItemType, StoreEdgeItemType } from './edge';
import { NextApiResponse } from 'next';
import type {
AppDetailType,
AppSchema,
McpToolConfigType,
HttpToolConfigType
} from '../../app/type';
import type { ParentIdType } from 'common/parentFolder/type';
import type { AppDetailType, AppSchema, HttpToolConfigType } from '../../app/type';
import type { McpToolConfigType } from '../../app/tool/mcpTool/type';
import type { ParentIdType } from '../../../common/parentFolder/type';
import { AppTypeEnum } from '../../app/constants';
import type { WorkflowInteractiveResponseType } from '../template/system/interactive/type';
import type { StoreSecretValueType } from '../../../common/secret/type';
import type { PluginStatusType } from '../../plugin/type';
export type NodeToolConfigType = {
mcpToolSet?: {
@ -105,6 +102,7 @@ export type PluginDataType = {
name?: string;
avatar?: string;
error?: string;
status?: PluginStatusType;
};
type HandleType = {
@ -117,6 +115,7 @@ type HandleType = {
export type FlowNodeTemplateType = FlowNodeCommonType & {
id: string; // node id, unique
templateType: string;
status?: PluginStatusType;
showSourceHandle?: boolean;
showTargetHandle?: boolean;
@ -131,9 +130,9 @@ export type FlowNodeTemplateType = FlowNodeCommonType & {
diagram?: string; // diagram url
courseUrl?: string; // course url
userGuide?: string; // user guide
tags?: string[] | null;
// @deprecated
// show handle
sourceHandle?: HandleType;
targetHandle?: HandleType;
};
@ -143,7 +142,8 @@ export type NodeTemplateListItemType = {
flowNodeType: FlowNodeTypeEnum; // render node card
parentId?: ParentIdType;
isFolder?: boolean;
templateType: string;
templateType?: string;
tags?: string[] | null;
avatar?: string;
name: string;
intro?: string; // template list intro

View File

@ -39,7 +39,6 @@ import {
defaultWhisperConfig
} from '../app/constants';
import { IfElseResultEnum } from './template/system/ifElse/constant';
import { type RuntimeNodeItemType } from './runtime/type';
import {
Input_Template_File_Link,
Input_Template_History,
@ -51,7 +50,6 @@ import { type RuntimeUserPromptType, type UserChatItemType } from '../../core/ch
import { getNanoid } from '../../common/string/tools';
import { ChatRoleEnum } from '../../core/chat/constants';
import { runtimePrompt2ChatsValue } from '../../core/chat/adapt';
import { getPluginRunContent } from '../../core/app/plugin/utils';
export const getHandleId = (
nodeId: string,
@ -262,7 +260,7 @@ export const appData2FlowNodeIO = ({
[VariableInputEnum.switch]: [FlowNodeInputTypeEnum.switch],
[VariableInputEnum.password]: [FlowNodeInputTypeEnum.password],
[VariableInputEnum.file]: [FlowNodeInputTypeEnum.fileSelect],
[VariableInputEnum.modelSelect]: [FlowNodeInputTypeEnum.selectLLMModel],
[VariableInputEnum.llmSelect]: [FlowNodeInputTypeEnum.selectLLMModel],
[VariableInputEnum.datasetSelect]: [FlowNodeInputTypeEnum.selectDataset],
[VariableInputEnum.internal]: [FlowNodeInputTypeEnum.hidden],
[VariableInputEnum.custom]: [FlowNodeInputTypeEnum.input, FlowNodeInputTypeEnum.reference]
@ -385,43 +383,8 @@ export const getElseIFLabel = (i: number) => {
return i === 0 ? IfElseResultEnum.IF : `${IfElseResultEnum.ELSE_IF} ${i}`;
};
// add value to plugin input node when run plugin
export const updatePluginInputByVariables = (
nodes: RuntimeNodeItemType[],
variables: Record<string, any>
) => {
return nodes.map((node) =>
node.flowNodeType === FlowNodeTypeEnum.pluginInput
? {
...node,
inputs: node.inputs.map((input) => {
const parseValue = (() => {
try {
if (
input.valueType === WorkflowIOValueTypeEnum.string ||
input.valueType === WorkflowIOValueTypeEnum.number ||
input.valueType === WorkflowIOValueTypeEnum.boolean
)
return variables[input.key];
return JSON.parse(variables[input.key]);
} catch (e) {
return variables[input.key];
}
})();
return {
...input,
value: parseValue ?? input.value
};
})
}
: node
);
};
/* Get plugin runtime input user query */
export const getPluginRunUserQuery = ({
export const clientGetWorkflowToolRunUserQuery = ({
pluginInputs,
variables,
files = []
@ -430,6 +393,25 @@ export const getPluginRunUserQuery = ({
variables: Record<string, any>;
files?: RuntimeUserPromptType['files'];
}): UserChatItemType & { dataId: string } => {
const getPluginRunContent = ({
pluginInputs,
variables
}: {
pluginInputs: FlowNodeInputItemType[];
variables: Record<string, any>;
}) => {
const pluginInputsWithValue = pluginInputs.map((input) => {
const { key } = input;
let value = variables?.hasOwnProperty(key) ? variables[key] : input.defaultValue;
return {
...input,
value
};
});
return JSON.stringify(pluginInputsWithValue);
};
return {
dataId: getNanoid(24),
obj: ChatRoleEnum.Human,

View File

@ -0,0 +1,7 @@
import { z } from 'zod';
export const PaginationSchema = z.object({
pageSize: z.union([z.number(), z.string()]),
offset: z.union([z.number(), z.string()]).optional(),
pageNum: z.union([z.number(), z.string()]).optional()
});

View File

@ -7,13 +7,14 @@ import {
UpdateFavouriteAppTagsParamsSchema
} from './api';
import { ObjectIdSchema } from '../../../../common/type/mongo';
import { TagsMap } from '../../../tag';
export const ChatFavouriteAppPath: OpenAPIPath = {
'/proApi/core/chat/setting/favourite/list': {
get: {
summary: '获取精选应用列表',
description: '获取团队配置的精选应用列表,支持按名称和标签筛选',
tags: ['对话页配置'],
tags: [TagsMap.chatSetting],
requestParams: {
query: GetChatFavouriteListParamsSchema
},
@ -33,7 +34,7 @@ export const ChatFavouriteAppPath: OpenAPIPath = {
post: {
summary: '更新精选应用',
description: '批量创建或更新精选应用配置,包括应用 ID、标签和排序信息',
tags: ['对话页配置'],
tags: [TagsMap.chatSetting],
requestBody: {
content: {
'application/json': {
@ -57,7 +58,7 @@ export const ChatFavouriteAppPath: OpenAPIPath = {
put: {
summary: '更新精选应用排序',
description: '批量更新精选应用的显示顺序',
tags: ['对话页配置'],
tags: [TagsMap.chatSetting],
requestBody: {
content: {
'application/json': {
@ -89,7 +90,7 @@ export const ChatFavouriteAppPath: OpenAPIPath = {
put: {
summary: '更新精选应用标签',
description: '批量更新精选应用的标签分类',
tags: ['对话页配置'],
tags: [TagsMap.chatSetting],
requestBody: {
content: {
'application/json': {
@ -113,7 +114,7 @@ export const ChatFavouriteAppPath: OpenAPIPath = {
delete: {
summary: '删除精选应用',
description: '根据 ID 删除指定的精选应用配置',
tags: ['对话页配置'],
tags: [TagsMap.chatSetting],
requestParams: {
query: z.object({
id: ObjectIdSchema

View File

@ -1,7 +1,61 @@
import type { OpenAPIPath } from '../../type';
import { ChatSettingPath } from './setting';
import { ChatFavouriteAppPath } from './favourite/index';
import { z } from 'zod';
import { CreatePostPresignedUrlResultSchema } from '../../../../service/common/s3/type';
import { PresignChatFileGetUrlSchema, PresignChatFilePostUrlSchema } from '../../../core/chat/api';
import { TagsMap } from '../../tag';
export const ChatPath = {
export const ChatPath: OpenAPIPath = {
...ChatSettingPath,
...ChatFavouriteAppPath
...ChatFavouriteAppPath,
'/core/chat/presignChatFileGetUrl': {
post: {
summary: '获取对话文件预签名 URL',
description: '获取对话文件的预签名 URL',
tags: [TagsMap.chatPage],
requestBody: {
content: {
'application/json': {
schema: PresignChatFileGetUrlSchema
}
}
},
responses: {
200: {
description: '成功获取对话文件预签名 URL',
content: {
'application/json': {
schema: z.string()
}
}
}
}
}
},
'/core/chat/presignChatFilePostUrl': {
post: {
summary: '上传对话文件预签名 URL',
description: '上传对话文件的预签名 URL',
tags: [TagsMap.chatPage],
requestBody: {
content: {
'application/json': {
schema: PresignChatFilePostUrlSchema
}
}
},
responses: {
200: {
description: '成功上传对话文件预签名 URL',
content: {
'application/json': {
schema: CreatePostPresignedUrlResultSchema
}
}
}
}
}
}
};

View File

@ -1,5 +1,6 @@
import type { OpenAPIPath } from '../../../type';
import { ChatSettingSchema, ChatSettingModelSchema } from '../../../../core/chat/setting/type';
import { TagsMap } from '../../../tag';
export const ChatSettingPath: OpenAPIPath = {
'/proApi/core/chat/setting/detail': {
@ -7,7 +8,7 @@ export const ChatSettingPath: OpenAPIPath = {
summary: '获取对话页设置',
description:
'获取当前团队的对话页设置,包括 slogan、对话提示、Logo、快捷应用、已选工具和精选应用标签等配置信息',
tags: ['对话页配置'],
tags: [TagsMap.chatSetting],
responses: {
200: {
description: '成功返回对话页设置信息',
@ -25,7 +26,7 @@ export const ChatSettingPath: OpenAPIPath = {
summary: '更新对话页设置',
description:
'更新团队的对话页设置配置,包括 slogan、对话提示、Logo、快捷应用、已选工具和精选应用标签等信息',
tags: ['对话页配置'],
tags: [TagsMap.chatSetting],
requestBody: {
content: {
'application/json': {

View File

@ -0,0 +1,52 @@
import { I18nStringSchema, I18nUnioStringSchema } from '../../../../core/plugin/type';
import { z } from 'zod';
/* ============ Pkg Plugin ============== */
// 1. Get Pkg Plugin Upload URL Schema
export const GetPkgPluginUploadURLQuerySchema = z.object({
filename: z.string()
});
export type GetPkgPluginUploadURLQueryType = z.infer<typeof GetPkgPluginUploadURLQuerySchema>;
export const GetPkgPluginUploadURLResponseSchema = z.object({
postURL: z.string(),
formData: z.record(z.string(), z.string()),
objectName: z.string()
});
export type GetPkgPluginUploadURLResponseType = z.infer<typeof GetPkgPluginUploadURLResponseSchema>;
// 2. Parse Uploaded Pkg Plugin Schema
export const ParseUploadedPkgPluginQuerySchema = z.object({
objectName: z.string()
});
export type ParseUploadedPkgPluginQueryType = z.infer<typeof ParseUploadedPkgPluginQuerySchema>;
export const ParseUploadedPkgPluginResponseSchema = z.array(
z.object({
toolId: z.string(),
name: I18nUnioStringSchema,
description: I18nStringSchema,
icon: z.string(),
parentId: z.string().optional(),
tags: z.array(z.string()).nullish()
})
);
export type ParseUploadedPkgPluginResponseType = z.infer<
typeof ParseUploadedPkgPluginResponseSchema
>;
// 3. Confirm Uploaded Pkg Plugin Schema
export const ConfirmUploadPkgPluginBodySchema = z.object({
toolIds: z.array(z.string())
});
export type ConfirmUploadPkgPluginBodyType = z.infer<typeof ConfirmUploadPkgPluginBodySchema>;
// 4. Delete Pkg Plugin Schema
export const DeletePkgPluginQuerySchema = z.object({
toolId: z.string()
});
export type DeletePkgPluginQueryType = z.infer<typeof DeletePkgPluginQuerySchema>;
// Install plugin from url
export const InstallPluginFromUrlBodySchema = z.object({
downloadUrls: z.array(z.string())
});
export type InstallPluginFromUrlBodyType = z.infer<typeof InstallPluginFromUrlBodySchema>;

View File

@ -0,0 +1,127 @@
import type { OpenAPIPath } from '../../../type';
import {
GetPkgPluginUploadURLQuerySchema,
GetPkgPluginUploadURLResponseSchema,
ParseUploadedPkgPluginQuerySchema,
ParseUploadedPkgPluginResponseSchema,
ConfirmUploadPkgPluginBodySchema,
DeletePkgPluginQuerySchema,
InstallPluginFromUrlBodySchema
} from './api';
import { TagsMap } from '../../../tag';
import { z } from 'zod';
import { AdminPluginToolPath } from './tool';
export const PluginAdminPath: OpenAPIPath = {
...AdminPluginToolPath,
// Pkg Plugin
'/core/plugin/admin/pkg/presign': {
get: {
summary: '获取插件包上传预签名URL',
description: '获取插件包上传到存储服务的预签名URL需要系统管理员权限',
tags: [TagsMap.pluginAdmin],
requestParams: {
query: GetPkgPluginUploadURLQuerySchema
},
responses: {
200: {
description: '成功获取上传URL',
content: {
'application/json': {
schema: GetPkgPluginUploadURLResponseSchema
}
}
}
}
}
},
'/core/plugin/admin/pkg/parse': {
get: {
summary: '解析已上传的插件包',
description: '解析已上传的插件包,返回插件包中包含的工具信息,需要系统管理员权限',
tags: [TagsMap.pluginAdmin],
requestParams: {
query: ParseUploadedPkgPluginQuerySchema
},
responses: {
200: {
description: '成功解析插件包',
content: {
'application/json': {
schema: ParseUploadedPkgPluginResponseSchema
}
}
}
}
}
},
'/core/plugin/admin/pkg/confirm': {
post: {
summary: '确认上传插件包',
description: '确认上传插件包,将解析的工具添加到系统中,需要系统管理员权限',
tags: [TagsMap.pluginAdmin],
requestBody: {
content: {
'application/json': {
schema: ConfirmUploadPkgPluginBodySchema
}
}
},
responses: {
200: {
description: '成功确认上传',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
'/core/plugin/admin/pkg/delete': {
delete: {
summary: '删除插件包',
description: '删除指定的插件包工具,需要系统管理员权限',
tags: [TagsMap.pluginAdmin],
requestParams: {
query: DeletePkgPluginQuerySchema
},
responses: {
200: {
description: '成功删除插件包',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
'/core/plugin/admin/installWithUrl': {
post: {
summary: '从URL安装插件',
description: '从URL安装插件需要系统管理员权限',
tags: [TagsMap.pluginAdmin],
requestBody: {
content: {
'application/json': {
schema: InstallPluginFromUrlBodySchema
}
}
},
responses: {
200: {
description: '成功安装插件',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
}
};

View File

@ -0,0 +1,87 @@
import type { AdminSystemToolDetailSchema } from '../../../../../core/plugin/admin/tool/type';
import {
AdminSystemToolListItemSchema,
ToolsetChildSchema
} from '../../../../../core/plugin/admin/tool/type';
import z from 'zod';
import { ParentIdSchema } from '../../../../../common/parentFolder/type';
import { PluginStatusSchema } from '../../../../../core/plugin/type';
// Admin tool list
export const GetAdminSystemToolsQuery = z.object({
parentId: ParentIdSchema
});
export type GetAdminSystemToolsQueryType = z.infer<typeof GetAdminSystemToolsQuery>;
export const GetAdminSystemToolsResponseSchema = z.array(AdminSystemToolListItemSchema);
export type GetAdminSystemToolsResponseType = z.infer<typeof GetAdminSystemToolsResponseSchema>;
// Admin tool detail
export const GetAdminSystemToolDetailQuerySchema = z.object({
toolId: z.string()
});
export type GetAdminSystemToolDetailQueryType = z.infer<typeof GetAdminSystemToolDetailQuerySchema>;
export type GetAdminSystemToolDetailResponseType = z.infer<typeof AdminSystemToolDetailSchema>;
// Update Tool Order Schema
export const UpdateToolOrderBodySchema = z.object({
plugins: z.array(
z.object({
pluginId: z.string(),
pluginOrder: z.number()
})
)
});
export type UpdateToolOrderBodyType = z.infer<typeof UpdateToolOrderBodySchema>;
// Update system tool Schema
const UpdateChildToolSchema = ToolsetChildSchema.omit({
name: true
});
export const UpdateToolBodySchema = z.object({
pluginId: z.string(),
status: PluginStatusSchema.optional(),
defaultInstalled: z.boolean().optional(),
originCost: z.number().optional(),
currentCost: z.number().nullish(),
systemKeyCost: z.number().optional(),
hasTokenFee: z.boolean().optional(),
inputListVal: z.record(z.string(), z.any()).nullish(),
childTools: z.array(UpdateChildToolSchema).optional(),
// App tool fields
name: z.string().optional(),
avatar: z.string().optional(),
intro: z.string().optional(),
tagIds: z.array(z.string()).nullish(),
associatedPluginId: z.string().optional(),
userGuide: z.string().nullish(),
author: z.string().optional()
});
export type UpdateToolBodyType = z.infer<typeof UpdateToolBodySchema>;
// Delete system Tool
export const DeleteSystemToolQuerySchema = z.object({
toolId: z.string()
});
export type DeleteSystemToolQueryType = z.infer<typeof DeleteSystemToolQuerySchema>;
/* ======= App type tool ====== */
// Get all system plugin apps
export const GetAllSystemAppsBodySchema = z.object({
searchKey: z.string().optional()
});
export type GetAllSystemAppsBodyType = z.infer<typeof GetAllSystemAppsBodySchema>;
export const GetAllSystemAppsResponseSchema = z.array(
z.object({
_id: z.string(),
avatar: z.string(),
name: z.string()
})
);
export type GetAllSystemAppTypeToolsResponse = z.infer<typeof GetAllSystemAppsResponseSchema>;
// Create app type tool
export const CreateAppToolBodySchema = UpdateToolBodySchema.omit({
childTools: true
});
export type CreateAppToolBodyType = z.infer<typeof CreateAppToolBodySchema>;

View File

@ -0,0 +1,178 @@
import type { OpenAPIPath } from '../../../../type';
import {
CreateAppToolBodySchema,
DeleteSystemToolQuerySchema,
GetAdminSystemToolDetailQuerySchema,
GetAdminSystemToolsQuery,
GetAdminSystemToolsResponseSchema,
GetAllSystemAppsBodySchema,
GetAllSystemAppsResponseSchema,
UpdateToolBodySchema,
UpdateToolOrderBodySchema
} from './api';
import { TagsMap } from '../../../../tag';
import { z } from 'zod';
import { AdminSystemToolDetailSchema } from '../../../../../core/plugin/admin/tool/type';
import { SystemToolTagPath } from './tag';
export const AdminPluginToolPath: OpenAPIPath = {
'/core/plugin/admin/tool/list': {
get: {
summary: '获取系统工具列表',
description: '获取系统工具列表,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestParams: {
query: GetAdminSystemToolsQuery
},
responses: {
'200': {
description: '成功获取系统工具列表',
content: {
'application/json': {
schema: GetAdminSystemToolsResponseSchema
}
}
}
}
}
},
'/core/plugin/admin/tool/detail': {
get: {
summary: '获取系统工具详情',
description: '获取系统工具详情,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestParams: {
query: GetAdminSystemToolDetailQuerySchema
},
responses: {
'200': {
description: '成功获取系统工具详情',
content: {
'application/json': {
schema: AdminSystemToolDetailSchema
}
}
}
}
}
},
'/core/plugin/admin/tool/update': {
put: {
summary: '更新系统工具',
description:
'更新系统工具配置,包括基础字段和自定义字段,支持递归更新子配置,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestBody: {
content: {
'application/json': {
schema: UpdateToolBodySchema
}
}
},
responses: {
200: {
description: '成功更新系统工具',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
'/core/plugin/admin/tool/updateOrder': {
put: {
summary: '更新系统工具顺序',
description: '批量更新系统工具的排序顺序,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestBody: {
content: {
'application/json': {
schema: UpdateToolOrderBodySchema
}
}
},
responses: {
200: {
description: '成功更新系统工具顺序',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
'/core/plugin/admin/tool/delete': {
delete: {
summary: '删除系统工具',
description: '根据ID删除系统工具需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestParams: {
query: DeleteSystemToolQuerySchema
},
responses: {
200: {
description: '成功删除系统工具',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
// Workflow tool
'/core/plugin/admin/tool/workflow/systemApps': {
post: {
summary: '获取所有系统工具类型应用',
description: '获取所有系统工具类型应用,用于选择系统上的应用作为系统工具。需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestBody: {
content: {
'application/json': {
schema: GetAllSystemAppsBodySchema
}
}
},
responses: {
200: {
description: '成功获取系统工具类型应用',
content: {
'application/json': {
schema: GetAllSystemAppsResponseSchema
}
}
}
}
}
},
'/core/plugin/admin/tool/workflow/create': {
post: {
summary: '将系统应用设置成系统工具',
description: '将系统应用设置成系统工具,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestBody: {
content: {
'application/json': {
schema: CreateAppToolBodySchema
}
}
},
responses: {
200: {
description: '成功将系统应用设置成系统工具',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
...SystemToolTagPath
};

View File

@ -0,0 +1,23 @@
import { PluginToolTagSchema } from '../../../../../../core/plugin/type';
import { z } from 'zod';
export const CreatePluginToolTagBodySchema = z.object({
tagName: z.string()
});
export type CreatePluginToolTagBody = z.infer<typeof CreatePluginToolTagBodySchema>;
export const DeletePluginToolTagQuerySchema = z.object({
tagId: z.string()
});
export type DeletePluginToolTagQuery = z.infer<typeof DeletePluginToolTagQuerySchema>;
export const UpdatePluginToolTagBodySchema = z.object({
tagId: z.string(),
tagName: z.string()
});
export type UpdatePluginToolTagBody = z.infer<typeof UpdatePluginToolTagBodySchema>;
export const UpdatePluginToolTagOrderBodySchema = z.object({
tags: z.array(PluginToolTagSchema)
});
export type UpdatePluginToolTagOrderBody = z.infer<typeof UpdatePluginToolTagOrderBodySchema>;

View File

@ -0,0 +1,104 @@
import type { OpenAPIPath } from '../../../../../type';
import { TagsMap } from '../../../../../tag';
import { z } from 'zod';
import {
CreatePluginToolTagBodySchema,
DeletePluginToolTagQuerySchema,
UpdatePluginToolTagBodySchema,
UpdatePluginToolTagOrderBodySchema
} from './api';
export const SystemToolTagPath: OpenAPIPath = {
'/core/plugin/toolTag/config/create': {
post: {
summary: '创建工具标签',
description: '创建新的工具标签,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestBody: {
content: {
'application/json': {
schema: CreatePluginToolTagBodySchema
}
}
},
responses: {
200: {
description: '成功创建工具标签',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
'/core/plugin/toolTag/config/delete': {
delete: {
summary: '删除工具标签',
description: '根据标签ID删除工具标签需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestParams: {
query: DeletePluginToolTagQuerySchema
},
responses: {
200: {
description: '成功删除工具标签',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
'/core/plugin/toolTag/config/update': {
put: {
summary: '更新工具标签',
description: '更新工具标签的名称,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestBody: {
content: {
'application/json': {
schema: UpdatePluginToolTagBodySchema
}
}
},
responses: {
200: {
description: '成功更新工具标签',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
},
'/core/plugin/toolTag/config/updateOrder': {
put: {
summary: '更新工具标签顺序',
description: '批量更新工具标签的排序顺序,需要系统管理员权限',
tags: [TagsMap.pluginToolAdmin],
requestBody: {
content: {
'application/json': {
schema: UpdatePluginToolTagOrderBodySchema
}
}
},
responses: {
200: {
description: '成功更新工具标签顺序',
content: {
'application/json': {
schema: z.object({})
}
}
}
}
}
}
};

View File

@ -0,0 +1,12 @@
import type { OpenAPIPath } from '../../type';
import { MarketplacePath } from './marketplace';
import { PluginToolTagPath } from './toolTag';
import { PluginAdminPath } from './admin';
import { PluginTeamPath } from './team';
export const PluginPath: OpenAPIPath = {
...MarketplacePath,
...PluginToolTagPath,
...PluginAdminPath,
...PluginTeamPath
};

View File

@ -0,0 +1,64 @@
import { z } from 'zod';
import { type ToolSimpleType } from '../../../../sdk/fastgpt-plugin';
import { PaginationSchema } from '../../../api';
import { PluginToolTagSchema } from '../../../../core/plugin/type';
const formatToolDetailSchema = z.object({});
const formatToolSimpleSchema = z.object({});
// Create intersection types for extended schemas
export const MarketplaceToolListItemSchema = formatToolSimpleSchema.extend({
downloadUrl: z.string()
});
export type MarketplaceToolListItemType = ToolSimpleType & {
downloadUrl: string;
};
export const MarketplaceToolDetailItemSchema = formatToolDetailSchema.extend({
readme: z.string().optional()
});
export const MarketplaceToolDetailSchema = z.object({
tools: z.array(MarketplaceToolDetailItemSchema),
downloadUrl: z.string()
});
// List
export const GetMarketplaceToolsBodySchema = PaginationSchema.extend({
searchKey: z.string().optional(),
tags: z.array(z.string()).nullish()
});
export type GetMarketplaceToolsBodyType = z.infer<typeof GetMarketplaceToolsBodySchema>;
export const MarketplaceToolsResponseSchema = z.object({
total: z.number(),
list: z.array(MarketplaceToolListItemSchema)
});
export type MarketplaceToolsResponseType = z.infer<typeof MarketplaceToolsResponseSchema>;
// Detail
export const GetMarketplaceToolDetailQuerySchema = z.object({
toolId: z.string()
});
export type GetMarketplaceToolDetailQueryType = z.infer<typeof GetMarketplaceToolDetailQuerySchema>;
export type GetMarketplaceToolDetailResponseType = z.infer<typeof MarketplaceToolDetailSchema>;
// Tags
export const GetMarketplaceToolTagsResponseSchema = z.array(PluginToolTagSchema);
export type GetMarketplaceToolTagsResponseType = z.infer<
typeof GetMarketplaceToolTagsResponseSchema
>;
// Get installed plugins
export const GetSystemInstalledPluginsQuerySchema = z.object({
type: z.enum(['tool']).optional()
});
export type GetSystemInstalledPluginsQueryType = z.infer<
typeof GetSystemInstalledPluginsQuerySchema
>;
export const GetSystemInstalledPluginsResponseSchema = z.object({
ids: z.array(z.string())
});
export type GetSystemInstalledPluginsResponseType = z.infer<
typeof GetSystemInstalledPluginsResponseSchema
>;

View File

@ -0,0 +1,87 @@
import { type OpenAPIPath } from '../../../type';
import {
GetMarketplaceToolDetailQuerySchema,
GetMarketplaceToolsBodySchema,
MarketplaceToolDetailSchema,
MarketplaceToolsResponseSchema,
GetMarketplaceToolTagsResponseSchema,
GetSystemInstalledPluginsQuerySchema,
GetSystemInstalledPluginsResponseSchema
} from './api';
import { TagsMap } from '../../../tag';
export const MarketplacePath: OpenAPIPath = {
'/core/plugin/admin/marketplace/installed': {
get: {
summary: '获取系统已安装插件的 ID 列表(管理员视角)',
tags: [TagsMap.pluginMarketplace],
requestParams: {
query: GetSystemInstalledPluginsQuerySchema
},
responses: {
200: {
description: '获取系统已安装插件的 ID 列表成功',
content: {
'application/json': {
schema: GetSystemInstalledPluginsResponseSchema
}
}
}
}
}
},
'/marketplace/api/tool/list': {
get: {
summary: '获取工具列表',
tags: [TagsMap.pluginMarketplace],
requestParams: {
query: GetMarketplaceToolsBodySchema
},
responses: {
200: {
description: '获取市场工具列表成功',
content: {
'application/json': {
schema: MarketplaceToolsResponseSchema
}
}
}
}
}
},
'/marketplace/api/tool/detail': {
get: {
summary: '获取单个工具详情',
tags: [TagsMap.pluginMarketplace],
requestParams: {
query: GetMarketplaceToolDetailQuerySchema
},
responses: {
200: {
description: '获取市场工具详情成功',
content: {
'application/json': {
schema: MarketplaceToolDetailSchema
}
}
}
}
}
},
'/marketplace/api/tool/tags': {
get: {
summary: '获取工具标签',
tags: [TagsMap.pluginMarketplace],
responses: {
200: {
description: '获取市场工具标签成功',
content: {
'application/json': {
schema: GetMarketplaceToolTagsResponseSchema
}
}
}
}
}
}
};

View File

@ -0,0 +1,27 @@
import { PluginStatusEnum, PluginStatusSchema } from '../../../../core/plugin/type';
import z from 'zod';
export const GetTeamSystemPluginListQuerySchema = z.object({
type: z.enum(['tool'])
});
export type GetTeamSystemPluginListQueryType = z.infer<typeof GetTeamSystemPluginListQuerySchema>;
export const TeamPluginListItemSchema = z.object({
id: z.string(),
avatar: z.string().optional(),
name: z.string(),
intro: z.string().optional(),
author: z.string().optional(),
tags: z.array(z.string()).nullish(),
status: PluginStatusSchema.optional().default(PluginStatusEnum.Normal),
installed: z.boolean(),
associatedPluginId: z.string().optional()
});
export const GetTeamPluginListResponseSchema = z.array(TeamPluginListItemSchema);
export type GetTeamPluginListResponseType = z.infer<typeof GetTeamPluginListResponseSchema>;
export const ToggleInstallPluginBodySchema = z.object({
pluginId: z.string(),
type: z.enum(['tool']),
installed: z.boolean()
});
export type ToggleInstallPluginBodyType = z.infer<typeof ToggleInstallPluginBodySchema>;

View File

@ -0,0 +1,66 @@
import type { OpenAPIPath } from '../../../type';
import { GetTeamPluginListResponseSchema, ToggleInstallPluginBodySchema } from './api';
import { TagsMap } from '../../../tag';
import { GetTeamToolDetailQuerySchema, TeamToolDetailSchema } from './toolApi';
export const PluginTeamPath: OpenAPIPath = {
'/core/plugin/team/list': {
get: {
summary: '获取团队插件列表',
description: '获取团队插件列表',
tags: [TagsMap.pluginTeam],
responses: {
200: {
description: '获取团队插件列表成功',
content: {
'application/json': {
schema: GetTeamPluginListResponseSchema
}
}
}
}
}
},
'/core/plugin/team/toggleInstall': {
post: {
summary: '切换插件安装状态',
description: '切换团队插件的安装状态,支持安装或卸载插件',
tags: [TagsMap.pluginTeam],
requestBody: {
content: {
'application/json': {
schema: ToggleInstallPluginBodySchema
}
}
},
responses: {
200: {
description: '请求成功',
content: {}
}
}
}
},
// Tool
'/core/plugin/team/toolDetail': {
get: {
summary: '获取工具卡片详情',
description: '获取工具片详情',
tags: [TagsMap.pluginTeam],
requestParams: {
query: GetTeamToolDetailQuerySchema
},
responses: {
200: {
description: '获取工具卡片详情成功',
content: {
'application/json': {
schema: TeamToolDetailSchema
}
}
}
}
}
}
};

View File

@ -0,0 +1,38 @@
import z from 'zod';
export const GetTeamToolDetailQuerySchema = z.object({
toolId: z.string()
});
export type GetTeamToolDetailQueryType = z.infer<typeof GetTeamToolDetailQuerySchema>;
export const ToolDetailItemSchema = z.object({
name: z.string(),
intro: z.string(),
icon: z.string().nullish(),
readme: z.string().nullish(),
versionList: z.array(
z.object({
inputs: z.array(
z.object({
key: z.string(),
label: z.string().nullish(),
description: z.string().nullish(),
valueType: z.string().nullish()
})
),
outputs: z.array(
z.object({
key: z.string(),
label: z.string().nullish(),
description: z.string().nullish(),
valueType: z.string().nullish()
})
)
})
)
});
export const TeamToolDetailSchema = z.object({
tools: z.array(ToolDetailItemSchema),
downloadUrl: z.string()
});
export type GetTeamToolDetailResponseType = z.infer<typeof TeamToolDetailSchema>;

View File

@ -0,0 +1,5 @@
import { PluginToolTagSchema } from '../../../../core/plugin/type';
import { z } from 'zod';
export const GetPluginToolTagsResponseSchema = z.array(PluginToolTagSchema);
export type GetPluginTagListResponse = z.infer<typeof GetPluginToolTagsResponseSchema>;

View File

@ -0,0 +1,24 @@
import type { OpenAPIPath } from '../../../type';
import { GetPluginToolTagsResponseSchema } from './api';
import { TagsMap } from '../../../tag';
import { z } from 'zod';
export const PluginToolTagPath: OpenAPIPath = {
'/core/plugin/toolTag/list': {
get: {
summary: '获取工具标签列表',
description: '获取所有工具标签列表,按排序顺序返回',
tags: [TagsMap.pluginToolTag],
responses: {
200: {
description: '成功获取工具标签列表',
content: {
'application/json': {
schema: GetPluginToolTagsResponseSchema
}
}
}
}
}
}
};

View File

@ -1,6 +1,8 @@
import { createDocument } from 'zod-openapi';
import { ChatPath } from './core/chat';
import { ApiKeyPath } from './support/openapi';
import { TagsMap } from './tag';
import { PluginPath } from './core/plugin';
export const openAPIDocument = createDocument({
openapi: '3.1.0',
@ -11,7 +13,26 @@ export const openAPIDocument = createDocument({
},
paths: {
...ChatPath,
...ApiKeyPath
...ApiKeyPath,
...PluginPath
},
servers: [{ url: '/api' }]
servers: [{ url: '/api' }],
'x-tagGroups': [
{
name: '对话',
tags: [TagsMap.chatSetting, TagsMap.chatPage]
},
{
name: '插件相关',
tags: [TagsMap.pluginToolTag, TagsMap.pluginTeam]
},
{
name: '插件-管理员',
tags: [TagsMap.pluginAdmin, TagsMap.pluginMarketplace, TagsMap.pluginToolAdmin]
},
{
name: 'ApiKey',
tags: [TagsMap.apiKey]
}
]
});

View File

@ -1,12 +1,13 @@
import { z } from 'zod';
import { formatSuccessResponse, getErrorResponse, type OpenAPIPath } from '../../type';
import { type OpenAPIPath } from '../../type';
import { ApiKeyHealthParamsSchema, ApiKeyHealthResponseSchema } from './api';
import { TagsMap } from '../../tag';
export const ApiKeyPath: OpenAPIPath = {
'/support/openapi/health': {
get: {
summary: '检查 API Key 是否健康',
tags: ['APIKey'],
tags: [TagsMap.apiKey],
requestParams: {
query: ApiKeyHealthParamsSchema
},

View File

@ -0,0 +1,10 @@
export const TagsMap = {
chatPage: '对话页',
chatSetting: '对话页配置',
pluginMarketplace: '插件市场(管理员视角)',
pluginToolTag: '工具标签',
pluginAdmin: '管理员插件管理',
pluginToolAdmin: '管理员系统工具管理',
pluginTeam: '团队插件管理',
apiKey: 'APIKey'
};

View File

@ -2,7 +2,7 @@
"name": "@fastgpt/global",
"version": "1.0.0",
"dependencies": {
"@fastgpt-sdk/plugin": "^0.1.19",
"@fastgpt-sdk/plugin": "0.2.15",
"@apidevtools/swagger-parser": "^10.1.0",
"@bany/curl-to-json": "^1.2.8",
"axios": "^1.12.1",

View File

@ -0,0 +1,29 @@
import { z } from 'zod';
export const OutLinkChatAuthSchema = z.union([
z
.object({
shareId: z.string().optional(),
outLinkUid: z.string().optional()
})
.meta({
description: '分享链接鉴权',
example: {
shareId: '1234567890',
outLinkUid: '1234567890'
}
}),
z
.object({
teamId: z.string().optional(),
teamToken: z.string().optional()
})
.meta({
description: '团队鉴权',
example: {
teamId: '1234567890',
teamToken: '1234567890'
}
})
]);
export type OutLinkChatAuthType = z.infer<typeof OutLinkChatAuthSchema>;

View File

@ -1,7 +1,7 @@
import { CommonErrEnum } from '@fastgpt/global/common/error/code/common';
import { type ApiRequestProps } from '../../type/next';
export function parsePaginationRequest(req: ApiRequestProps) {
export const parsePaginationRequest = (req: ApiRequestProps) => {
const {
pageSize = 10,
pageNum = 1,
@ -18,4 +18,4 @@ export function parsePaginationRequest(req: ApiRequestProps) {
pageSize: Number(pageSize),
offset: offset ? Number(offset) : (Number(pageNum) - 1) * Number(pageSize)
};
}
};

View File

@ -21,6 +21,7 @@ const defaultWorkerOpts: Omit<ConnectionOptions, 'connection'> = {
export enum QueueNames {
datasetSync = 'datasetSync',
evaluation = 'evaluation',
s3FileDelete = 's3FileDelete',
// abondoned
websiteSync = 'websiteSync'
}

View File

@ -1,5 +1,5 @@
import { SystemCacheKeyEnum } from './type';
import { refreshSystemTools } from '../../core/app/plugin/controller';
import { refreshSystemTools } from '../../core/app/tool/controller';
export const initCache = () => {
global.systemCache = {

View File

@ -1,4 +1,4 @@
import type { SystemPluginTemplateItemType } from '@fastgpt/global/core/app/plugin/type';
import type { AppToolTemplateItemType } from '@fastgpt/global/core/app/tool/type';
export enum SystemCacheKeyEnum {
systemTool = 'systemTool',
@ -6,7 +6,7 @@ export enum SystemCacheKeyEnum {
}
export type SystemCacheDataType = {
[SystemCacheKeyEnum.systemTool]: SystemPluginTemplateItemType[];
[SystemCacheKeyEnum.systemTool]: AppToolTemplateItemType[];
[SystemCacheKeyEnum.modelPermission]: null;
};

View File

@ -53,20 +53,20 @@ export async function connectMongo(props: {
} catch (error) {}
});
const options = {
await db.connect(url, {
bufferCommands: true,
maxConnecting: maxConnecting,
maxPoolSize: maxConnecting,
minPoolSize: 20,
connectTimeoutMS: 60000,
waitQueueTimeoutMS: 60000,
socketTimeoutMS: 60000,
maxIdleTimeMS: 300000,
retryWrites: true,
retryReads: true
};
await db.connect(url, options);
maxConnecting: maxConnecting, // 最大连接数: 防止连接数过多时无法满足需求
maxPoolSize: maxConnecting, // 最大连接池大小: 防止连接池过大时无法满足需求
minPoolSize: 20, // 最小连接数: 20,防止连接数过少时无法满足需求
connectTimeoutMS: 60000, // 连接超时: 60秒,防止连接失败时长时间阻塞
waitQueueTimeoutMS: 60000, // 等待队列超时: 60秒,防止等待队列长时间阻塞
socketTimeoutMS: 60000, // Socket 超时: 60秒,防止Socket连接失败时长时间阻塞
maxIdleTimeMS: 300000, // 空闲连接超时: 5分钟,防止空闲连接长时间占用资源
retryWrites: true, // 重试写入: 重试写入失败的操作
retryReads: true, // 重试读取: 重试读取失败的操作
serverSelectionTimeoutMS: 10000, // 服务器选择超时: 10秒,防止副本集故障时长时间阻塞
w: 'majority' // 写入确认策略: 多数节点确认后返回,保证数据安全性
});
console.log('mongo connected');
connectedCb?.();

View File

@ -1,16 +1,19 @@
import { Client, type RemoveOptions, type CopyConditions } from 'minio';
import { Client, type RemoveOptions, type CopyConditions, InvalidObjectNameError } from 'minio';
import {
type CreatePostPresignedUrlOptions,
type CreatePostPresignedUrlParams,
type CreatePostPresignedUrlResult,
type S3OptionsType
type S3OptionsType,
type createPreviewUrlParams,
CreateGetPresignedUrlParamsSchema
} from '../type';
import { defaultS3Options, Mimes } from '../constants';
import { defaultS3Options, getSystemMaxFileSize, Mimes } from '../constants';
import path from 'node:path';
import { MongoS3TTL } from '../schema';
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { addHours } from 'date-fns';
import { addLog } from '../../system/log';
import { addS3DelJob } from '../mq';
export class S3BaseBucket {
private _client: Client;
@ -84,8 +87,27 @@ export class S3BaseBucket {
return this.client.bucketExists(this.name);
}
delete(objectKey: string, options?: RemoveOptions): Promise<void> {
return this.client.removeObject(this.name, objectKey, options);
async delete(objectKey: string, options?: RemoveOptions): Promise<void> {
try {
if (!objectKey) return Promise.resolve();
return await this.client.removeObject(this.name, objectKey, options);
} catch (error) {
if (error instanceof InvalidObjectNameError) {
addLog.warn(`${this.name} delete object not found: ${objectKey}`, error);
return Promise.resolve();
}
return Promise.reject(error);
}
}
addDeleteJob({ prefix, key }: { prefix?: string; key?: string }): Promise<void> {
return addS3DelJob({ prefix, key, bucketName: this.name });
}
listObjectsV2(
...params: Parameters<Client['listObjectsV2']> extends [string, ...infer R] ? R : never
) {
return this.client.listObjectsV2(this.name, ...params);
}
async createPostPresignedUrl(
@ -93,11 +115,11 @@ export class S3BaseBucket {
options: CreatePostPresignedUrlOptions = {}
): Promise<CreatePostPresignedUrlResult> {
try {
const { expiredHours } = options;
const { expiredHours, maxFileSize = getSystemMaxFileSize() } = options;
const formatMaxFileSize = maxFileSize * 1024 * 1024;
const filename = params.filename;
const ext = path.extname(filename).toLowerCase();
const contentType = Mimes[ext as keyof typeof Mimes] ?? 'application/octet-stream';
const maxFileSize = this.options.maxFileSize;
const key = (() => {
if ('rawKey' in params) return params.rawKey;
@ -109,8 +131,8 @@ export class S3BaseBucket {
policy.setKey(key);
policy.setBucket(this.name);
policy.setContentType(contentType);
if (maxFileSize) {
policy.setContentLengthRange(1, maxFileSize);
if (formatMaxFileSize) {
policy.setContentLengthRange(1, formatMaxFileSize);
}
policy.setExpires(new Date(Date.now() + 10 * 60 * 1000));
policy.setUserMetaData({
@ -131,11 +153,29 @@ export class S3BaseBucket {
return {
url: postURL,
fields: formData
fields: formData,
maxSize: formatMaxFileSize
};
} catch (error) {
addLog.error('Failed to create post presigned url', error);
return Promise.reject('Failed to create post presigned url');
}
}
async createExtenalUrl(params: createPreviewUrlParams) {
const parsed = CreateGetPresignedUrlParamsSchema.parse(params);
const { key, expiredHours } = parsed;
const expires = expiredHours ? expiredHours * 60 * 60 : 30 * 60; // expires 的单位是秒 默认 30 分钟
return await this.externalClient.presignedGetObject(this.name, key, expires);
}
async createPreviewlUrl(params: createPreviewUrlParams) {
const parsed = CreateGetPresignedUrlParamsSchema.parse(params);
const { key, expiredHours } = parsed;
const expires = expiredHours ? expiredHours * 60 * 60 : 30 * 60; // expires 的单位是秒 默认 30 分钟
return await this.client.presignedGetObject(this.name, key, expires);
}
}

View File

@ -1,5 +1,3 @@
import type { S3PrivateBucket } from './buckets/private';
import type { S3PublicBucket } from './buckets/public';
import { HttpProxyAgent } from 'http-proxy-agent';
import { HttpsProxyAgent } from 'https-proxy-agent';
import type { ClientOptions } from 'minio';
@ -29,11 +27,8 @@ export const Mimes = {
export const defaultS3Options: {
externalBaseURL?: string;
maxFileSize?: number;
afterInit?: () => Promise<void> | void;
} & ClientOptions = {
maxFileSize: 1024 ** 3, // 1GB
useSSL: process.env.S3_USE_SSL === 'true',
endPoint: process.env.S3_ENDPOINT || 'localhost',
externalBaseURL: process.env.S3_EXTERNAL_BASE_URL,
@ -51,3 +46,8 @@ export const S3Buckets = {
public: process.env.S3_PUBLIC_BUCKET || 'fastgpt-public',
private: process.env.S3_PRIVATE_BUCKET || 'fastgpt-private'
} as const;
export const getSystemMaxFileSize = () => {
const config = global.feConfigs?.uploadFileMaxSize || 1024; // MB, default 1024MB
return config; // bytes
};

View File

@ -1,5 +1,7 @@
import { S3PublicBucket } from './buckets/public';
import { S3PrivateBucket } from './buckets/private';
import { addLog } from '../system/log';
import { startS3DelWorker } from './mq';
export function initS3Buckets() {
const publicBucket = new S3PublicBucket();
@ -10,3 +12,8 @@ export function initS3Buckets() {
[privateBucket.name]: privateBucket
};
}
export const initS3MQWorker = async () => {
addLog.info('Init S3 Delete Worker...');
await startS3DelWorker();
};

View File

@ -0,0 +1,77 @@
import { getQueue, getWorker, QueueNames } from '../bullmq';
import pLimit from 'p-limit';
import { retryFn } from '@fastgpt/global/common/system/utils';
export type S3MQJobData = {
key?: string;
prefix?: string;
bucketName: string;
};
export const addS3DelJob = async (data: S3MQJobData): Promise<void> => {
const queue = getQueue<S3MQJobData>(QueueNames.s3FileDelete);
await queue.add(
'delete-s3-files',
{ ...data },
{
attempts: 3,
removeOnFail: false,
removeOnComplete: true,
backoff: {
delay: 2000,
type: 'exponential'
}
}
);
};
export const startS3DelWorker = async () => {
return getWorker<S3MQJobData>(
QueueNames.s3FileDelete,
async (job) => {
const { prefix, bucketName, key } = job.data;
const limit = pLimit(10);
const tasks: Promise<void>[] = [];
const bucket = s3BucketMap[bucketName];
if (!bucket) {
return Promise.reject(`Bucket not found: ${bucketName}`);
}
if (key) {
await bucket.delete(key);
}
if (prefix) {
return new Promise<void>(async (resolve, reject) => {
const stream = bucket.listObjectsV2(prefix, true);
stream.on('data', async (file) => {
if (!file.name) return;
const p = limit(() => retryFn(() => bucket.delete(file.name)));
tasks.push(p);
});
stream.on('end', async () => {
try {
const results = await Promise.allSettled(tasks);
const failed = results.filter((r) => r.status === 'rejected');
if (failed.length > 0) {
reject('Some deletes failed');
}
resolve();
} catch (err) {
reject(err);
}
});
stream.on('error', (err) => {
console.error('listObjects stream error', err);
reject(err);
});
});
}
},
{
concurrency: 1
}
);
};

View File

@ -31,7 +31,10 @@ class S3AvatarSource {
}) {
return this.bucket.createPostPresignedUrl(
{ filename, teamId, source: S3Sources.avatar },
{ expiredHours: autoExpired ? 1 : undefined } // 1 Hourse
{
expiredHours: autoExpired ? 1 : undefined, // 1 Hours
maxFileSize: 5 // 5MB
}
);
}

View File

@ -0,0 +1,59 @@
import { getNanoid } from '@fastgpt/global/common/string/tools';
import { S3PrivateBucket } from '../../buckets/private';
import { S3Sources } from '../../type';
import {
type CheckChatFileKeys,
type DelChatFileByPrefixParams,
ChatFileUploadSchema,
DelChatFileByPrefixSchema
} from './type';
import { MongoS3TTL } from '../../schema';
import { addHours } from 'date-fns';
class S3ChatSource {
private bucket: S3PrivateBucket;
private static instance: S3ChatSource;
constructor() {
this.bucket = new S3PrivateBucket();
}
static getInstance() {
return (this.instance ??= new S3ChatSource());
}
async createGetChatFileURL(params: { key: string; expiredHours?: number; external: boolean }) {
const { key, expiredHours = 1, external = false } = params; // 默认一个小时
if (external) {
return await this.bucket.createExtenalUrl({ key, expiredHours });
}
return await this.bucket.createPreviewlUrl({ key, expiredHours });
}
async createUploadChatFileURL(params: CheckChatFileKeys) {
const { appId, chatId, uId, filename } = ChatFileUploadSchema.parse(params);
const rawKey = [S3Sources.chat, appId, uId, chatId, `${getNanoid(6)}-${filename}`].join('/');
await MongoS3TTL.create({
minioKey: rawKey,
bucketName: this.bucket.name,
expiredTime: addHours(new Date(), 24)
});
return await this.bucket.createPostPresignedUrl({ rawKey, filename });
}
deleteChatFilesByPrefix(params: DelChatFileByPrefixParams) {
const { appId, chatId, uId } = DelChatFileByPrefixSchema.parse(params);
const prefix = [S3Sources.chat, appId, uId, chatId].filter(Boolean).join('/');
return this.bucket.addDeleteJob({ prefix });
}
deleteChatFileByKey(key: string) {
return this.bucket.addDeleteJob({ key });
}
}
export function getS3ChatSource() {
return S3ChatSource.getInstance();
}

View File

@ -0,0 +1,17 @@
import { z } from 'zod';
import { ObjectIdSchema } from '@fastgpt/global/common/type/mongo';
export const ChatFileUploadSchema = z.object({
appId: ObjectIdSchema,
chatId: z.string().length(24),
uId: z.string().nonempty(),
filename: z.string().nonempty()
});
export type CheckChatFileKeys = z.infer<typeof ChatFileUploadSchema>;
export const DelChatFileByPrefixSchema = z.object({
appId: ObjectIdSchema,
chatId: z.string().length(24).optional(),
uId: z.string().nonempty().optional()
});
export type DelChatFileByPrefixParams = z.infer<typeof DelChatFileByPrefixSchema>;

View File

@ -17,7 +17,7 @@ export type ExtensionType = keyof typeof Mimes;
export type S3OptionsType = typeof defaultS3Options;
export const S3SourcesSchema = z.enum(['avatar']);
export const S3SourcesSchema = z.enum(['avatar', 'chat']);
export const S3Sources = S3SourcesSchema.enum;
export type S3SourceType = z.infer<typeof S3SourcesSchema>;
@ -37,16 +37,24 @@ export const CreatePostPresignedUrlParamsSchema = z.union([
export type CreatePostPresignedUrlParams = z.infer<typeof CreatePostPresignedUrlParamsSchema>;
export const CreatePostPresignedUrlOptionsSchema = z.object({
expiredHours: z.number().positive().optional() // TTL in Hours, default 7 * 24
expiredHours: z.number().positive().optional(), // TTL in Hours, default 7 * 24
maxFileSize: z.number().positive().optional() // MB
});
export type CreatePostPresignedUrlOptions = z.infer<typeof CreatePostPresignedUrlOptionsSchema>;
export const CreatePostPresignedUrlResultSchema = z.object({
url: z.string().min(1),
fields: z.record(z.string(), z.string())
url: z.string().nonempty(),
fields: z.record(z.string(), z.string()),
maxSize: z.number().positive().optional() // bytes
});
export type CreatePostPresignedUrlResult = z.infer<typeof CreatePostPresignedUrlResultSchema>;
export const CreateGetPresignedUrlParamsSchema = z.object({
key: z.string().nonempty(),
expiredHours: z.number().positive().optional()
});
export type createPreviewUrlParams = z.infer<typeof CreateGetPresignedUrlParamsSchema>;
declare global {
var s3BucketMap: {
[key: string]: S3BaseBucket;

View File

@ -5,6 +5,7 @@ import { HeaderSecretTypeEnum } from '@fastgpt/global/common/secret/constants';
import { isSecretValue } from '../../../global/common/secret/utils';
export const encryptSecretValue = (value: SecretValueType): SecretValueType => {
if (typeof value !== 'object' || value === null) return value;
if (!value.value) {
return value;
}
@ -51,7 +52,19 @@ export const getSecretValue = ({
};
export const anyValueDecrypt = (value: any) => {
if (!isSecretValue(value)) return value;
const val = (() => {
try {
return JSON.parse(value);
} catch (error) {
return value;
}
})();
return decryptSecret(value.secret);
if (typeof val === 'object' && val !== null && val.value) {
return val.value;
}
if (!isSecretValue(val)) return val;
return decryptSecret(val.secret);
};

View File

@ -12,6 +12,8 @@ import { ChatCompletionRequestMessageRoleEnum } from '@fastgpt/global/core/ai/co
import { i18nT } from '../../../../web/i18n/utils';
import { addLog } from '../../../common/system/log';
import { getImageBase64 } from '../../../common/file/image/utils';
import { getS3ChatSource } from '../../../common/s3/sources/chat';
import { isInternalAddress } from '../../../common/system/utils';
export const filterGPTMessageByMaxContext = async ({
messages = [],
@ -80,7 +82,7 @@ export const filterGPTMessageByMaxContext = async ({
return [...systemPrompts, ...chats];
};
/*
/*
Format requested messages
1. If not useVision, only retain text.
2. Remove file_url
@ -150,12 +152,7 @@ export const loadRequestMessages = async ({
content.map(async (item) => {
if (item.type === 'image_url') {
// Remove url origin
const imgUrl = (() => {
if (origin && item.image_url.url.startsWith(origin)) {
return item.image_url.url.replace(origin, '');
}
return item.image_url.url;
})();
const imgUrl = item.image_url.url;
// base64 image
if (imgUrl.startsWith('data:image/')) {
@ -164,8 +161,23 @@ export const loadRequestMessages = async ({
try {
// If imgUrl is a local path, load image from local, and set url to base64
if (imgUrl.startsWith('/') || process.env.MULTIPLE_DATA_TO_BASE64 === 'true') {
const { completeBase64: base64 } = await getImageBase64(imgUrl);
if (
imgUrl.startsWith('/') ||
process.env.MULTIPLE_DATA_TO_BASE64 === 'true' ||
isInternalAddress(imgUrl)
) {
const url = await (async () => {
if (item.key) {
try {
return await getS3ChatSource().createGetChatFileURL({
key: item.key,
external: false
});
} catch (error) {}
}
return imgUrl;
})();
const { completeBase64: base64 } = await getImageBase64(url);
return {
...item,
@ -185,7 +197,7 @@ export const loadRequestMessages = async ({
return;
}
} catch (error: any) {
if (error?.response?.status === 405) {
if (error?.response?.status === 405 || error?.response?.status === 403) {
return item;
}
addLog.warn(`Filter invalid image: ${imgUrl}`, { error });

View File

@ -38,7 +38,7 @@ export type SystemDefaultModelType = {
};
declare global {
var ModelProviderRawCache: { provider: string; value: I18nStringStrictType }[];
var ModelProviderRawCache: { provider: string; value: I18nStringStrictType; avatar: string }[];
var ModelProviderListCache: Record<langType, ModelProviderItemType[]>;
var ModelProviderMapCache: Record<langType, Record<string, ModelProviderItemType>>;
var aiproxyIdMapCache: AiproxyMapProviderType;

View File

@ -1,11 +1,14 @@
import { type AppSchema } from '@fastgpt/global/core/app/type';
import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants';
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
import {
FlowNodeInputTypeEnum,
FlowNodeTypeEnum
} from '@fastgpt/global/core/workflow/node/constant';
import { AppTypeEnum } from '@fastgpt/global/core/app/constants';
import { MongoApp } from './schema';
import type { StoreNodeItemType } from '@fastgpt/global/core/workflow/type/node';
import { encryptSecretValue, storeSecretValue } from '../../common/secret/utils';
import { SystemToolInputTypeEnum } from '@fastgpt/global/core/app/systemTool/constants';
import { SystemToolSecretInputTypeEnum } from '@fastgpt/global/core/app/tool/systemTool/constants';
import { type ClientSession } from '../../common/mongo';
import { MongoEvaluation } from './evaluation/evalSchema';
import { removeEvaluationJob } from './evaluation/mq';
@ -24,6 +27,7 @@ import { removeImageByPath } from '../../common/file/image/controller';
import { mongoSessionRun } from '../../common/mongo/sessionRun';
import { MongoAppLogKeys } from './logs/logkeysSchema';
import { MongoChatItemResponse } from '../chat/chatItemResponseSchema';
import { getS3ChatSource } from '../../common/s3/sources/chat';
export const beforeUpdateAppFormat = ({ nodes }: { nodes?: StoreNodeItemType[] }) => {
if (!nodes) return;
@ -34,11 +38,14 @@ export const beforeUpdateAppFormat = ({ nodes }: { nodes?: StoreNodeItemType[] }
if (input.key === NodeInputKeyEnum.headerSecret && typeof input.value === 'object') {
input.value = storeSecretValue(input.value);
}
if (input.renderTypeList.includes(FlowNodeInputTypeEnum.password)) {
input.value = encryptSecretValue(input.value);
}
if (input.key === NodeInputKeyEnum.systemInputConfig && typeof input.value === 'object') {
input.inputList?.forEach((inputItem) => {
if (
inputItem.inputType === 'secret' &&
input.value?.type === SystemToolInputTypeEnum.manual &&
input.value?.type === SystemToolSecretInputTypeEnum.manual &&
input.value?.value
) {
input.value.value[inputItem.key] = encryptSecretValue(input.value.value[inputItem.key]);
@ -158,80 +165,84 @@ export const onDelOneApp = async ({
).lean();
await Promise.all(evalJobs.map((evalJob) => removeEvaluationJob(evalJob._id)));
// Delete chats
await deleteChatFiles({ appId });
await MongoChatItemResponse.deleteMany({
appId
});
await MongoChatItem.deleteMany({
appId
});
await MongoChat.deleteMany({
appId
});
const del = async (app: AppSchema, session: ClientSession) => {
const appId = String(app._id);
const del = async (session: ClientSession) => {
for await (const app of apps) {
const appId = app._id;
// 删除分享链接
await MongoOutLink.deleteMany({
appId
}).session(session);
// Openapi
await MongoOpenApi.deleteMany({
appId
}).session(session);
// 删除分享链接
await MongoOutLink.deleteMany({
appId
}).session(session);
// Openapi
await MongoOpenApi.deleteMany({
appId
}).session(session);
// delete version
await MongoAppVersion.deleteMany({
appId
}).session(session);
// delete version
await MongoAppVersion.deleteMany({
appId
}).session(session);
await MongoChatInputGuide.deleteMany({
appId
}).session(session);
await MongoChatInputGuide.deleteMany({
appId
}).session(session);
// 删除精选应用记录
await MongoChatFavouriteApp.deleteMany({
teamId,
appId
}).session(session);
// 删除精选应用记录
await MongoChatFavouriteApp.deleteMany({
teamId,
appId
}).session(session);
// 从快捷应用中移除对应应用
await MongoChatSetting.updateMany(
{ teamId },
{ $pull: { quickAppIds: { id: String(appId) } } }
).session(session);
// 从快捷应用中移除对应应用
await MongoChatSetting.updateMany(
{ teamId },
{ $pull: { quickAppIds: { id: String(appId) } } }
).session(session);
// Del permission
await MongoResourcePermission.deleteMany({
resourceType: PerResourceTypeEnum.app,
teamId,
resourceId: appId
}).session(session);
// Del permission
await MongoResourcePermission.deleteMany({
resourceType: PerResourceTypeEnum.app,
teamId,
resourceId: appId
}).session(session);
await MongoAppLogKeys.deleteMany({
appId
}).session(session);
await MongoAppLogKeys.deleteMany({
appId
}).session(session);
// delete app
await MongoApp.deleteOne(
{
_id: appId
},
{ session }
);
// delete app
await MongoApp.deleteOne(
{
_id: appId
},
{ session }
);
await removeImageByPath(app.avatar, session);
}
await removeImageByPath(app.avatar, session);
};
if (session) {
await del(session);
return deletedAppIds;
// Delete chats
for await (const app of apps) {
const appId = String(app._id);
await deleteChatFiles({ appId });
await MongoChatItemResponse.deleteMany({
appId
});
await MongoChatItem.deleteMany({
appId
});
await MongoChat.deleteMany({
appId
});
await getS3ChatSource().deleteChatFilesByPrefix({ appId });
}
await mongoSessionRun(del);
return deletedAppIds;
for await (const app of apps) {
if (session) {
await del(app, session);
return deletedAppIds;
}
await mongoSessionRun((session) => del(app, session));
return deletedAppIds;
}
};

View File

@ -2,12 +2,12 @@ import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js';
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
import type { AppSchema } from '@fastgpt/global/core/app/type';
import { type McpToolConfigType } from '@fastgpt/global/core/app/type';
import { type McpToolConfigType } from '@fastgpt/global/core/app/tool/mcpTool/type';
import { addLog } from '../../common/system/log';
import { retryFn } from '@fastgpt/global/common/system/utils';
import { PluginSourceEnum } from '@fastgpt/global/core/app/plugin/constants';
import { AppToolSourceEnum } from '@fastgpt/global/core/app/tool/constants';
import { MongoApp } from './schema';
import type { McpToolDataType } from '@fastgpt/global/core/app/mcpTools/type';
import type { McpToolDataType } from '@fastgpt/global/core/app/tool/mcpTool/type';
import { UserError } from '@fastgpt/global/common/error/utils';
export class MCPClient {
@ -142,7 +142,7 @@ export const getMCPChildren = async (app: AppSchema) => {
return (
app.modules[0].toolConfig?.mcpToolSet?.toolList.map((item) => ({
...item,
id: `${PluginSourceEnum.mcp}-${id}/${item.name}`,
id: `${AppToolSourceEnum.mcp}-${id}/${item.name}`,
avatar: app.avatar
})) ?? []
);
@ -159,7 +159,7 @@ export const getMCPChildren = async (app: AppSchema) => {
return {
avatar: app.avatar,
id: `${PluginSourceEnum.mcp}-${id}/${item.name}`,
id: `${AppToolSourceEnum.mcp}-${id}/${item.name}`,
...toolData
};
});

Some files were not shown because too many files have changed in this diff Show More