mirror of
https://github.com/labring/FastGPT.git
synced 2025-12-25 20:02:47 +00:00
V4.13.0 features (#5693)
* feat: concat usage code (#5657) * feat: dataset parse queue (#5661) * feat: chat usage concat (#5669) * perf: search test usage * feat: chat usage concat * fix: ts * fix: ts * feat: chat node response store (#5675) * feat: chat node response store * limit export * test * add ai generate node (#5506) * add node copilot * apply code * update dynamic input & output * add code test * usage * dynamic input border render * optimize input & output * optimize code * update style * change card to popover * prompt editor basic * prompt editor * handle key down * update prompt * merge * fix * fix * fix * perf: workflow performance (#5677) * feat: chat node response store * limit export * perf: workflow performance * remove log * fix: app template get duplicate (#5682) * fix: dynamic input lock & code param (#5680) * fix: dynamic input lock & code param * fix * fix * feat: multi node data sync & system tool hot-swapping (#5575) * Enhance file upload functionality and system tool integration (#5257) * Enhance file upload functionality and system tool integration * Add supplementary documents and optimize the upload interface * Refactor file plugin types and update upload configurations * Refactor MinIO configuration variables and clean up API plugin handlers for improved readability and consistency * File name change * Refactor SystemTools component layout * fix i18n * fix * fix * fix * optimize app logs sort (#5310) * log keys config modal * multiple select * api * fontsize * code * chatid * fix build * fix * fix component * change name * log keys config * fix * delete unused * fix * chore: minio service class rewrite * chore: s3 plugin upload * feat: system global cache with multi node sync feature * feat: cache * chore: move images * docs: update & remove useless code * chore: resolve merge conflicts * chore: adjust the code * chore: adjust * deps: upgrade @fastgpt-sdk/plugin to 0.1.17 * perf(s3): s3 config * fix: cache syncKey refresh * fix: update @fastgpt-sdk/plugin to v0.1.18 removing mongo definition for fixing vitest * chore: adjust --------- Co-authored-by: Ctrlz <143257420+ctrlz526@users.noreply.github.com> Co-authored-by: heheer <heheer@sealos.io> Co-authored-by: Archer <545436317@qq.com> * perf: s3 api code * fix: toolbox empty when second open modal * feat: http tool set (#5599) * feat: http toolSet manual create front end * feat: http toolSet manual create i18n * feat: http toolSet manual create back end * feat: auth, as tool param, adapt mcp * fix: delete unused httpPlugin * fix: delete FlowNodeTypeEnum.httpPlugin * fix: AppTypeEnum include httpToolSet and httpPlugin * fix * delete console * fix * output schema * fix * fix bg * fix base url * fix --------- Co-authored-by: heheer <zhiyu44@qq.com> * feat: app count * perf: type check * feat: catch error * perf: plugin hot-swapping (#5688) * perf: plugin hot-swapping * chore: adjust code * perf: cite data auth * fix http toolset (#5689) * temp * fix http tool set * fix * template author hide * dynamic IO ui * fix: auth test * fix dynamic input & output (#5690) Co-authored-by: Archer <545436317@qq.com> * fix: dynamic output id * doc * feat: model permission (#5666) * feat(permission): model permission definition & api * chore: support update model's collaborators * feat: remove unauthedmodel when paste and import * fix: type error * fix: test setup global model list * fix: http tool api * chore: update fastgpt-sdk version * chore: remove useless code * chore: myModelList cache * perf: user who is not manager can not configure model permission (FE) * perf: model => Set * feat: getMyModels moved to opensource code; cache the myModelList * fix: type error * fix dynamic input reference select type (#5694) * remove unique index * read file usage * perf: connection error * fix: abort token count * fix: debug usage concat * fix: immer clone object * fix: immer clone object * perf: throw error when error chat * update audit i18n * fix: 修复识别pptx文件后，返回内容顺序错乱问题 (#5696) * fix: pptx sort error * fix prompt editor (#5695) * fix prompt editor * fix * fix: redis cache prefix (#5697) * fix: redis cache prefix * fix: cache * fix: get model collaborator by model.model * feat: hint for model per * rename bucket name * model ui * doc * doc --------- Co-authored-by: heheer <heheer@sealos.io> Co-authored-by: Finley Ge <32237950+FinleyGe@users.noreply.github.com> Co-authored-by: Ctrlz <143257420+ctrlz526@users.noreply.github.com> Co-authored-by: Zeng Qingwen <143274079+fishwww-ww@users.noreply.github.com> Co-authored-by: heheer <zhiyu44@qq.com> Co-authored-by: Deepturn <33342819+Deepturn@users.noreply.github.com>
This commit is contained in:
Archer
GitHub
parent
9f8f8dd3de
commit
051455238c
|
|
@ -12,4 +12,4 @@
|
|||
|
||||
1. 选中的值存储在 hook 里，便于判断是否触发底部悬浮层
|
||||
2. 悬浮层外层 Box 在 hook 里，child 由调用组件实现
|
||||
3. FastGPT/packages/web/hooks/useTableMultipleSelect.tsx 在这个文件下实现
|
||||
3. FastGPT/packages/web/hooks/useTableMultipleSelect.tsx 在这个文件下实现
|
||||
|
|
|
|||
|
|
@ -1,5 +1,14 @@
|
|||
{
|
||||
"title": "系统插件",
|
||||
"description": "介绍如何使用和提交系统插件，以及各插件的填写说明",
|
||||
"pages": ["dev_system_tool","how_to_submit_system_plugin","searxng_plugin_guide","google_search_plugin_guide","bing_search_plugin","doc2x_plugin_guide"]
|
||||
}
|
||||
"pages": [
|
||||
"dev_system_tool",
|
||||
"how_to_submit_system_plugin",
|
||||
"upload_system_tool",
|
||||
"searxng_plugin_guide",
|
||||
"google_search_plugin_guide",
|
||||
"bing_search_plugin",
|
||||
"doc2x_plugin_guide",
|
||||
"deepseek_plugin_guide"
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,123 @@
|
|||
---
|
||||
title: 如何在线上传系统工具
|
||||
description: FastGPT 系统工具在线上传指南
|
||||
---
|
||||
|
||||
> 从 FastGPT 4.14.0 版本开始，系统管理员可以通过 Web 界面直接上传和更新系统工具，无需重新部署服务
|
||||
|
||||
## 权限要求
|
||||
|
||||
⚠️ **重要提示**：只有 **root 用户** 才能使用在线上传系统工具功能。
|
||||
|
||||
- 确保您已使用 `root` 账户登录 FastGPT
|
||||
- 普通用户无法看到"导入/更新"按钮和删除功能
|
||||
|
||||
## 支持的文件格式
|
||||
|
||||
- **文件类型**：`.js` 文件
|
||||
- **文件大小**：最大 10MB
|
||||
- **文件数量**：每次只能上传一个文件
|
||||
|
||||
## 上传步骤
|
||||
|
||||
### 1. 进入系统工具页面
|
||||
|
||||
1. 登录 FastGPT 管理后台
|
||||
2. 导航到：**工作台** → **系统工具**
|
||||
3. 确认页面右上角显示"导入/更新"按钮（只有 root 用户可见）
|
||||
|
||||
|
||||
|
||||
### 2. 准备工具文件
|
||||
|
||||
在上传之前，请确保您的 `.js` 文件是从 fastgpt-plugin 项目中通过 `bun run build` 命令打包后的 dist/tools/built-in 文件夹下得到的
|
||||
|
||||
|
||||
|
||||
### 3. 执行上传
|
||||
|
||||
1. 点击 **"导入/更新"** 按钮
|
||||
2. 在弹出的对话框中，点击文件选择区域
|
||||
3. 选择您准备好的 `.js` 工具文件
|
||||
4. 确认文件信息无误后，点击 **"确认导入"**
|
||||
|
||||
### 4. 上传过程
|
||||
|
||||
- 上传成功后会显示成功提示
|
||||
- 页面自动刷新，新工具会出现在工具列表中
|
||||
|
||||
## 功能特点
|
||||
|
||||
### 工具管理
|
||||
|
||||
- **查看工具**：所有用户都可以查看已安装的系统工具
|
||||
- **上传工具**：仅 root 用户可以上传新工具或更新现有工具
|
||||
- **删除工具**：仅 root 用户可以删除已上传的工具
|
||||
|
||||
### 工具类型识别
|
||||
|
||||
系统会根据工具的配置自动识别工具类型：
|
||||
|
||||
- 🔧 **工具 (tools)**
|
||||
- 🔍 **搜索 (search)**
|
||||
- 🎨 **多模态 (multimodal)**
|
||||
- 💬 **通讯 (communication)**
|
||||
- 📦 **其他 (other)**
|
||||
|
||||
## 常见问题
|
||||
|
||||
### Q: 上传失败，提示"文件内容存在错误"
|
||||
|
||||
**可能原因：**
|
||||
- fastgpt-plugin 项目不是最新的，导致打包的 `.js` 文件缺少正确的内容
|
||||
- 工具配置格式不正确
|
||||
|
||||
**解决方案：**
|
||||
1. 拉取最新的 fastgpt-plugin 项目重新进行 `bun run build` 获得打包后的 `.js` 文件
|
||||
2. 检查本地插件运行是否成功
|
||||
|
||||
### Q: 无法看到"导入/更新"按钮
|
||||
|
||||
**原因：** 当前用户不是 root 用户
|
||||
|
||||
**解决方案：** 使用 root 账户重新登录
|
||||
|
||||
### Q: 文件上传超时
|
||||
|
||||
**可能原因：**
|
||||
- 文件过大（超过 10MB）
|
||||
- 网络连接不稳定
|
||||
|
||||
**解决方案：**
|
||||
1. 确认文件大小在限制范围内
|
||||
2. 检查网络连接
|
||||
3. 尝试重新上传
|
||||
|
||||
## 最佳实践
|
||||
|
||||
### 上传前检查
|
||||
|
||||
1. **代码测试**：在本地环境测试工具功能
|
||||
2. **格式验证**：确保符合 FastGPT 工具规范
|
||||
3. **文件大小**：保持文件在合理大小范围内
|
||||
|
||||
### 版本管理
|
||||
|
||||
- 建议为工具添加版本号注释
|
||||
- 更新工具时，先备份原有版本
|
||||
- 记录更新日志和功能变更
|
||||
|
||||
### 安全考虑
|
||||
|
||||
- 仅上传来源可信的工具文件
|
||||
- 避免包含敏感信息或凭据
|
||||
- 定期审查已安装的工具
|
||||
|
||||
### 存储方式
|
||||
|
||||
- 工具文件存储在 MinIO 中
|
||||
- 工具元数据保存在 MongoDB 中
|
||||
|
||||
---
|
||||
|
||||
通过在线上传功能，您可以快速部署和管理系统工具，提高 FastGPT 的扩展性和灵活性。如遇到问题，请参考上述常见问题或联系技术支持。
|
||||
|
|
@ -90,6 +90,7 @@ description: FastGPT 文档目录
|
|||
- [/docs/introduction/guide/plugins/doc2x_plugin_guide](/docs/introduction/guide/plugins/doc2x_plugin_guide)
|
||||
- [/docs/introduction/guide/plugins/google_search_plugin_guide](/docs/introduction/guide/plugins/google_search_plugin_guide)
|
||||
- [/docs/introduction/guide/plugins/searxng_plugin_guide](/docs/introduction/guide/plugins/searxng_plugin_guide)
|
||||
- [/docs/introduction/guide/plugins/upload_system_tool](/docs/introduction/guide/plugins/upload_system_tool)
|
||||
- [/docs/introduction/guide/team_permissions/invitation_link](/docs/introduction/guide/team_permissions/invitation_link)
|
||||
- [/docs/introduction/guide/team_permissions/team_roles_permissions](/docs/introduction/guide/team_permissions/team_roles_permissions)
|
||||
- [/docs/introduction/index](/docs/introduction/index)
|
||||
|
|
@ -105,7 +106,7 @@ description: FastGPT 文档目录
|
|||
- [/docs/upgrading/4-12/4122](/docs/upgrading/4-12/4122)
|
||||
- [/docs/upgrading/4-12/4123](/docs/upgrading/4-12/4123)
|
||||
- [/docs/upgrading/4-12/4124](/docs/upgrading/4-12/4124)
|
||||
- [/docs/upgrading/4-12/4125](/docs/upgrading/4-12/4125)
|
||||
- [/docs/upgrading/4-13/4130](/docs/upgrading/4-13/4130)
|
||||
- [/docs/upgrading/4-8/40](/docs/upgrading/4-8/40)
|
||||
- [/docs/upgrading/4-8/41](/docs/upgrading/4-8/41)
|
||||
- [/docs/upgrading/4-8/42](/docs/upgrading/4-8/42)
|
||||
|
|
|
|||
|
|
@ -1,26 +0,0 @@
|
|||
---
|
||||
title: 'V4.12.5(进行中)'
|
||||
description: 'FastGPT V4.12.5 更新说明'
|
||||
---
|
||||
|
||||
## 🚀 新增内容
|
||||
|
||||
|
||||
## ⚙️ 优化
|
||||
|
||||
1. 系统工具增加对应 author 名字显示。同时使用安全的 I18n 翻译。
|
||||
|
||||
## 🐛 修复
|
||||
|
||||
1. debug 模式下，全局变量未传递。
|
||||
2. debug 模式下，前方节点参数无法传递至后方节点。
|
||||
3. 调试模式下，开启“自动执行”，会跳过外部变量的填写。
|
||||
4. 自动语音回复未生效。
|
||||
5. 节点复制，报错捕获配置丢失。
|
||||
6. “猜你想问”的自定义提示词，保存时，上一次的值会被置空。
|
||||
7. 配置了二级路由的情况下，知识库检索出来的图片地址拼接异常。
|
||||
8. Prompt 编辑器，键盘输入时会清除掉 Markdown 标记。
|
||||
|
||||
## 🔨 插件更新
|
||||
|
||||
1. 新增火山引擎融合信息搜索工具。
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"title": "4.12.x",
|
||||
"description": "",
|
||||
"pages": ["4125", "4124", "4123", "4122", "4121", "4120"]
|
||||
"pages": ["4124", "4123", "4122", "4121", "4120"]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,83 @@
|
|||
---
|
||||
title: 'V4.13.0(进行中)'
|
||||
description: 'FastGPT V4.13.0 更新说明'
|
||||
---
|
||||
|
||||
## 更新指南
|
||||
|
||||
### 1. 更新镜像：
|
||||
|
||||
- 更新 FastGPT 镜像tag: v4.13.0
|
||||
- 更新 FastGPT 商业版镜像tag: v4.13.0
|
||||
- 更新 fastgpt-plugin 镜像 tag: v0.2.0
|
||||
- mcp_server 无需更新
|
||||
- Sandbox 无需更新
|
||||
- AIProxy 无需更新
|
||||
|
||||
### 2. 更新环境变量
|
||||
|
||||
1. 更新 `fastgpt-plugin` 环境变量名字，并新增`S3_PLUGIN_BUCKET`、`MONGODB_URI`、`REDIS_URL`值。
|
||||
|
||||
```
|
||||
|
||||
S3_EXTERNAL_BASE_URL=https://xxx.com # S3 外网地址
|
||||
S3_ENDPOINT=localhost
|
||||
S3_PORT=9000
|
||||
S3_USE_SSL=false
|
||||
S3_ACCESS_KEY=minioadmin
|
||||
S3_SECRET_KEY=minioadmin
|
||||
S3_TOOL_BUCKET=fastgpt-tool # 系统工具，创建的临时文件，存储的桶，要求公开读私有写。
|
||||
S3_PLUGIN_BUCKET=fastgpt-plugin # 系统插件热安装文件的桶，私有读写。
|
||||
RETENTION_DAYS=15 # 系统工具临时文件保存天数
|
||||
MONGODB_URI=mongodb://myusername:mypassword@mongo:27017/fastgpt?authSource=admin # MongoDB 链接参数
|
||||
REDIS_URL=redis://default:mypassword@redis:6379 # Redis 链接参数
|
||||
```
|
||||
|
||||
2. 增加`fastgpt`和`fastgpt-pro(商业版)` s3 相关环境变量。
|
||||
|
||||
```
|
||||
# S3 外网地址
|
||||
S3_EXTERNAL_BASE_URL=
|
||||
S3_ENDPOINT=localhost
|
||||
S3_PORT=9000
|
||||
S3_USE_SSL=false
|
||||
S3_ACCESS_KEY=minioadmin
|
||||
S3_SECRET_KEY=minioadmin
|
||||
S3_PLUGIN_BUCKET=fastgpt-plugin # 系统插件热安装文件的桶，私有读写。
|
||||
```
|
||||
|
||||
## 🚀 新增内容
|
||||
|
||||
1. 应用新增 HTTP 工具集类型，取代原 HTTP 插件。
|
||||
2. 支持系统管理员通过文件形式快速安装系统工具。
|
||||
3. 团队管理员支持分配模型权限。
|
||||
4. 代码运行节点支持 AI 辅助生成。
|
||||
5. 知识库文件解析支持配置最大并发数。(开源版通过 config.json 文件中`systemEnv.datasetParseMaxProcess`属性配置，商业版通过 admin 后台配置。)
|
||||
|
||||
## ⚙️ 优化
|
||||
|
||||
1. 系统工具增加对应 author 名字显示。同时使用安全的 I18n 翻译。
|
||||
2. 计量计费账单推送和合并逻辑。
|
||||
3. 对话记录中，节点详情单独分表存储。
|
||||
4. 删除 chat_items 中无效的 dataId 索引。
|
||||
5. 工作流UI性能优化，减少 UI 重绘。
|
||||
6. 对话中，知识库引用鉴权采用整个对话框鉴权，而不是单条记录。
|
||||
7. 工作流动态输入输出变量交互优化。
|
||||
|
||||
## 🐛 修复
|
||||
|
||||
1. debug 模式下，全局变量未传递。
|
||||
2. debug 模式下，前方节点参数无法传递至后方节点。
|
||||
3. 调试模式下，开启“自动执行”，会跳过外部变量的填写。
|
||||
4. 自动语音回复未生效。
|
||||
5. 节点复制，报错捕获配置丢失。
|
||||
6. “猜你想问”的自定义提示词，保存时，上一次的值会被置空。
|
||||
7. 配置了二级路由的情况下，知识库检索出来的图片地址拼接异常。
|
||||
8. Prompt 编辑器，键盘输入时会清除掉 Markdown 标记。
|
||||
9. 知识库集合页面，有训练数据时候无法自动刷新页面。
|
||||
10. 工作流快速添加节点弹窗，工具箱页面二次打开时为空。
|
||||
11. PPTX 文件解析顺序错误。
|
||||
|
||||
## 🔨 插件更新
|
||||
|
||||
1. 新增火山引擎融合信息搜索工具。
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
{
|
||||
"title": "4.13.x",
|
||||
"description": "",
|
||||
"pages": ["4130"]
|
||||
}
|
||||
|
|
@ -4,6 +4,8 @@
|
|||
"description": "FastGPT 版本更新介绍及升级操作",
|
||||
"pages": [
|
||||
"index",
|
||||
"---4.13.x---",
|
||||
"...4-13",
|
||||
"---4.12.x---",
|
||||
"...4-12",
|
||||
"---4.11.x---",
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
"document/content/docs/faq/other.mdx": "2025-08-04T22:07:52+08:00",
|
||||
"document/content/docs/faq/points_consumption.mdx": "2025-08-02T19:38:37+08:00",
|
||||
"document/content/docs/introduction/cloud.mdx": "2025-08-02T19:38:37+08:00",
|
||||
"document/content/docs/introduction/commercial.mdx": "2025-08-04T22:07:52+08:00",
|
||||
"document/content/docs/introduction/commercial.mdx": "2025-09-21T23:09:46+08:00",
|
||||
"document/content/docs/introduction/development/community.mdx": "2025-08-02T19:38:37+08:00",
|
||||
"document/content/docs/introduction/development/configuration.mdx": "2025-08-05T23:20:39+08:00",
|
||||
"document/content/docs/introduction/development/custom-models/bge-rerank.mdx": "2025-07-23T21:35:03+08:00",
|
||||
|
|
@ -87,6 +87,7 @@
|
|||
"document/content/docs/introduction/guide/plugins/doc2x_plugin_guide.mdx": "2025-07-23T21:35:03+08:00",
|
||||
"document/content/docs/introduction/guide/plugins/google_search_plugin_guide.mdx": "2025-07-23T21:35:03+08:00",
|
||||
"document/content/docs/introduction/guide/plugins/searxng_plugin_guide.mdx": "2025-07-23T21:35:03+08:00",
|
||||
"document/content/docs/introduction/guide/plugins/upload_system_tool.mdx": "2025-09-20T19:49:21+08:00",
|
||||
"document/content/docs/introduction/guide/team_permissions/invitation_link.mdx": "2025-07-23T21:35:03+08:00",
|
||||
"document/content/docs/introduction/guide/team_permissions/team_roles_permissions.mdx": "2025-07-23T21:35:03+08:00",
|
||||
"document/content/docs/introduction/index.en.mdx": "2025-07-23T21:35:03+08:00",
|
||||
|
|
@ -99,7 +100,7 @@
|
|||
"document/content/docs/protocol/terms.en.mdx": "2025-08-03T22:37:45+08:00",
|
||||
"document/content/docs/protocol/terms.mdx": "2025-08-03T22:37:45+08:00",
|
||||
"document/content/docs/toc.en.mdx": "2025-08-04T13:42:36+08:00",
|
||||
"document/content/docs/toc.mdx": "2025-09-17T22:29:56+08:00",
|
||||
"document/content/docs/toc.mdx": "2025-09-23T14:19:37+08:00",
|
||||
"document/content/docs/upgrading/4-10/4100.mdx": "2025-08-02T19:38:37+08:00",
|
||||
"document/content/docs/upgrading/4-10/4101.mdx": "2025-09-08T20:07:20+08:00",
|
||||
"document/content/docs/upgrading/4-11/4110.mdx": "2025-08-05T23:20:39+08:00",
|
||||
|
|
@ -109,7 +110,7 @@
|
|||
"document/content/docs/upgrading/4-12/4122.mdx": "2025-09-07T14:41:48+08:00",
|
||||
"document/content/docs/upgrading/4-12/4123.mdx": "2025-09-07T20:55:14+08:00",
|
||||
"document/content/docs/upgrading/4-12/4124.mdx": "2025-09-17T22:29:56+08:00",
|
||||
"document/content/docs/upgrading/4-12/4125.mdx": "2025-09-18T16:15:12+08:00",
|
||||
"document/content/docs/upgrading/4-13/4130.mdx": "2025-09-24T21:54:28+08:00",
|
||||
"document/content/docs/upgrading/4-8/40.mdx": "2025-08-02T19:38:37+08:00",
|
||||
"document/content/docs/upgrading/4-8/41.mdx": "2025-08-02T19:38:37+08:00",
|
||||
"document/content/docs/upgrading/4-8/42.mdx": "2025-08-02T19:38:37+08:00",
|
||||
|
|
|
|||
Binary file not shown.
|
After Width: | Height: | Size: 513 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 49 KiB |
|
|
@ -130,10 +130,12 @@ export type FastGPTFeConfigsType = {
|
|||
|
||||
export type SystemEnvType = {
|
||||
openapiPrefix?: string;
|
||||
tokenWorkers: number; // token count max worker
|
||||
|
||||
datasetParseMaxProcess: number;
|
||||
vectorMaxProcess: number;
|
||||
qaMaxProcess: number;
|
||||
vlmMaxProcess: number;
|
||||
tokenWorkers: number; // token count max worker
|
||||
|
||||
hnswEfSearch: number;
|
||||
hnswMaxScanTuples: number;
|
||||
|
|
|
|||
|
|
@ -11,10 +11,13 @@ export enum AppTypeEnum {
|
|||
simple = 'simple',
|
||||
workflow = 'advanced',
|
||||
plugin = 'plugin',
|
||||
httpPlugin = 'httpPlugin',
|
||||
toolSet = 'toolSet',
|
||||
toolSet = 'toolSet', // 'mcp'
|
||||
httpToolSet = 'httpToolSet',
|
||||
tool = 'tool',
|
||||
hidden = 'hidden'
|
||||
hidden = 'hidden',
|
||||
|
||||
// deprecated
|
||||
httpPlugin = 'httpPlugin'
|
||||
}
|
||||
|
||||
export const AppFolderTypeList = [AppTypeEnum.folder, AppTypeEnum.httpPlugin];
|
||||
|
|
|
|||
|
|
@ -1,390 +0,0 @@
|
|||
import { getNanoid } from '../../../common/string/tools';
|
||||
import { type OpenApiJsonSchema } from './type';
|
||||
import yaml from 'js-yaml';
|
||||
import type { OpenAPIV3 } from 'openapi-types';
|
||||
import { type FlowNodeInputItemType, type FlowNodeOutputItemType } from '../../workflow/type/io';
|
||||
import { FlowNodeInputTypeEnum, FlowNodeOutputTypeEnum } from '../../workflow/node/constant';
|
||||
import { WorkflowIOValueTypeEnum } from '../../workflow/constants';
|
||||
import { PluginInputModule } from '../../workflow/template/system/pluginInput';
|
||||
import { PluginOutputModule } from '../../workflow/template/system/pluginOutput';
|
||||
import { HttpNode468 } from '../../workflow/template/system/http468';
|
||||
import { type HttpParamAndHeaderItemType } from '../../workflow/type/io';
|
||||
import { type StoreNodeItemType } from '../../workflow/type/node';
|
||||
import { HttpImgUrl } from '../../../common/file/image/constants';
|
||||
import SwaggerParser from '@apidevtools/swagger-parser';
|
||||
import { getHandleId } from '../../workflow/utils';
|
||||
import { type CreateHttpPluginChildrenPros } from '../controller';
|
||||
import { AppTypeEnum } from '../constants';
|
||||
import type { StoreEdgeItemType } from '../../workflow/type/edge';
|
||||
|
||||
export const str2OpenApiSchema = async (yamlStr = ''): Promise<OpenApiJsonSchema> => {
|
||||
try {
|
||||
const data = (() => {
|
||||
try {
|
||||
return JSON.parse(yamlStr);
|
||||
} catch (jsonError) {
|
||||
return yaml.load(yamlStr, { schema: yaml.FAILSAFE_SCHEMA });
|
||||
}
|
||||
})();
|
||||
const jsonSchema = (await SwaggerParser.parse(data)) as OpenAPIV3.Document;
|
||||
|
||||
const serverPath = jsonSchema.servers?.[0].url || '';
|
||||
const pathData = Object.keys(jsonSchema.paths)
|
||||
.map((path) => {
|
||||
const methodData: any = data.paths[path];
|
||||
return Object.keys(methodData)
|
||||
.filter((method) =>
|
||||
['get', 'post', 'put', 'delete', 'patch'].includes(method.toLocaleLowerCase())
|
||||
)
|
||||
.map((method) => {
|
||||
const methodInfo = methodData[method];
|
||||
if (methodInfo.deprecated) return;
|
||||
const result = {
|
||||
path,
|
||||
method,
|
||||
name: methodInfo.operationId || path,
|
||||
description: methodInfo.description || methodInfo.summary,
|
||||
params: methodInfo.parameters,
|
||||
request: methodInfo?.requestBody,
|
||||
response: methodInfo.responses
|
||||
};
|
||||
return result;
|
||||
});
|
||||
})
|
||||
.flat()
|
||||
.filter(Boolean) as OpenApiJsonSchema['pathData'];
|
||||
return { pathData, serverPath };
|
||||
} catch (err) {
|
||||
throw new Error('Invalid Schema');
|
||||
}
|
||||
};
|
||||
|
||||
export const getType = (schema: { type: string; items?: { type: string } }) => {
|
||||
const typeMap: { [key: string]: WorkflowIOValueTypeEnum } = {
|
||||
string: WorkflowIOValueTypeEnum.arrayString,
|
||||
number: WorkflowIOValueTypeEnum.arrayNumber,
|
||||
integer: WorkflowIOValueTypeEnum.arrayNumber,
|
||||
boolean: WorkflowIOValueTypeEnum.arrayBoolean,
|
||||
object: WorkflowIOValueTypeEnum.arrayObject
|
||||
};
|
||||
|
||||
if (schema?.type === 'integer') {
|
||||
return WorkflowIOValueTypeEnum.number;
|
||||
}
|
||||
|
||||
if (schema?.type === 'array' && schema?.items) {
|
||||
const itemType = typeMap[schema.items.type];
|
||||
if (itemType) {
|
||||
return itemType;
|
||||
}
|
||||
}
|
||||
|
||||
return schema?.type as WorkflowIOValueTypeEnum;
|
||||
};
|
||||
|
||||
export const httpApiSchema2Plugins = async ({
|
||||
parentId,
|
||||
apiSchemaStr = '',
|
||||
customHeader = ''
|
||||
}: {
|
||||
parentId: string;
|
||||
apiSchemaStr?: string;
|
||||
customHeader?: string;
|
||||
}): Promise<CreateHttpPluginChildrenPros[]> => {
|
||||
const jsonSchema = await str2OpenApiSchema(apiSchemaStr);
|
||||
|
||||
const baseUrl = jsonSchema.serverPath;
|
||||
|
||||
return jsonSchema.pathData.map((item) => {
|
||||
const pluginOutputId = getNanoid();
|
||||
const httpId = getNanoid();
|
||||
const pluginInputId = getNanoid();
|
||||
const inputIdMap = new Map();
|
||||
|
||||
const pluginOutputKey = 'result';
|
||||
|
||||
const properties = item.request?.content?.['application/json']?.schema?.properties;
|
||||
const propsKeys = properties ? Object.keys(properties) : [];
|
||||
|
||||
const pluginInputs: FlowNodeInputItemType[] = [
|
||||
...(item.params?.map((param: any) => {
|
||||
return {
|
||||
key: param.name,
|
||||
valueType: getType(param.schema),
|
||||
label: param.name,
|
||||
renderTypeList: [FlowNodeInputTypeEnum.reference],
|
||||
required: param.required,
|
||||
description: param.description,
|
||||
toolDescription: param.description,
|
||||
canEdit: true
|
||||
};
|
||||
}) || []),
|
||||
...(propsKeys?.map((key) => {
|
||||
const prop = properties[key];
|
||||
return {
|
||||
key,
|
||||
valueType: getType(prop),
|
||||
label: key,
|
||||
renderTypeList: [FlowNodeInputTypeEnum.reference],
|
||||
required: false,
|
||||
description: prop.description,
|
||||
toolDescription: prop.description,
|
||||
canEdit: true
|
||||
};
|
||||
}) || [])
|
||||
];
|
||||
|
||||
const pluginOutputs: FlowNodeOutputItemType[] = [
|
||||
...(item.params?.map((param: any) => {
|
||||
const id = getNanoid();
|
||||
inputIdMap.set(param.name, id);
|
||||
return {
|
||||
id,
|
||||
key: param.name,
|
||||
valueType: getType(param.schema),
|
||||
label: param.name,
|
||||
type: FlowNodeOutputTypeEnum.source
|
||||
};
|
||||
}) || []),
|
||||
...(propsKeys?.map((key) => {
|
||||
const id = getNanoid();
|
||||
inputIdMap.set(key, id);
|
||||
return {
|
||||
id,
|
||||
key,
|
||||
valueType: getType(properties[key]),
|
||||
label: key,
|
||||
type: FlowNodeOutputTypeEnum.source,
|
||||
edit: true
|
||||
};
|
||||
}) || [])
|
||||
];
|
||||
|
||||
const httpInputs: FlowNodeInputItemType[] = [
|
||||
...(item.params?.map((param: any) => {
|
||||
return {
|
||||
key: param.name,
|
||||
valueType: getType(param.schema),
|
||||
label: param.name,
|
||||
renderTypeList: [FlowNodeInputTypeEnum.reference],
|
||||
canEdit: true,
|
||||
value: [pluginInputId, inputIdMap.get(param.name)]
|
||||
};
|
||||
}) || []),
|
||||
...(propsKeys?.map((key) => {
|
||||
return {
|
||||
key,
|
||||
valueType: getType(properties[key]),
|
||||
label: key,
|
||||
renderTypeList: [FlowNodeInputTypeEnum.reference],
|
||||
canEdit: true,
|
||||
value: [pluginInputId, inputIdMap.get(key)]
|
||||
};
|
||||
}) || [])
|
||||
];
|
||||
|
||||
/* http node setting */
|
||||
const httpNodeParams: HttpParamAndHeaderItemType[] = [];
|
||||
const httpNodeHeaders: HttpParamAndHeaderItemType[] = [];
|
||||
let httpNodeBody = '{}';
|
||||
const requestUrl = `${baseUrl}${item.path}`;
|
||||
|
||||
if (item.params && item.params.length > 0) {
|
||||
for (const param of item.params) {
|
||||
if (param.in === 'header') {
|
||||
httpNodeHeaders.push({
|
||||
key: param.name,
|
||||
type: getType(param.schema) || WorkflowIOValueTypeEnum.string,
|
||||
value: `{{${param.name}}}`
|
||||
});
|
||||
} else if (param.in === 'body') {
|
||||
httpNodeBody = JSON.stringify(
|
||||
{ ...JSON.parse(httpNodeBody), [param.name]: `{{${param.name}}}` },
|
||||
null,
|
||||
2
|
||||
);
|
||||
} else if (param.in === 'query') {
|
||||
httpNodeParams.push({
|
||||
key: param.name,
|
||||
type: getType(param.schema) || WorkflowIOValueTypeEnum.string,
|
||||
value: `{{${param.name}}}`
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
if (item.request) {
|
||||
const properties = item.request?.content?.['application/json']?.schema?.properties || {};
|
||||
const keys = Object.keys(properties);
|
||||
if (keys.length > 0) {
|
||||
httpNodeBody = JSON.stringify(
|
||||
keys.reduce((acc: any, key) => {
|
||||
acc[key] = `{{${key}}}`;
|
||||
return acc;
|
||||
}, {}),
|
||||
null,
|
||||
2
|
||||
);
|
||||
}
|
||||
}
|
||||
if (customHeader) {
|
||||
const headersObj = (() => {
|
||||
try {
|
||||
return JSON.parse(customHeader) as Record<string, string>;
|
||||
} catch (err) {
|
||||
return {};
|
||||
}
|
||||
})();
|
||||
for (const key in headersObj) {
|
||||
httpNodeHeaders.push({
|
||||
key,
|
||||
type: WorkflowIOValueTypeEnum.string,
|
||||
// @ts-ignore
|
||||
value: headersObj[key]
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/* Combine complete modules */
|
||||
const nodes: StoreNodeItemType[] = [
|
||||
{
|
||||
nodeId: pluginInputId,
|
||||
name: PluginInputModule.name,
|
||||
intro: PluginInputModule.intro,
|
||||
avatar: PluginInputModule.avatar,
|
||||
flowNodeType: PluginInputModule.flowNodeType,
|
||||
showStatus: PluginInputModule.showStatus,
|
||||
position: {
|
||||
x: 473.55206291900333,
|
||||
y: -145.65080850146154
|
||||
},
|
||||
version: PluginInputModule.version,
|
||||
inputs: pluginInputs,
|
||||
outputs: pluginOutputs
|
||||
},
|
||||
{
|
||||
nodeId: pluginOutputId,
|
||||
name: PluginOutputModule.name,
|
||||
intro: PluginOutputModule.intro,
|
||||
avatar: PluginOutputModule.avatar,
|
||||
flowNodeType: PluginOutputModule.flowNodeType,
|
||||
showStatus: PluginOutputModule.showStatus,
|
||||
position: {
|
||||
x: 1847.5956872650024,
|
||||
y: 5.114324648101558
|
||||
},
|
||||
version: PluginOutputModule.version,
|
||||
inputs: [
|
||||
{
|
||||
key: pluginOutputKey,
|
||||
valueType: WorkflowIOValueTypeEnum.string,
|
||||
label: pluginOutputKey,
|
||||
renderTypeList: [FlowNodeInputTypeEnum.reference],
|
||||
required: false,
|
||||
description: '',
|
||||
canEdit: true,
|
||||
value: [httpId, 'httpRawResponse']
|
||||
}
|
||||
],
|
||||
outputs: [
|
||||
{
|
||||
id: pluginOutputId,
|
||||
key: pluginOutputKey,
|
||||
valueType: WorkflowIOValueTypeEnum.string,
|
||||
label: pluginOutputKey,
|
||||
type: FlowNodeOutputTypeEnum.static
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
nodeId: httpId,
|
||||
name: HttpNode468.name,
|
||||
intro: HttpNode468.intro,
|
||||
avatar: HttpNode468.avatar,
|
||||
flowNodeType: HttpNode468.flowNodeType,
|
||||
showStatus: true,
|
||||
position: {
|
||||
x: 1188.947986995841,
|
||||
y: -473.52694296182904
|
||||
},
|
||||
version: HttpNode468.version,
|
||||
inputs: [
|
||||
HttpNode468.inputs[0],
|
||||
...httpInputs,
|
||||
{
|
||||
key: 'system_httpMethod',
|
||||
renderTypeList: [FlowNodeInputTypeEnum.custom],
|
||||
valueType: WorkflowIOValueTypeEnum.string,
|
||||
label: '',
|
||||
value: item.method.toUpperCase(),
|
||||
required: true
|
||||
},
|
||||
{
|
||||
key: 'system_httpReqUrl',
|
||||
renderTypeList: [FlowNodeInputTypeEnum.hidden],
|
||||
valueType: WorkflowIOValueTypeEnum.string,
|
||||
label: '',
|
||||
description: 'core.module.input.description.Http Request Url',
|
||||
placeholder: 'https://api.ai.com/getInventory',
|
||||
required: false,
|
||||
value: requestUrl
|
||||
},
|
||||
{
|
||||
key: 'system_httpHeader',
|
||||
renderTypeList: [FlowNodeInputTypeEnum.custom],
|
||||
valueType: WorkflowIOValueTypeEnum.any,
|
||||
value: httpNodeHeaders,
|
||||
label: '',
|
||||
description: 'core.module.input.description.Http Request Header',
|
||||
placeholder: 'core.module.input.description.Http Request Header',
|
||||
required: false
|
||||
},
|
||||
{
|
||||
key: 'system_httpParams',
|
||||
renderTypeList: [FlowNodeInputTypeEnum.hidden],
|
||||
valueType: WorkflowIOValueTypeEnum.any,
|
||||
value: httpNodeParams,
|
||||
label: '',
|
||||
required: false
|
||||
},
|
||||
{
|
||||
key: 'system_httpJsonBody',
|
||||
renderTypeList: [FlowNodeInputTypeEnum.hidden],
|
||||
valueType: WorkflowIOValueTypeEnum.any,
|
||||
value: httpNodeBody,
|
||||
label: '',
|
||||
required: false
|
||||
}
|
||||
],
|
||||
outputs: HttpNode468.outputs
|
||||
}
|
||||
];
|
||||
|
||||
const edges: StoreEdgeItemType[] = [
|
||||
{
|
||||
source: pluginInputId,
|
||||
target: httpId,
|
||||
sourceHandle: getHandleId(pluginInputId, 'source', 'right'),
|
||||
targetHandle: getHandleId(httpId, 'target', 'left')
|
||||
},
|
||||
{
|
||||
source: httpId,
|
||||
target: pluginOutputId,
|
||||
sourceHandle: getHandleId(httpId, 'source', 'right'),
|
||||
targetHandle: getHandleId(pluginOutputId, 'target', 'left')
|
||||
}
|
||||
];
|
||||
|
||||
return {
|
||||
name: item.name,
|
||||
avatar: HttpImgUrl,
|
||||
intro: item.description,
|
||||
parentId,
|
||||
type: AppTypeEnum.plugin,
|
||||
modules: nodes,
|
||||
edges,
|
||||
pluginData: {
|
||||
pluginUniId: item.name
|
||||
}
|
||||
};
|
||||
});
|
||||
};
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
export type PathDataType = {
|
||||
type PathDataType = {
|
||||
name: string;
|
||||
description: string;
|
||||
method: string;
|
||||
|
|
@ -0,0 +1,174 @@
|
|||
import { getNanoid } from '../../../common/string/tools';
|
||||
import type { PathDataType } from './type';
|
||||
import { type RuntimeNodeItemType } from '../../workflow/runtime/type';
|
||||
import { FlowNodeOutputTypeEnum, FlowNodeTypeEnum } from '../../workflow/node/constant';
|
||||
import { type HttpToolConfigType } from '../type';
|
||||
import { PluginSourceEnum } from '../plugin/constants';
|
||||
import { jsonSchema2NodeInput, jsonSchema2NodeOutput } from '../jsonschema';
|
||||
import { type StoreSecretValueType } from '../../../common/secret/type';
|
||||
import { type JsonSchemaPropertiesItemType } from '../jsonschema';
|
||||
import { NodeOutputKeyEnum, WorkflowIOValueTypeEnum } from '../../workflow/constants';
|
||||
import { i18nT } from '../../../../web/i18n/utils';
|
||||
|
||||
export const getHTTPToolSetRuntimeNode = ({
|
||||
name,
|
||||
avatar,
|
||||
baseUrl = '',
|
||||
customHeaders = '',
|
||||
apiSchemaStr = '',
|
||||
toolList = [],
|
||||
headerSecret
|
||||
}: {
|
||||
name?: string;
|
||||
avatar?: string;
|
||||
baseUrl?: string;
|
||||
customHeaders?: string;
|
||||
apiSchemaStr?: string;
|
||||
toolList?: HttpToolConfigType[];
|
||||
headerSecret?: StoreSecretValueType;
|
||||
}): RuntimeNodeItemType => {
|
||||
return {
|
||||
nodeId: getNanoid(16),
|
||||
flowNodeType: FlowNodeTypeEnum.toolSet,
|
||||
avatar,
|
||||
intro: 'HTTP Tools',
|
||||
toolConfig: {
|
||||
httpToolSet: {
|
||||
baseUrl,
|
||||
toolList,
|
||||
headerSecret,
|
||||
customHeaders,
|
||||
apiSchemaStr,
|
||||
toolId: ''
|
||||
}
|
||||
},
|
||||
inputs: [],
|
||||
outputs: [],
|
||||
name: name || '',
|
||||
version: ''
|
||||
};
|
||||
};
|
||||
|
||||
export const getHTTPToolRuntimeNode = ({
|
||||
tool,
|
||||
nodeId,
|
||||
avatar = 'core/app/type/httpToolsFill',
|
||||
parentId
|
||||
}: {
|
||||
tool: Omit<HttpToolConfigType, 'path' | 'method'>;
|
||||
nodeId?: string;
|
||||
avatar?: string;
|
||||
parentId: string;
|
||||
}): RuntimeNodeItemType => {
|
||||
return {
|
||||
nodeId: nodeId || getNanoid(16),
|
||||
flowNodeType: FlowNodeTypeEnum.tool,
|
||||
avatar,
|
||||
intro: tool.description,
|
||||
toolConfig: {
|
||||
httpTool: {
|
||||
toolId: `${PluginSourceEnum.http}-${parentId}/${tool.name}`
|
||||
}
|
||||
},
|
||||
inputs: jsonSchema2NodeInput(tool.inputSchema),
|
||||
outputs: [
|
||||
...jsonSchema2NodeOutput(tool.outputSchema),
|
||||
{
|
||||
id: NodeOutputKeyEnum.rawResponse,
|
||||
key: NodeOutputKeyEnum.rawResponse,
|
||||
required: true,
|
||||
label: i18nT('workflow:raw_response'),
|
||||
description: i18nT('workflow:tool_raw_response_description'),
|
||||
valueType: WorkflowIOValueTypeEnum.any,
|
||||
type: FlowNodeOutputTypeEnum.static
|
||||
}
|
||||
],
|
||||
name: tool.name,
|
||||
version: ''
|
||||
};
|
||||
};
|
||||
|
||||
export const pathData2ToolList = async (
|
||||
pathData: PathDataType[]
|
||||
): Promise<HttpToolConfigType[]> => {
|
||||
try {
|
||||
return pathData.map((pathItem) => {
|
||||
const inputProperties: Record<string, JsonSchemaPropertiesItemType> = {};
|
||||
const inputRequired: string[] = [];
|
||||
const outputProperties: Record<string, JsonSchemaPropertiesItemType> = {};
|
||||
const outputRequired: string[] = [];
|
||||
|
||||
if (pathItem.params && Array.isArray(pathItem.params)) {
|
||||
pathItem.params.forEach((param) => {
|
||||
if (param.name && param.schema) {
|
||||
inputProperties[param.name] = {
|
||||
type: param.schema.type || 'any',
|
||||
description: param.description || ''
|
||||
};
|
||||
|
||||
if (param.required) {
|
||||
inputRequired.push(param.name);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
if (pathItem.request?.content?.['application/json']?.schema) {
|
||||
const requestSchema = pathItem.request.content['application/json'].schema;
|
||||
|
||||
if (requestSchema.properties) {
|
||||
Object.entries(requestSchema.properties).forEach(([key, value]: [string, any]) => {
|
||||
inputProperties[key] = {
|
||||
type: value.type || 'any',
|
||||
description: value.description || ''
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
if (requestSchema.required && Array.isArray(requestSchema.required)) {
|
||||
inputRequired.push(...requestSchema.required);
|
||||
}
|
||||
}
|
||||
|
||||
const responseToProcess =
|
||||
pathItem.response?.['200'] ||
|
||||
pathItem.response?.['201'] ||
|
||||
pathItem.response?.['202'] ||
|
||||
pathItem.response?.default;
|
||||
|
||||
if (responseToProcess?.content?.['application/json']?.schema) {
|
||||
const responseSchema = responseToProcess.content['application/json'].schema;
|
||||
if (responseSchema.properties) {
|
||||
Object.entries(responseSchema.properties).forEach(([key, value]: [string, any]) => {
|
||||
outputProperties[key] = {
|
||||
type: value.type || 'any',
|
||||
description: value.description || ''
|
||||
};
|
||||
});
|
||||
}
|
||||
if (responseSchema.required && Array.isArray(responseSchema.required)) {
|
||||
outputRequired.push(...responseSchema.required);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
name: pathItem.name,
|
||||
description: pathItem.description || pathItem.name,
|
||||
path: pathItem.path,
|
||||
method: pathItem.method?.toLowerCase(),
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: inputProperties,
|
||||
required: inputRequired
|
||||
},
|
||||
outputSchema: {
|
||||
type: 'object',
|
||||
properties: outputProperties,
|
||||
required: outputRequired
|
||||
}
|
||||
};
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error converting API schema to tool list:', error);
|
||||
return [];
|
||||
}
|
||||
};
|
||||
|
|
@ -1,10 +1,15 @@
|
|||
import { WorkflowIOValueTypeEnum } from '../workflow/constants';
|
||||
import { FlowNodeInputTypeEnum } from '../workflow/node/constant';
|
||||
import type { FlowNodeInputItemType } from '../workflow/type/io';
|
||||
import { FlowNodeInputTypeEnum, FlowNodeOutputTypeEnum } from '../workflow/node/constant';
|
||||
import type { FlowNodeInputItemType, FlowNodeOutputItemType } from '../workflow/type/io';
|
||||
import SwaggerParser from '@apidevtools/swagger-parser';
|
||||
import yaml from 'js-yaml';
|
||||
import type { OpenAPIV3 } from 'openapi-types';
|
||||
import type { OpenApiJsonSchema } from './httpTools/type';
|
||||
|
||||
type SchemaInputValueType = 'string' | 'number' | 'integer' | 'boolean' | 'array' | 'object';
|
||||
export type JsonSchemaPropertiesItemType = {
|
||||
description?: string;
|
||||
'x-tool-description'?: string;
|
||||
type: SchemaInputValueType;
|
||||
enum?: string[];
|
||||
minimum?: number;
|
||||
|
|
@ -16,6 +21,11 @@ export type JSONSchemaInputType = {
|
|||
properties?: Record<string, JsonSchemaPropertiesItemType>;
|
||||
required?: string[];
|
||||
};
|
||||
export type JSONSchemaOutputType = {
|
||||
type: SchemaInputValueType;
|
||||
properties?: Record<string, JsonSchemaPropertiesItemType>;
|
||||
required?: string[];
|
||||
};
|
||||
|
||||
export const getNodeInputTypeFromSchemaInputType = ({
|
||||
type,
|
||||
|
|
@ -79,8 +89,119 @@ export const jsonSchema2NodeInput = (jsonSchema: JSONSchemaInputType): FlowNodeI
|
|||
label: key,
|
||||
valueType: getNodeInputTypeFromSchemaInputType({ type: value.type, arrayItems: value.items }),
|
||||
description: value.description,
|
||||
toolDescription: value.description || key,
|
||||
toolDescription: value['x-tool-description'] ?? value.description ?? key,
|
||||
required: jsonSchema?.required?.includes(key),
|
||||
...getNodeInputRenderTypeFromSchemaInputType(value)
|
||||
}));
|
||||
};
|
||||
export const jsonSchema2NodeOutput = (
|
||||
jsonSchema: JSONSchemaOutputType
|
||||
): FlowNodeOutputItemType[] => {
|
||||
return Object.entries(jsonSchema?.properties || {}).map(([key, value]) => ({
|
||||
id: key,
|
||||
key,
|
||||
label: key,
|
||||
required: jsonSchema?.required?.includes(key),
|
||||
type: FlowNodeOutputTypeEnum.static,
|
||||
valueType: getNodeInputTypeFromSchemaInputType({ type: value.type, arrayItems: value.items }),
|
||||
description: value.description,
|
||||
toolDescription: value['x-tool-description'] ?? value.description ?? key
|
||||
}));
|
||||
};
|
||||
export const str2OpenApiSchema = async (yamlStr = ''): Promise<OpenApiJsonSchema> => {
|
||||
try {
|
||||
const data = (() => {
|
||||
try {
|
||||
return JSON.parse(yamlStr);
|
||||
} catch (jsonError) {
|
||||
return yaml.load(yamlStr, { schema: yaml.FAILSAFE_SCHEMA });
|
||||
}
|
||||
})();
|
||||
const jsonSchema = (await SwaggerParser.dereference(data)) as OpenAPIV3.Document;
|
||||
|
||||
const serverPath = (() => {
|
||||
if (jsonSchema.servers && jsonSchema.servers.length > 0) {
|
||||
return jsonSchema.servers[0].url || '';
|
||||
}
|
||||
if (data.host || data.basePath) {
|
||||
const scheme = data.schemes && data.schemes.length > 0 ? data.schemes[0] : 'https';
|
||||
const host = data.host || '';
|
||||
const basePath = data.basePath || '';
|
||||
return `${scheme}://${host}${basePath}`;
|
||||
}
|
||||
return '';
|
||||
})();
|
||||
|
||||
const pathData = Object.keys(jsonSchema.paths)
|
||||
.map((path) => {
|
||||
const methodData: any = jsonSchema.paths[path];
|
||||
return Object.keys(methodData)
|
||||
.filter((method) =>
|
||||
['get', 'post', 'put', 'delete', 'patch'].includes(method.toLocaleLowerCase())
|
||||
)
|
||||
.map((method) => {
|
||||
const methodInfo = methodData[method];
|
||||
if (methodInfo.deprecated) return;
|
||||
|
||||
const requestBody = (() => {
|
||||
if (methodInfo?.requestBody) {
|
||||
return methodInfo.requestBody;
|
||||
}
|
||||
if (methodInfo.parameters) {
|
||||
const bodyParam = methodInfo.parameters.find(
|
||||
(param: OpenAPIV3.ParameterObject) => param.in === 'body'
|
||||
);
|
||||
if (bodyParam) {
|
||||
return {
|
||||
content: {
|
||||
'application/json': {
|
||||
schema: bodyParam.schema
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
})();
|
||||
|
||||
const result = {
|
||||
path,
|
||||
method,
|
||||
name: methodInfo.operationId || path,
|
||||
description: methodInfo.description || methodInfo.summary,
|
||||
params: methodInfo.parameters,
|
||||
request: requestBody,
|
||||
response: methodInfo.responses
|
||||
};
|
||||
return result;
|
||||
});
|
||||
})
|
||||
.flat()
|
||||
.filter(Boolean) as OpenApiJsonSchema['pathData'];
|
||||
return { pathData, serverPath };
|
||||
} catch (err) {
|
||||
throw new Error('Invalid Schema');
|
||||
}
|
||||
};
|
||||
export const getSchemaValueType = (schema: { type: string; items?: { type: string } }) => {
|
||||
const typeMap: { [key: string]: WorkflowIOValueTypeEnum } = {
|
||||
string: WorkflowIOValueTypeEnum.arrayString,
|
||||
number: WorkflowIOValueTypeEnum.arrayNumber,
|
||||
integer: WorkflowIOValueTypeEnum.arrayNumber,
|
||||
boolean: WorkflowIOValueTypeEnum.arrayBoolean,
|
||||
object: WorkflowIOValueTypeEnum.arrayObject
|
||||
};
|
||||
|
||||
if (schema?.type === 'integer') {
|
||||
return WorkflowIOValueTypeEnum.number;
|
||||
}
|
||||
|
||||
if (schema?.type === 'array' && schema?.items) {
|
||||
const itemType = typeMap[schema.items.type];
|
||||
if (itemType) {
|
||||
return itemType;
|
||||
}
|
||||
}
|
||||
|
||||
return schema?.type as WorkflowIOValueTypeEnum;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,14 +1,6 @@
|
|||
import {
|
||||
NodeInputKeyEnum,
|
||||
NodeOutputKeyEnum,
|
||||
WorkflowIOValueTypeEnum
|
||||
} from '../../workflow/constants';
|
||||
import { NodeOutputKeyEnum, WorkflowIOValueTypeEnum } from '../../workflow/constants';
|
||||
import { i18nT } from '../../../../web/i18n/utils';
|
||||
import {
|
||||
FlowNodeInputTypeEnum,
|
||||
FlowNodeOutputTypeEnum,
|
||||
FlowNodeTypeEnum
|
||||
} from '../../workflow/node/constant';
|
||||
import { FlowNodeOutputTypeEnum, FlowNodeTypeEnum } from '../../workflow/node/constant';
|
||||
import { type McpToolConfigType } from '../type';
|
||||
import { type RuntimeNodeItemType } from '../../workflow/runtime/type';
|
||||
import { type StoreSecretValueType } from '../../../common/secret/type';
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ export enum PluginSourceEnum {
|
|||
systemTool = 'systemTool', // FastGPT-plugin tools, pure code.
|
||||
commercial = 'commercial', // configured in Pro, with associatedPluginId. Specially, commercial-dalle3 is a systemTool
|
||||
mcp = 'mcp', // mcp
|
||||
http = 'http', // http
|
||||
// @deprecated
|
||||
community = 'community' // this is deprecated, will be replaced by systemTool
|
||||
}
|
||||
|
|
|
|||
|
|
@ -56,6 +56,9 @@ export type SystemPluginTemplateItemType = WorkflowTemplateType & {
|
|||
inputList?: FlowNodeInputItemType['inputList'];
|
||||
inputListVal?: Record<string, any>;
|
||||
hasSystemSecret?: boolean;
|
||||
|
||||
// Plugin source type
|
||||
toolSource?: 'uploaded' | 'built-in';
|
||||
};
|
||||
|
||||
export type SystemPluginTemplateListItemType = Omit<
|
||||
|
|
|
|||
|
|
@ -60,5 +60,11 @@ export function splitCombinePluginId(id: string) {
|
|||
pluginId
|
||||
};
|
||||
}
|
||||
if (source === 'http') {
|
||||
return {
|
||||
source: PluginSourceEnum.http,
|
||||
pluginId
|
||||
};
|
||||
}
|
||||
return { source, pluginId: id };
|
||||
}
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ import type { ParentIdType } from '../../common/parentFolder/type';
|
|||
import { FlowNodeInputTypeEnum } from '../../core/workflow/node/constant';
|
||||
import type { WorkflowTemplateBasicType } from '@fastgpt/global/core/workflow/type';
|
||||
import type { SourceMemberType } from '../../support/user/type';
|
||||
import type { JSONSchemaInputType } from './jsonschema';
|
||||
import type { JSONSchemaInputType, JSONSchemaOutputType } from './jsonschema';
|
||||
|
||||
export type AppSchema = {
|
||||
_id: string;
|
||||
|
|
@ -120,6 +120,15 @@ export type McpToolConfigType = {
|
|||
inputSchema: JSONSchemaInputType;
|
||||
};
|
||||
|
||||
export type HttpToolConfigType = {
|
||||
name: string;
|
||||
description: string;
|
||||
inputSchema: JSONSchemaInputType;
|
||||
outputSchema: JSONSchemaOutputType;
|
||||
path: string;
|
||||
method: string;
|
||||
};
|
||||
|
||||
/* app chat config type */
|
||||
export type AppChatConfigType = {
|
||||
welcomeText?: string;
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@ import type { WorkflowInteractiveResponseType } from '../workflow/template/syste
|
|||
import type { FlowNodeInputItemType } from '../workflow/type/io';
|
||||
import type { FlowNodeTemplateType } from '../workflow/type/node.d';
|
||||
|
||||
/* --------- chat ---------- */
|
||||
export type ChatSchemaType = {
|
||||
_id: string;
|
||||
chatId: string;
|
||||
|
|
@ -49,6 +50,7 @@ export type ChatWithAppSchema = Omit<ChatSchemaType, 'appId'> & {
|
|||
appId: AppSchema;
|
||||
};
|
||||
|
||||
/* --------- chat item ---------- */
|
||||
export type UserChatItemValueItemType = {
|
||||
type: ChatItemValueTypeEnum.text | ChatItemValueTypeEnum.file;
|
||||
text?: {
|
||||
|
|
@ -65,6 +67,7 @@ export type UserChatItemType = {
|
|||
value: UserChatItemValueItemType[];
|
||||
hideInUI?: boolean;
|
||||
};
|
||||
|
||||
export type SystemChatItemValueItemType = {
|
||||
type: ChatItemValueTypeEnum.text;
|
||||
text?: {
|
||||
|
|
@ -92,7 +95,6 @@ export type AIChatItemValueItemType = {
|
|||
tools?: ToolModuleResponseItemType[];
|
||||
interactive?: WorkflowInteractiveResponseType;
|
||||
};
|
||||
|
||||
export type AIChatItemType = {
|
||||
obj: ChatRoleEnum.AI;
|
||||
value: AIChatItemValueItemType[];
|
||||
|
|
@ -101,14 +103,22 @@ export type AIChatItemType = {
|
|||
userBadFeedback?: string;
|
||||
customFeedbacks?: string[];
|
||||
adminFeedback?: AdminFbkType;
|
||||
|
||||
durationSeconds?: number;
|
||||
errorMsg?: string;
|
||||
citeCollectionIds?: string[];
|
||||
|
||||
// @deprecated 不再存储在 chatItemSchema 里，分别存储到 chatItemResponseSchema
|
||||
[DispatchNodeResponseKeyEnum.nodeResponse]?: ChatHistoryItemResType[];
|
||||
};
|
||||
|
||||
export type ChatItemValueItemType =
|
||||
| UserChatItemValueItemType
|
||||
| SystemChatItemValueItemType
|
||||
| AIChatItemValueItemType;
|
||||
export type ChatItemMergeType = UserChatItemType | SystemChatItemType | AIChatItemType;
|
||||
|
||||
export type ChatItemSchema = (UserChatItemType | SystemChatItemType | AIChatItemType) & {
|
||||
export type ChatItemSchema = ChatItemMergeType & {
|
||||
dataId: string;
|
||||
chatId: string;
|
||||
userId: string;
|
||||
|
|
@ -116,8 +126,6 @@ export type ChatItemSchema = (UserChatItemType | SystemChatItemType | AIChatItem
|
|||
tmbId: string;
|
||||
appId: string;
|
||||
time: Date;
|
||||
durationSeconds?: number;
|
||||
errorMsg?: string;
|
||||
};
|
||||
|
||||
export type AdminFbkType = {
|
||||
|
|
@ -128,7 +136,6 @@ export type AdminFbkType = {
|
|||
a?: string;
|
||||
};
|
||||
|
||||
/* --------- chat item ---------- */
|
||||
export type ResponseTagItemType = {
|
||||
totalQuoteList?: SearchDataResponseItemType[];
|
||||
llmModuleAccount?: number;
|
||||
|
|
@ -136,12 +143,12 @@ export type ResponseTagItemType = {
|
|||
toolCiteLinks?: ToolCiteLinksType[];
|
||||
};
|
||||
|
||||
export type ChatItemType = (UserChatItemType | SystemChatItemType | AIChatItemType) & {
|
||||
export type ChatItemType = ChatItemMergeType & {
|
||||
dataId?: string;
|
||||
} & ResponseTagItemType;
|
||||
|
||||
// Frontend type
|
||||
export type ChatSiteItemType = (UserChatItemType | SystemChatItemType | AIChatItemType) & {
|
||||
export type ChatSiteItemType = ChatItemMergeType & {
|
||||
_id?: string;
|
||||
dataId: string;
|
||||
status: `${ChatStatusEnum}`;
|
||||
|
|
@ -153,6 +160,16 @@ export type ChatSiteItemType = (UserChatItemType | SystemChatItemType | AIChatIt
|
|||
errorMsg?: string;
|
||||
} & ChatBoxInputType &
|
||||
ResponseTagItemType;
|
||||
|
||||
/* --------- chat item response ---------- */
|
||||
export type ChatItemResponseSchemaType = {
|
||||
teamId: string;
|
||||
appId: string;
|
||||
chatId: string;
|
||||
chatItemDataId: string;
|
||||
data: ChatHistoryItemResType;
|
||||
};
|
||||
|
||||
/* --------- team chat --------- */
|
||||
export type ChatAppListSchema = {
|
||||
apps: AppType[];
|
||||
|
|
|
|||
|
|
@ -85,10 +85,10 @@ export const getHistoryPreview = (
|
|||
|
||||
// Filter workflow public response
|
||||
export const filterPublicNodeResponseData = ({
|
||||
flowResponses = [],
|
||||
nodeRespones = [],
|
||||
responseDetail = false
|
||||
}: {
|
||||
flowResponses?: ChatHistoryItemResType[];
|
||||
nodeRespones?: ChatHistoryItemResType[];
|
||||
responseDetail?: boolean;
|
||||
}) => {
|
||||
const publicNodeMap: Record<string, any> = {
|
||||
|
|
@ -98,19 +98,28 @@ export const filterPublicNodeResponseData = ({
|
|||
[FlowNodeTypeEnum.pluginOutput]: true
|
||||
};
|
||||
|
||||
const filedList = responseDetail
|
||||
? ['quoteList', 'moduleType', 'pluginOutput', 'runningTime']
|
||||
: ['moduleType', 'pluginOutput', 'runningTime'];
|
||||
const filedMap: Record<string, boolean> = responseDetail
|
||||
? {
|
||||
quoteList: true,
|
||||
moduleType: true,
|
||||
pluginOutput: true,
|
||||
runningTime: true
|
||||
}
|
||||
: {
|
||||
moduleType: true,
|
||||
pluginOutput: true,
|
||||
runningTime: true
|
||||
};
|
||||
|
||||
return flowResponses
|
||||
return nodeRespones
|
||||
.filter((item) => publicNodeMap[item.moduleType])
|
||||
.map((item) => {
|
||||
const obj: DispatchNodeResponseType = {};
|
||||
for (let key in item) {
|
||||
if (key === 'toolDetail' || key === 'pluginDetail') {
|
||||
// @ts-ignore
|
||||
obj[key] = filterPublicNodeResponseData({ flowResponses: item[key], responseDetail });
|
||||
} else if (filedList.includes(key)) {
|
||||
obj[key] = filterPublicNodeResponseData({ nodeRespones: item[key], responseDetail });
|
||||
} else if (filedMap[key]) {
|
||||
// @ts-ignore
|
||||
obj[key] = item[key];
|
||||
}
|
||||
|
|
@ -211,38 +220,32 @@ export const mergeChatResponseData = (
|
|||
return item;
|
||||
});
|
||||
|
||||
let lastResponse: ChatHistoryItemResType | undefined = undefined;
|
||||
let hasMerged = false;
|
||||
const result: ChatHistoryItemResType[] = [];
|
||||
const mergeMap = new Map<string, number>(); // mergeSignId -> result index
|
||||
|
||||
const firstPassResult = responseWithMergedPlugins.reduce<ChatHistoryItemResType[]>(
|
||||
(acc, curr) => {
|
||||
if (
|
||||
lastResponse &&
|
||||
lastResponse.mergeSignId &&
|
||||
curr.mergeSignId === lastResponse.mergeSignId
|
||||
) {
|
||||
const concatResponse: ChatHistoryItemResType = {
|
||||
...curr,
|
||||
runningTime: +((lastResponse.runningTime || 0) + (curr.runningTime || 0)).toFixed(2),
|
||||
totalPoints: (lastResponse.totalPoints || 0) + (curr.totalPoints || 0),
|
||||
childTotalPoints: (lastResponse.childTotalPoints || 0) + (curr.childTotalPoints || 0),
|
||||
toolDetail: [...(lastResponse.toolDetail || []), ...(curr.toolDetail || [])],
|
||||
loopDetail: [...(lastResponse.loopDetail || []), ...(curr.loopDetail || [])],
|
||||
pluginDetail: [...(lastResponse.pluginDetail || []), ...(curr.pluginDetail || [])]
|
||||
};
|
||||
hasMerged = true;
|
||||
return [...acc.slice(0, -1), concatResponse];
|
||||
} else {
|
||||
lastResponse = curr;
|
||||
return [...acc, curr];
|
||||
for (const item of responseWithMergedPlugins) {
|
||||
if (item.mergeSignId && mergeMap.has(item.mergeSignId)) {
|
||||
// Merge with existing item
|
||||
const existingIndex = mergeMap.get(item.mergeSignId)!;
|
||||
const existing = result[existingIndex];
|
||||
|
||||
result[existingIndex] = {
|
||||
...item,
|
||||
runningTime: +((existing.runningTime || 0) + (item.runningTime || 0)).toFixed(2),
|
||||
totalPoints: (existing.totalPoints || 0) + (item.totalPoints || 0),
|
||||
childTotalPoints: (existing.childTotalPoints || 0) + (item.childTotalPoints || 0),
|
||||
toolDetail: [...(existing.toolDetail || []), ...(item.toolDetail || [])],
|
||||
loopDetail: [...(existing.loopDetail || []), ...(item.loopDetail || [])],
|
||||
pluginDetail: [...(existing.pluginDetail || []), ...(item.pluginDetail || [])]
|
||||
};
|
||||
} else {
|
||||
// Add new item
|
||||
result.push(item);
|
||||
if (item.mergeSignId) {
|
||||
mergeMap.set(item.mergeSignId, result.length - 1);
|
||||
}
|
||||
},
|
||||
[]
|
||||
);
|
||||
|
||||
if (hasMerged && firstPassResult.length > 1) {
|
||||
return mergeChatResponseData(firstPassResult);
|
||||
}
|
||||
}
|
||||
|
||||
return firstPassResult;
|
||||
return result;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -48,6 +48,7 @@ export type ChatDispatchProps = {
|
|||
id: string; // May be the id of the system plug-in (cannot be used directly to look up the table)
|
||||
teamId: string;
|
||||
tmbId: string; // App tmbId
|
||||
name: string;
|
||||
isChildApp?: boolean;
|
||||
};
|
||||
runningUserInfo: {
|
||||
|
|
@ -78,6 +79,7 @@ export type ChatDispatchProps = {
|
|||
|
||||
responseAllData?: boolean;
|
||||
responseDetail?: boolean;
|
||||
usageId?: string;
|
||||
};
|
||||
|
||||
export type ModuleDispatchProps<T> = ChatDispatchProps & {
|
||||
|
|
|
|||
|
|
@ -36,7 +36,8 @@ export const HttpNode468: FlowNodeTemplateType = {
|
|||
selectValueTypeList: Object.values(WorkflowIOValueTypeEnum),
|
||||
showDescription: false,
|
||||
showDefaultValue: true
|
||||
}
|
||||
},
|
||||
deprecated: true
|
||||
},
|
||||
{
|
||||
key: NodeInputKeyEnum.httpMethod,
|
||||
|
|
|
|||
|
|
@ -14,6 +14,8 @@ type InteractiveBasicType = {
|
|||
memoryMessages: ChatCompletionMessageParam[]; // 这轮工具中，产生的新的 messages
|
||||
toolCallId: string; // 记录对应 tool 的id，用于后续交互节点可以替换掉 tool 的 response
|
||||
};
|
||||
|
||||
usageId?: string;
|
||||
};
|
||||
|
||||
type InteractiveNodeType = {
|
||||
|
|
|
|||
|
|
@ -19,7 +19,12 @@ import { ChatNodeUsageType } from '../../../support/wallet/bill/type';
|
|||
import { RuntimeNodeItemType } from '../runtime/type';
|
||||
import { RuntimeEdgeItemType, StoreEdgeItemType } from './edge';
|
||||
import { NextApiResponse } from 'next';
|
||||
import type { AppDetailType, AppSchema, McpToolConfigType } from '../../app/type';
|
||||
import type {
|
||||
AppDetailType,
|
||||
AppSchema,
|
||||
McpToolConfigType,
|
||||
HttpToolConfigType
|
||||
} from '../../app/type';
|
||||
import type { ParentIdType } from 'common/parentFolder/type';
|
||||
import { AppTypeEnum } from '../../app/constants';
|
||||
import type { WorkflowInteractiveResponseType } from '../template/system/interactive/type';
|
||||
|
|
@ -46,6 +51,17 @@ export type NodeToolConfigType = {
|
|||
description: string;
|
||||
}[];
|
||||
};
|
||||
httpToolSet?: {
|
||||
toolId: string;
|
||||
baseUrl: string;
|
||||
toolList: HttpToolConfigType[];
|
||||
apiSchemaStr: string;
|
||||
customHeaders: string;
|
||||
headerSecret?: StoreSecretValueType;
|
||||
};
|
||||
httpTool?: {
|
||||
toolId: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type FlowNodeCommonType = {
|
||||
|
|
@ -142,6 +158,7 @@ export type NodeTemplateListItemType = {
|
|||
instructions?: string; // 使用说明
|
||||
courseUrl?: string; // 教程链接
|
||||
sourceMember?: SourceMember;
|
||||
toolSource?: 'uploaded' | 'built-in'; // Plugin source type
|
||||
};
|
||||
|
||||
export type NodeTemplateListType = {
|
||||
|
|
|
|||
|
|
@ -27,7 +27,8 @@ import type {
|
|||
ChatInputGuideConfigType,
|
||||
AppChatConfigType,
|
||||
AppAutoExecuteConfigType,
|
||||
AppQGConfigType
|
||||
AppQGConfigType,
|
||||
AppSchema
|
||||
} from '../app/type';
|
||||
import { type EditorVariablePickerType } from '../../../web/components/common/Textarea/PromptEditor/type';
|
||||
import {
|
||||
|
|
@ -441,3 +442,24 @@ export const getPluginRunUserQuery = ({
|
|||
})
|
||||
};
|
||||
};
|
||||
|
||||
export const removeUnauthModels = async ({
|
||||
modules,
|
||||
allowedModels = new Set()
|
||||
}: {
|
||||
modules: AppSchema['modules'];
|
||||
allowedModels?: Set<string>;
|
||||
}) => {
|
||||
if (modules) {
|
||||
modules.forEach((module) => {
|
||||
module.inputs.forEach((input) => {
|
||||
if (input.key === 'model') {
|
||||
if (!allowedModels.has(input.value)) {
|
||||
input.value = undefined;
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
return modules;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
"name": "@fastgpt/global",
|
||||
"version": "1.0.0",
|
||||
"dependencies": {
|
||||
"@fastgpt-sdk/plugin": "^0.1.16",
|
||||
"@fastgpt-sdk/plugin": "^0.1.19",
|
||||
"@apidevtools/swagger-parser": "^10.1.0",
|
||||
"@bany/curl-to-json": "^1.2.8",
|
||||
"axios": "^1.12.1",
|
||||
|
|
|
|||
|
|
@ -49,7 +49,8 @@ export const PermissionTypeMap = {
|
|||
export enum PerResourceTypeEnum {
|
||||
team = 'team',
|
||||
app = 'app',
|
||||
dataset = 'dataset'
|
||||
dataset = 'dataset',
|
||||
model = 'model'
|
||||
}
|
||||
|
||||
/* new permission */
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
import {
|
||||
NullRoleVal,
|
||||
CommonPerKeyEnum,
|
||||
CommonRoleList,
|
||||
CommonPerList,
|
||||
CommonRoleKeyEnum
|
||||
} from '../constant';
|
||||
|
||||
export const ModelDefaultRole = NullRoleVal;
|
||||
export const ModelReadPerVal = CommonPerList[CommonPerKeyEnum.read];
|
||||
export const ModelReadRolVal = CommonRoleList[CommonRoleKeyEnum.read];
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
import { Permission } from '../controller';
|
||||
|
||||
export class ModelPermission extends Permission {}
|
||||
|
|
@ -1,8 +1,8 @@
|
|||
import type { UserModelSchema } from '../user/type';
|
||||
import type { RequireOnlyOne } from '../../common/type/utils';
|
||||
import type { TeamMemberSchema } from '../user/team/type';
|
||||
import type { CommonRoleKeyEnum } from './constant';
|
||||
import { type CommonPerKeyEnum, type PerResourceTypeEnum } from './constant';
|
||||
import type { CollaboratorIdType } from './collaborator';
|
||||
|
||||
// PermissionValueType, the type of permission's value is a number, which is a bit field actually.
|
||||
// It is spired by the permission system in Linux.
|
||||
|
|
@ -63,11 +63,8 @@ export type ResourcePermissionType = {
|
|||
resourceType: ResourceType;
|
||||
permission: PermissionValueType;
|
||||
resourceId: string;
|
||||
} & RequireOnlyOne<{
|
||||
tmbId: string;
|
||||
groupId: string;
|
||||
orgId: string;
|
||||
}>;
|
||||
resourceName: string;
|
||||
} & CollaboratorIdType;
|
||||
|
||||
export type ResourcePerWithTmbWithUser = Omit<ResourcePermissionType, 'tmbId'> & {
|
||||
tmbId: TeamMemberSchema & { user: UserModelSchema };
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import type { UsageSourceEnum } from './constants';
|
||||
import type { UsageListItemCountType, UsageListItemType } from './type';
|
||||
import type { UsageItemTypeEnum, UsageSourceEnum } from './constants';
|
||||
import type { UsageItemCountType, UsageItemType, UsageListItemType, UsageSchemaType } from './type';
|
||||
|
||||
export type CreateTrainingUsageProps = {
|
||||
name: string;
|
||||
|
|
@ -22,21 +22,15 @@ export type GetUsageDashboardResponseItem = {
|
|||
totalPoints: number;
|
||||
};
|
||||
|
||||
export type ConcatUsageProps = UsageListItemCountType & {
|
||||
export type CreateUsageProps = Omit<UsageSchemaType, '_id' | 'time'>;
|
||||
export type ConcatUsageProps = {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
billId?: string;
|
||||
usageId: string;
|
||||
totalPoints: number;
|
||||
listIndex?: number;
|
||||
};
|
||||
|
||||
export type CreateUsageProps = {
|
||||
itemType: UsageItemTypeEnum;
|
||||
} & UsageItemCountType;
|
||||
export type PushUsageItemsProps = {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
appName: string;
|
||||
appId?: string;
|
||||
pluginId?: string;
|
||||
totalPoints: number;
|
||||
source: `${UsageSourceEnum}`;
|
||||
list: UsageListItemType[];
|
||||
usageId: string;
|
||||
list: UsageItemType[];
|
||||
};
|
||||
|
|
|
|||
|
|
@ -14,7 +14,8 @@ export enum UsageSourceEnum {
|
|||
pdfParse = 'pdfParse',
|
||||
mcp = 'mcp',
|
||||
evaluation = 'evaluation',
|
||||
optimize_prompt = 'optimize_prompt'
|
||||
optimize_prompt = 'optimize_prompt',
|
||||
code_copilot = 'code_copilot'
|
||||
}
|
||||
|
||||
export const UsageSourceMap = {
|
||||
|
|
@ -59,5 +60,20 @@ export const UsageSourceMap = {
|
|||
},
|
||||
[UsageSourceEnum.optimize_prompt]: {
|
||||
label: i18nT('common:support.wallet.usage.Optimize Prompt')
|
||||
},
|
||||
[UsageSourceEnum.code_copilot]: {
|
||||
label: i18nT('common:support.wallet.usage.Code Copilot')
|
||||
}
|
||||
};
|
||||
|
||||
export enum UsageItemTypeEnum {
|
||||
training_vector = 1,
|
||||
training_qa = 2,
|
||||
training_autoIndex = 3,
|
||||
training_imageIndex = 4,
|
||||
training_paragraph = 5,
|
||||
training_imageParse = 6,
|
||||
|
||||
evaluation_generateAnswer = 7,
|
||||
evaluation_answerAccuracy = 8
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,8 +1,33 @@
|
|||
import type { SourceMemberType } from '../../../support/user/type';
|
||||
import type { CreateUsageProps } from './api';
|
||||
import { UsageSourceEnum } from './constants';
|
||||
import type { UsageItemTypeEnum, UsageSourceEnum } from './constants';
|
||||
|
||||
export type UsageListItemCountType = {
|
||||
export type UsageSchemaType = {
|
||||
_id: string;
|
||||
time: Date;
|
||||
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
appName: string;
|
||||
appId?: string;
|
||||
pluginId?: string;
|
||||
totalPoints: number;
|
||||
source: `${UsageSourceEnum}`;
|
||||
|
||||
// @deprecated
|
||||
list?: UsageItemType[];
|
||||
};
|
||||
export type UsageItemSchemaType = {
|
||||
_id: string;
|
||||
teamId: string;
|
||||
usageId: string;
|
||||
name: string;
|
||||
amount: number;
|
||||
time: Date;
|
||||
itemType?: UsageItemTypeEnum; // Use in usage concat
|
||||
} & UsageItemCountType;
|
||||
|
||||
export type UsageItemCountType = {
|
||||
model?: string;
|
||||
inputTokens?: number;
|
||||
outputTokens?: number;
|
||||
charsLength?: number;
|
||||
|
|
@ -14,24 +39,18 @@ export type UsageListItemCountType = {
|
|||
tokens?: number;
|
||||
};
|
||||
|
||||
export type UsageListItemType = UsageListItemCountType & {
|
||||
export type UsageItemType = UsageItemCountType & {
|
||||
moduleName: string;
|
||||
amount: number;
|
||||
model?: string;
|
||||
count?: number;
|
||||
itemType?: UsageItemTypeEnum;
|
||||
};
|
||||
|
||||
export type UsageSchemaType = CreateUsageProps & {
|
||||
_id: string;
|
||||
time: Date;
|
||||
};
|
||||
|
||||
export type UsageItemType = {
|
||||
export type UsageListItemType = {
|
||||
id: string;
|
||||
time: Date;
|
||||
appName: string;
|
||||
source: UsageSchemaType['source'];
|
||||
totalPoints: number;
|
||||
list: UsageSchemaType['list'];
|
||||
list: Omit<UsageItemType, 'itemType'>[];
|
||||
sourceMember: SourceMemberType;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -8,7 +8,11 @@ import type {
|
|||
SearchDatasetDataResponse
|
||||
} from '../../core/dataset/search/controller';
|
||||
import type { AuthOpenApiLimitProps } from '../../support/openapi/auth';
|
||||
import type { CreateUsageProps, ConcatUsageProps } from '@fastgpt/global/support/wallet/usage/api';
|
||||
import type {
|
||||
CreateUsageProps,
|
||||
ConcatUsageProps,
|
||||
PushUsageItemsProps
|
||||
} from '@fastgpt/global/support/wallet/usage/api';
|
||||
|
||||
declare global {
|
||||
var textCensorHandler: (params: { text: string }) => Promise<{ code: number; message?: string }>;
|
||||
|
|
@ -16,4 +20,5 @@ declare global {
|
|||
var authOpenApiHandler: (data: AuthOpenApiLimitProps) => Promise<any>;
|
||||
var createUsageHandler: (data: CreateUsageProps) => any;
|
||||
var concatUsageHandler: (data: ConcatUsageProps) => any;
|
||||
var pushUsageItemsHandler: (data: PushUsageItemsProps) => any;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,60 @@
|
|||
import './init';
|
||||
import { getGlobalRedisConnection } from '../../common/redis';
|
||||
import type { SystemCacheKeyEnum } from './type';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { initCache } from './init';
|
||||
|
||||
const cachePrefix = `VERSION_KEY:`;
|
||||
|
||||
/**
|
||||
*
|
||||
* @param key SystemCacheKeyEnum
|
||||
* @param id string (teamId, tmbId, etc), if '*' is used, all keys will be refreshed
|
||||
*/
|
||||
export const refreshVersionKey = async (key: `${SystemCacheKeyEnum}`, id?: string | '*') => {
|
||||
const redis = getGlobalRedisConnection();
|
||||
if (!global.systemCache) initCache();
|
||||
|
||||
const val = randomUUID();
|
||||
const versionKey = id ? `${cachePrefix}${key}:${id}` : `${cachePrefix}${key}`;
|
||||
if (id === '*') {
|
||||
const pattern = `${cachePrefix}${key}:*`;
|
||||
const keys = await redis.keys(pattern);
|
||||
if (keys.length > 0) {
|
||||
await redis.del(keys);
|
||||
}
|
||||
} else {
|
||||
await redis.set(versionKey, val);
|
||||
}
|
||||
};
|
||||
|
||||
export const getVersionKey = async (key: `${SystemCacheKeyEnum}`, id?: string) => {
|
||||
const redis = getGlobalRedisConnection();
|
||||
if (!global.systemCache) initCache();
|
||||
|
||||
const versionKey = id ? `${cachePrefix}${key}:${id}` : `${cachePrefix}${key}`;
|
||||
const val = await redis.get(versionKey);
|
||||
if (val) return val;
|
||||
|
||||
// if there is no val set to the key, init a new val.
|
||||
const initVal = randomUUID();
|
||||
await redis.set(versionKey, initVal);
|
||||
return initVal;
|
||||
};
|
||||
|
||||
export const getCachedData = async <T extends SystemCacheKeyEnum>(key: T, id?: string) => {
|
||||
if (!global.systemCache) initCache();
|
||||
|
||||
const versionKey = await getVersionKey(key, id);
|
||||
const isDisableCache = process.env.DISABLE_CACHE === 'true';
|
||||
|
||||
// 命中缓存
|
||||
if (global.systemCache[key].versionKey === versionKey && !isDisableCache) {
|
||||
return global.systemCache[key].data;
|
||||
}
|
||||
|
||||
const refreshedData = await global.systemCache[key].refreshFunc();
|
||||
global.systemCache[key].data = refreshedData;
|
||||
global.systemCache[key].versionKey = versionKey;
|
||||
return global.systemCache[key].data;
|
||||
};
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
import { SystemCacheKeyEnum } from './type';
|
||||
import { refreshSystemTools } from '../../core/app/plugin/controller';
|
||||
|
||||
export const initCache = () => {
|
||||
global.systemCache = {
|
||||
[SystemCacheKeyEnum.systemTool]: {
|
||||
versionKey: '',
|
||||
data: [],
|
||||
refreshFunc: refreshSystemTools
|
||||
},
|
||||
[SystemCacheKeyEnum.modelPermission]: {
|
||||
versionKey: '',
|
||||
data: null,
|
||||
refreshFunc: () => Promise.resolve(null)
|
||||
}
|
||||
};
|
||||
};
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
import type { SystemPluginTemplateItemType } from '@fastgpt/global/core/app/plugin/type';
|
||||
|
||||
export enum SystemCacheKeyEnum {
|
||||
systemTool = 'systemTool',
|
||||
modelPermission = 'modelPermission'
|
||||
}
|
||||
|
||||
export type SystemCacheDataType = {
|
||||
[SystemCacheKeyEnum.systemTool]: SystemPluginTemplateItemType[];
|
||||
[SystemCacheKeyEnum.modelPermission]: null;
|
||||
};
|
||||
|
||||
type SystemCacheType = {
|
||||
[K in SystemCacheKeyEnum]: {
|
||||
versionKey: string;
|
||||
data: SystemCacheDataType[K];
|
||||
refreshFunc: () => Promise<SystemCacheDataType[K]>;
|
||||
};
|
||||
};
|
||||
|
||||
declare global {
|
||||
var systemCache: SystemCacheType;
|
||||
}
|
||||
|
|
@ -196,7 +196,8 @@ export const readFileContentFromMongo = async ({
|
|||
bucketName,
|
||||
fileId,
|
||||
customPdfParse = false,
|
||||
getFormatText
|
||||
getFormatText,
|
||||
usageId
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
|
|
@ -204,6 +205,7 @@ export const readFileContentFromMongo = async ({
|
|||
fileId: string;
|
||||
customPdfParse?: boolean;
|
||||
getFormatText?: boolean; // 数据类型都尽可能转化成 markdown 格式
|
||||
usageId?: string;
|
||||
}): Promise<{
|
||||
rawText: string;
|
||||
filename: string;
|
||||
|
|
@ -237,6 +239,7 @@ export const readFileContentFromMongo = async ({
|
|||
// Get raw text
|
||||
const { rawText } = await readRawContentByFileBuffer({
|
||||
customPdfParse,
|
||||
usageId,
|
||||
getFormatText,
|
||||
extension,
|
||||
teamId,
|
||||
|
|
|
|||
|
|
@ -47,6 +47,7 @@ export const readRawContentByFileBuffer = async ({
|
|||
encoding,
|
||||
metadata,
|
||||
customPdfParse = false,
|
||||
usageId,
|
||||
getFormatText = true
|
||||
}: {
|
||||
teamId: string;
|
||||
|
|
@ -58,6 +59,7 @@ export const readRawContentByFileBuffer = async ({
|
|||
metadata?: Record<string, any>;
|
||||
|
||||
customPdfParse?: boolean;
|
||||
usageId?: string;
|
||||
getFormatText?: boolean;
|
||||
}): Promise<{
|
||||
rawText: string;
|
||||
|
|
@ -104,7 +106,8 @@ export const readRawContentByFileBuffer = async ({
|
|||
createPdfParseUsage({
|
||||
teamId,
|
||||
tmbId,
|
||||
pages: response.pages
|
||||
pages: response.pages,
|
||||
usageId
|
||||
});
|
||||
|
||||
return {
|
||||
|
|
@ -123,7 +126,8 @@ export const readRawContentByFileBuffer = async ({
|
|||
createPdfParseUsage({
|
||||
teamId,
|
||||
tmbId,
|
||||
pages
|
||||
pages,
|
||||
usageId
|
||||
});
|
||||
|
||||
return {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,10 @@
|
|||
import type { S3ServiceConfig } from './type';
|
||||
|
||||
export const defualtS3Config: Omit<S3ServiceConfig, 'bucket'> = {
|
||||
endPoint: process.env.S3_ENDPOINT || 'localhost',
|
||||
port: process.env.S3_PORT ? parseInt(process.env.S3_PORT) : 9000,
|
||||
useSSL: process.env.S3_USE_SSL === 'true',
|
||||
accessKey: process.env.S3_ACCESS_KEY || 'minioadmin',
|
||||
secretKey: process.env.S3_SECRET_KEY || 'minioadmin',
|
||||
externalBaseURL: process.env.S3_EXTERNAL_BASE_URL
|
||||
};
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
export const mimeMap: Record<string, string> = {
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.png': 'image/png',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
'.svg': 'image/svg+xml',
|
||||
'.pdf': 'application/pdf',
|
||||
'.txt': 'text/plain',
|
||||
'.json': 'application/json',
|
||||
'.csv': 'text/csv',
|
||||
'.zip': 'application/zip',
|
||||
'.docx': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'.xlsx': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
'.pptx': 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
'.doc': 'application/msword',
|
||||
'.xls': 'application/vnd.ms-excel',
|
||||
'.ppt': 'application/vnd.ms-powerpoint',
|
||||
'.js': 'application/javascript'
|
||||
};
|
||||
|
|
@ -0,0 +1,168 @@
|
|||
import { Client } from 'minio';
|
||||
import {
|
||||
type FileMetadataType,
|
||||
type PresignedUrlInput as UploadPresignedURLProps,
|
||||
type UploadPresignedURLResponse,
|
||||
type S3ServiceConfig
|
||||
} from './type';
|
||||
import { defualtS3Config } from './config';
|
||||
import { randomBytes } from 'crypto';
|
||||
import { HttpProxyAgent } from 'http-proxy-agent';
|
||||
import { HttpsProxyAgent } from 'https-proxy-agent';
|
||||
import { extname } from 'path';
|
||||
import { addLog } from '../../common/system/log';
|
||||
import { getErrText } from '@fastgpt/global/common/error/utils';
|
||||
import { mimeMap } from './const';
|
||||
|
||||
export class S3Service {
|
||||
private client: Client;
|
||||
private config: S3ServiceConfig;
|
||||
private initialized: boolean = false;
|
||||
initFunction?: () => Promise<any>;
|
||||
|
||||
constructor(config?: Partial<S3ServiceConfig>) {
|
||||
this.config = { ...defualtS3Config, ...config } as S3ServiceConfig;
|
||||
|
||||
this.client = new Client({
|
||||
endPoint: this.config.endPoint,
|
||||
port: this.config.port,
|
||||
useSSL: this.config.useSSL,
|
||||
accessKey: this.config.accessKey,
|
||||
secretKey: this.config.secretKey,
|
||||
transportAgent: process.env.HTTP_PROXY
|
||||
? new HttpProxyAgent(process.env.HTTP_PROXY)
|
||||
: process.env.HTTPS_PROXY
|
||||
? new HttpsProxyAgent(process.env.HTTPS_PROXY)
|
||||
: undefined
|
||||
});
|
||||
|
||||
this.initFunction = config?.initFunction;
|
||||
}
|
||||
|
||||
public async init() {
|
||||
if (!this.initialized) {
|
||||
if (!(await this.client.bucketExists(this.config.bucket))) {
|
||||
addLog.debug(`Creating bucket: ${this.config.bucket}`);
|
||||
await this.client.makeBucket(this.config.bucket);
|
||||
}
|
||||
|
||||
await this.initFunction?.();
|
||||
this.initialized = true;
|
||||
}
|
||||
}
|
||||
|
||||
private generateFileId(): string {
|
||||
return randomBytes(16).toString('hex');
|
||||
}
|
||||
|
||||
private generateAccessUrl(filename: string): string {
|
||||
const protocol = this.config.useSSL ? 'https' : 'http';
|
||||
const port =
|
||||
this.config.port && this.config.port !== (this.config.useSSL ? 443 : 80)
|
||||
? `:${this.config.port}`
|
||||
: '';
|
||||
|
||||
const externalBaseURL = this.config.externalBaseURL;
|
||||
return externalBaseURL
|
||||
? `${externalBaseURL}/${this.config.bucket}/${encodeURIComponent(filename)}`
|
||||
: `${protocol}://${this.config.endPoint}${port}/${this.config.bucket}/${encodeURIComponent(filename)}`;
|
||||
}
|
||||
|
||||
uploadFile = async (fileBuffer: Buffer, originalFilename: string): Promise<FileMetadataType> => {
|
||||
await this.init();
|
||||
const inferContentType = (filename: string) => {
|
||||
const ext = extname(filename).toLowerCase();
|
||||
return mimeMap[ext] || 'application/octet-stream';
|
||||
};
|
||||
|
||||
if (this.config.maxFileSize && fileBuffer.length > this.config.maxFileSize) {
|
||||
return Promise.reject(
|
||||
`File size ${fileBuffer.length} exceeds limit ${this.config.maxFileSize}`
|
||||
);
|
||||
}
|
||||
|
||||
const fileId = this.generateFileId();
|
||||
const objectName = `${fileId}-${originalFilename}`;
|
||||
const uploadTime = new Date();
|
||||
|
||||
const contentType = inferContentType(originalFilename);
|
||||
await this.client.putObject(this.config.bucket, objectName, fileBuffer, fileBuffer.length, {
|
||||
'Content-Type': contentType,
|
||||
'Content-Disposition': `attachment; filename="${encodeURIComponent(originalFilename)}"`,
|
||||
'x-amz-meta-original-filename': encodeURIComponent(originalFilename),
|
||||
'x-amz-meta-upload-time': uploadTime.toISOString()
|
||||
});
|
||||
|
||||
const metadata: FileMetadataType = {
|
||||
fileId,
|
||||
originalFilename,
|
||||
contentType,
|
||||
size: fileBuffer.length,
|
||||
uploadTime,
|
||||
accessUrl: this.generateAccessUrl(objectName)
|
||||
};
|
||||
|
||||
return metadata;
|
||||
};
|
||||
|
||||
generateUploadPresignedURL = async ({
|
||||
filepath,
|
||||
contentType,
|
||||
metadata,
|
||||
filename
|
||||
}: UploadPresignedURLProps): Promise<UploadPresignedURLResponse> => {
|
||||
await this.init();
|
||||
const objectName = `${filepath}/${filename}`;
|
||||
|
||||
try {
|
||||
const policy = this.client.newPostPolicy();
|
||||
|
||||
policy.setBucket(this.config.bucket);
|
||||
policy.setKey(objectName);
|
||||
if (contentType) {
|
||||
policy.setContentType(contentType);
|
||||
}
|
||||
if (this.config.maxFileSize) {
|
||||
policy.setContentLengthRange(1, this.config.maxFileSize);
|
||||
}
|
||||
policy.setExpires(new Date(Date.now() + 10 * 60 * 1000)); // 10 mins
|
||||
|
||||
policy.setUserMetaData({
|
||||
'original-filename': encodeURIComponent(filename),
|
||||
'upload-time': new Date().toISOString(),
|
||||
...metadata
|
||||
});
|
||||
|
||||
const { postURL, formData } = await this.client.presignedPostPolicy(policy);
|
||||
|
||||
const response: UploadPresignedURLResponse = {
|
||||
objectName,
|
||||
uploadUrl: postURL,
|
||||
formData
|
||||
};
|
||||
|
||||
return response;
|
||||
} catch (error) {
|
||||
addLog.error('Failed to generate Upload Presigned URL', error);
|
||||
return Promise.reject(`Failed to generate Upload Presigned URL: ${getErrText(error)}`);
|
||||
}
|
||||
};
|
||||
|
||||
generateDownloadUrl = (objectName: string): string => {
|
||||
const pathParts = objectName.split('/');
|
||||
const encodedParts = pathParts.map((part) => encodeURIComponent(part));
|
||||
const encodedObjectName = encodedParts.join('/');
|
||||
return `${this.config.bucket}/${encodedObjectName}`;
|
||||
};
|
||||
|
||||
getFile = async (objectName: string): Promise<string> => {
|
||||
const stat = await this.client.statObject(this.config.bucket, objectName);
|
||||
|
||||
if (stat.size > 0) {
|
||||
const accessUrl = this.generateDownloadUrl(objectName);
|
||||
return accessUrl;
|
||||
}
|
||||
|
||||
return Promise.reject(`File ${objectName} not found`);
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
import { S3Service } from './controller';
|
||||
|
||||
export const PluginS3Service = (() => {
|
||||
if (!global.pluginS3Service) {
|
||||
global.pluginS3Service = new S3Service({
|
||||
bucket: process.env.S3_PLUGIN_BUCKET || 'fastgpt-plugin',
|
||||
maxFileSize: 50 * 1024 * 1024 // 50MB
|
||||
});
|
||||
}
|
||||
|
||||
return global.pluginS3Service;
|
||||
})();
|
||||
|
||||
declare global {
|
||||
var pluginS3Service: S3Service;
|
||||
}
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
import type { ClientOptions } from 'minio';
|
||||
|
||||
export type S3ServiceConfig = {
|
||||
bucket: string;
|
||||
externalBaseURL?: string;
|
||||
/**
|
||||
* Unit: Byte
|
||||
*/
|
||||
maxFileSize?: number;
|
||||
/**
|
||||
* for executing some init function for the s3 service
|
||||
*/
|
||||
initFunction?: () => Promise<any>;
|
||||
} & ClientOptions;
|
||||
|
||||
export type FileMetadataType = {
|
||||
fileId: string;
|
||||
originalFilename: string;
|
||||
contentType: string;
|
||||
size: number;
|
||||
uploadTime: Date;
|
||||
accessUrl: string;
|
||||
};
|
||||
|
||||
export type PresignedUrlInput = {
|
||||
filepath: string;
|
||||
filename: string;
|
||||
contentType?: string;
|
||||
metadata?: Record<string, string>;
|
||||
};
|
||||
|
||||
export type UploadPresignedURLResponse = {
|
||||
objectName: string;
|
||||
uploadUrl: string;
|
||||
formData: Record<string, string>;
|
||||
};
|
||||
|
||||
export type FileUploadInput = {
|
||||
buffer: Buffer;
|
||||
filename: string;
|
||||
};
|
||||
|
||||
export enum PluginTypeEnum {
|
||||
tool = 'tool'
|
||||
}
|
||||
|
||||
export const PluginFilePath = {
|
||||
[PluginTypeEnum.tool]: 'plugin/tools'
|
||||
};
|
||||
|
|
@ -2,3 +2,7 @@ export const FastGPTProUrl = process.env.PRO_URL ? `${process.env.PRO_URL}/api`
|
|||
export const FastGPTPluginUrl = process.env.PLUGIN_BASE_URL ? `${process.env.PLUGIN_BASE_URL}` : '';
|
||||
// @ts-ignore
|
||||
export const isFastGPTProService = () => !!global.systemConfig;
|
||||
|
||||
export const isProVersion = () => {
|
||||
return !!global.feConfigs?.isPlus;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -2,12 +2,8 @@
|
|||
import { PgVectorCtrl } from './pg';
|
||||
import { ObVectorCtrl } from './oceanbase';
|
||||
import { getVectorsByText } from '../../core/ai/embedding';
|
||||
import type {
|
||||
EmbeddingRecallCtrlProps} from './controller.d';
|
||||
import {
|
||||
type DelDatasetVectorCtrlProps,
|
||||
type InsertVectorProps
|
||||
} from './controller.d';
|
||||
import type { EmbeddingRecallCtrlProps } from './controller.d';
|
||||
import { type DelDatasetVectorCtrlProps, type InsertVectorProps } from './controller.d';
|
||||
import { type EmbeddingModelItemType } from '@fastgpt/global/core/ai/model.d';
|
||||
import { MILVUS_ADDRESS, PG_ADDRESS, OCEANBASE_ADDRESS } from './constants';
|
||||
import { MilvusCtrl } from './milvus';
|
||||
|
|
|
|||
|
|
@ -19,6 +19,8 @@ import { delay } from '@fastgpt/global/common/system/utils';
|
|||
import { pluginClient } from '../../../thirdProvider/fastgptPlugin';
|
||||
import { setCron } from '../../../common/system/cron';
|
||||
import { preloadModelProviders } from '../../../core/app/provider/controller';
|
||||
import { refreshVersionKey } from '../../../common/cache';
|
||||
import { SystemCacheKeyEnum } from '../../../common/cache/type';
|
||||
|
||||
export const loadSystemModels = async (init = false, language = 'en') => {
|
||||
const pushModel = (model: SystemModelItemType) => {
|
||||
|
|
@ -253,6 +255,7 @@ export const updatedReloadSystemModel = async () => {
|
|||
await loadSystemModels(true);
|
||||
// 2. 更新缓存（仅主节点触发）
|
||||
await updateFastGPTConfigBuffer();
|
||||
await refreshVersionKey(SystemCacheKeyEnum.modelPermission, '*');
|
||||
// 3. 延迟1秒，等待其他节点刷新
|
||||
await delay(1000);
|
||||
};
|
||||
|
|
|
|||
|
|
@ -142,8 +142,8 @@ export const createLLMResponse = async <T extends CompletionsBodyType>(
|
|||
|
||||
// Usage count
|
||||
const inputTokens =
|
||||
usage?.prompt_tokens ?? (await countGptMessagesTokens(requestBody.messages, requestBody.tools));
|
||||
const outputTokens = usage?.completion_tokens ?? (await countGptMessagesTokens(assistantMessage));
|
||||
usage?.prompt_tokens || (await countGptMessagesTokens(requestBody.messages, requestBody.tools));
|
||||
const outputTokens = usage?.completion_tokens || (await countGptMessagesTokens(assistantMessage));
|
||||
|
||||
return {
|
||||
isStreamResponse,
|
||||
|
|
@ -645,4 +645,4 @@ const createChatCompletion = async ({
|
|||
}
|
||||
return Promise.reject(error);
|
||||
}
|
||||
};
|
||||
};
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant
|
|||
import { removeImageByPath } from '../../common/file/image/controller';
|
||||
import { mongoSessionRun } from '../../common/mongo/sessionRun';
|
||||
import { MongoAppLogKeys } from './logs/logkeysSchema';
|
||||
import { MongoChatItemResponse } from '../chat/chatItemResponseSchema';
|
||||
|
||||
export const beforeUpdateAppFormat = ({ nodes }: { nodes?: StoreNodeItemType[] }) => {
|
||||
if (!nodes) return;
|
||||
|
|
@ -159,6 +160,9 @@ export const onDelOneApp = async ({
|
|||
|
||||
// Delete chats
|
||||
await deleteChatFiles({ appId });
|
||||
await MongoChatItemResponse.deleteMany({
|
||||
appId
|
||||
});
|
||||
await MongoChatItem.deleteMany({
|
||||
appId
|
||||
});
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ import {
|
|||
import { connectionMongo, getMongoModel } from '../../../common/mongo';
|
||||
import { AppCollectionName } from '../schema';
|
||||
import type { EvaluationSchemaType } from '@fastgpt/global/core/app/evaluation/type';
|
||||
import { UsageCollectionName } from '../../../support/wallet/usage/schema';
|
||||
import { UsageCollectionName } from '../../../support/wallet/usage/constants';
|
||||
const { Schema } = connectionMongo;
|
||||
|
||||
export const EvaluationCollectionName = 'eval';
|
||||
|
|
|
|||
|
|
@ -0,0 +1,58 @@
|
|||
import { type StoreSecretValueType } from '@fastgpt/global/common/secret/type';
|
||||
import { getSecretValue } from '../../common/secret/utils';
|
||||
import axios from 'axios';
|
||||
import { getErrText } from '@fastgpt/global/common/error/utils';
|
||||
import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
|
||||
|
||||
export type RunHTTPToolParams = {
|
||||
baseUrl: string;
|
||||
toolPath: string;
|
||||
method: string;
|
||||
params: Record<string, any>;
|
||||
headerSecret?: StoreSecretValueType;
|
||||
customHeaders?: Record<string, string>;
|
||||
};
|
||||
|
||||
export type RunHTTPToolResult = RequireOnlyOne<{
|
||||
data?: any;
|
||||
errorMsg?: string;
|
||||
}>;
|
||||
|
||||
export async function runHTTPTool({
|
||||
baseUrl,
|
||||
toolPath,
|
||||
method = 'POST',
|
||||
params,
|
||||
headerSecret,
|
||||
customHeaders
|
||||
}: RunHTTPToolParams): Promise<RunHTTPToolResult> {
|
||||
try {
|
||||
const headers = {
|
||||
'Content-Type': 'application/json',
|
||||
...(customHeaders || {}),
|
||||
...(headerSecret ? getSecretValue({ storeSecret: headerSecret }) : {})
|
||||
};
|
||||
|
||||
const { data } = await axios({
|
||||
method: method.toUpperCase(),
|
||||
baseURL: baseUrl.startsWith('https://') ? baseUrl : `https://${baseUrl}`,
|
||||
url: toolPath,
|
||||
headers,
|
||||
data: params,
|
||||
params,
|
||||
timeout: 300000,
|
||||
httpsAgent: new (require('https').Agent)({
|
||||
rejectUnauthorized: false
|
||||
})
|
||||
});
|
||||
|
||||
return {
|
||||
data
|
||||
};
|
||||
} catch (error: any) {
|
||||
console.log(error);
|
||||
return {
|
||||
errorMsg: getErrText(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
@ -43,10 +43,13 @@ import { isProduction } from '@fastgpt/global/common/system/constants';
|
|||
import { Output_Template_Error_Message } from '@fastgpt/global/core/workflow/template/output';
|
||||
import { splitCombinePluginId } from '@fastgpt/global/core/app/plugin/utils';
|
||||
import { getMCPToolRuntimeNode } from '@fastgpt/global/core/app/mcpTools/utils';
|
||||
import { getHTTPToolRuntimeNode } from '@fastgpt/global/core/app/httpTools/utils';
|
||||
import { AppTypeEnum } from '@fastgpt/global/core/app/constants';
|
||||
import { getMCPChildren } from '../mcp';
|
||||
import { cloneDeep } from 'lodash';
|
||||
import { UserError } from '@fastgpt/global/common/error/utils';
|
||||
import { getCachedData } from '../../../common/cache';
|
||||
import { SystemCacheKeyEnum } from '../../../common/cache/type';
|
||||
|
||||
type ChildAppType = SystemPluginTemplateItemType & {
|
||||
teamId?: string;
|
||||
|
|
@ -56,6 +59,8 @@ type ChildAppType = SystemPluginTemplateItemType & {
|
|||
isLatestVersion?: boolean; // Auto computed
|
||||
};
|
||||
|
||||
export const getSystemTools = () => getCachedData(SystemCacheKeyEnum.systemTool);
|
||||
|
||||
export const getSystemPluginByIdAndVersionId = async (
|
||||
pluginId: string,
|
||||
versionId?: string
|
||||
|
|
@ -190,11 +195,11 @@ export async function getChildAppPreviewNode({
|
|||
mcpToolSet: {
|
||||
toolId: pluginId,
|
||||
toolList: children,
|
||||
url: ''
|
||||
url: '',
|
||||
headerSecret: {}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
id: String(item._id),
|
||||
teamId: String(item.teamId),
|
||||
|
|
@ -260,6 +265,45 @@ export async function getChildAppPreviewNode({
|
|||
isLatestVersion: true
|
||||
};
|
||||
}
|
||||
// http tool
|
||||
else if (source === PluginSourceEnum.http) {
|
||||
const [parentId, toolName] = pluginId.split('/');
|
||||
const item = await MongoApp.findById(parentId).lean();
|
||||
if (!item) return Promise.reject(PluginErrEnum.unExist);
|
||||
|
||||
const version = await getAppVersionById({ appId: parentId, versionId, app: item });
|
||||
const toolConfig = version.nodes[0].toolConfig?.httpToolSet;
|
||||
const tool = await (async () => {
|
||||
if (toolConfig?.toolList) {
|
||||
return toolConfig.toolList.find((item) => item.name === toolName);
|
||||
}
|
||||
return undefined;
|
||||
})();
|
||||
if (!tool) return Promise.reject(PluginErrEnum.unExist);
|
||||
return {
|
||||
avatar: item.avatar,
|
||||
id: appId,
|
||||
name: tool.name,
|
||||
templateType: FlowNodeTemplateTypeEnum.tools,
|
||||
workflow: {
|
||||
nodes: [
|
||||
getHTTPToolRuntimeNode({
|
||||
tool: {
|
||||
description: tool.description,
|
||||
inputSchema: tool.inputSchema,
|
||||
outputSchema: tool.outputSchema,
|
||||
name: `${item.name}/${tool.name}`
|
||||
},
|
||||
avatar: item.avatar,
|
||||
parentId: item._id
|
||||
})
|
||||
],
|
||||
edges: []
|
||||
},
|
||||
version: '',
|
||||
isLatestVersion: true
|
||||
};
|
||||
}
|
||||
// 1. System Tools
|
||||
// 2. System Plugins configured in Pro (has associatedPluginId)
|
||||
else {
|
||||
|
|
@ -503,97 +547,63 @@ const dbPluginFormat = (item: SystemPluginConfigSchemaType): SystemPluginTemplat
|
|||
};
|
||||
|
||||
/* FastsGPT-Pluign api: */
|
||||
function getCachedSystemPlugins() {
|
||||
if (!global.systemToolsCache) {
|
||||
global.systemToolsCache = {
|
||||
expires: 0,
|
||||
data: [] as SystemPluginTemplateItemType[]
|
||||
export const refreshSystemTools = async (): Promise<SystemPluginTemplateItemType[]> => {
|
||||
const tools = await APIGetSystemToolList();
|
||||
// 从数据库里加载插件配置进行替换
|
||||
const systemToolsArray = await MongoSystemPlugin.find({}).lean();
|
||||
const systemTools = new Map(systemToolsArray.map((plugin) => [plugin.pluginId, plugin]));
|
||||
|
||||
const formatTools = tools.map<SystemPluginTemplateItemType>((item) => {
|
||||
const dbPluginConfig = systemTools.get(item.id);
|
||||
const isFolder = tools.some((tool) => tool.parentId === item.id);
|
||||
|
||||
const versionList = (item.versionList as SystemPluginTemplateItemType['versionList']) || [];
|
||||
|
||||
return {
|
||||
id: item.id,
|
||||
parentId: item.parentId,
|
||||
isFolder,
|
||||
name: item.name,
|
||||
avatar: item.avatar,
|
||||
intro: item.description,
|
||||
toolDescription: item.toolDescription,
|
||||
author: item.author,
|
||||
courseUrl: item.courseUrl,
|
||||
instructions: dbPluginConfig?.customConfig?.userGuide,
|
||||
weight: item.weight,
|
||||
toolSource: item.toolSource || 'built-in',
|
||||
workflow: {
|
||||
nodes: [],
|
||||
edges: []
|
||||
},
|
||||
versionList,
|
||||
templateType: item.templateType,
|
||||
showStatus: true,
|
||||
isActive: dbPluginConfig?.isActive ?? item.isActive ?? true,
|
||||
inputList: item?.secretInputConfig,
|
||||
hasSystemSecret: !!dbPluginConfig?.inputListVal,
|
||||
|
||||
originCost: dbPluginConfig?.originCost ?? 0,
|
||||
currentCost: dbPluginConfig?.currentCost ?? 0,
|
||||
systemKeyCost: dbPluginConfig?.systemKeyCost ?? 0,
|
||||
hasTokenFee: dbPluginConfig?.hasTokenFee ?? false,
|
||||
pluginOrder: dbPluginConfig?.pluginOrder
|
||||
};
|
||||
}
|
||||
return global.systemToolsCache;
|
||||
}
|
||||
|
||||
const cleanSystemPluginCache = () => {
|
||||
global.systemToolsCache = undefined;
|
||||
};
|
||||
|
||||
export const refetchSystemPlugins = () => {
|
||||
const changeStream = MongoSystemPlugin.watch();
|
||||
|
||||
changeStream.on('change', () => {
|
||||
try {
|
||||
cleanSystemPluginCache();
|
||||
} catch (error) {}
|
||||
});
|
||||
};
|
||||
|
||||
export const getSystemTools = async (): Promise<SystemPluginTemplateItemType[]> => {
|
||||
if (getCachedSystemPlugins().expires > Date.now() && isProduction) {
|
||||
return getCachedSystemPlugins().data;
|
||||
} else {
|
||||
const tools = await APIGetSystemToolList();
|
||||
const dbPlugins = systemToolsArray
|
||||
.filter((item) => item.customConfig?.associatedPluginId)
|
||||
.map((item) => dbPluginFormat(item));
|
||||
|
||||
// 从数据库里加载插件配置进行替换
|
||||
const systemToolsArray = await MongoSystemPlugin.find({}).lean();
|
||||
const systemTools = new Map(systemToolsArray.map((plugin) => [plugin.pluginId, plugin]));
|
||||
const concatTools = [...formatTools, ...dbPlugins];
|
||||
concatTools.sort((a, b) => (a.pluginOrder ?? 0) - (b.pluginOrder ?? 0));
|
||||
|
||||
const formatTools = tools.map<SystemPluginTemplateItemType>((item) => {
|
||||
const dbPluginConfig = systemTools.get(item.id);
|
||||
const isFolder = tools.some((tool) => tool.parentId === item.id);
|
||||
global.systemToolsTypeCache = {};
|
||||
concatTools.forEach((item) => {
|
||||
global.systemToolsTypeCache[item.templateType] = 1;
|
||||
});
|
||||
|
||||
const versionList = (item.versionList as SystemPluginTemplateItemType['versionList']) || [];
|
||||
|
||||
return {
|
||||
id: item.id,
|
||||
parentId: item.parentId,
|
||||
isFolder,
|
||||
name: item.name,
|
||||
avatar: item.avatar,
|
||||
intro: item.description,
|
||||
toolDescription: item.toolDescription,
|
||||
author: item.author,
|
||||
courseUrl: item.courseUrl,
|
||||
instructions: dbPluginConfig?.customConfig?.userGuide,
|
||||
weight: item.weight,
|
||||
workflow: {
|
||||
nodes: [],
|
||||
edges: []
|
||||
},
|
||||
versionList,
|
||||
templateType: item.templateType,
|
||||
showStatus: true,
|
||||
isActive: dbPluginConfig?.isActive ?? item.isActive ?? true,
|
||||
inputList: item?.secretInputConfig,
|
||||
hasSystemSecret: !!dbPluginConfig?.inputListVal,
|
||||
|
||||
originCost: dbPluginConfig?.originCost ?? 0,
|
||||
currentCost: dbPluginConfig?.currentCost ?? 0,
|
||||
systemKeyCost: dbPluginConfig?.systemKeyCost ?? 0,
|
||||
hasTokenFee: dbPluginConfig?.hasTokenFee ?? false,
|
||||
pluginOrder: dbPluginConfig?.pluginOrder
|
||||
};
|
||||
});
|
||||
|
||||
// TODO: Check the app exists
|
||||
const dbPlugins = systemToolsArray
|
||||
.filter((item) => item.customConfig?.associatedPluginId)
|
||||
.map((item) => dbPluginFormat(item));
|
||||
|
||||
const concatTools = [...formatTools, ...dbPlugins];
|
||||
concatTools.sort((a, b) => (a.pluginOrder ?? 0) - (b.pluginOrder ?? 0));
|
||||
|
||||
global.systemToolsCache = {
|
||||
expires: Date.now() + 30 * 60 * 1000, // 30 minutes
|
||||
data: concatTools
|
||||
};
|
||||
|
||||
global.systemToolsTypeCache = {};
|
||||
concatTools.forEach((item) => {
|
||||
global.systemToolsTypeCache[item.templateType] = 1;
|
||||
});
|
||||
|
||||
return concatTools;
|
||||
}
|
||||
return concatTools;
|
||||
};
|
||||
|
||||
export const getSystemToolById = async (id: string): Promise<SystemPluginTemplateItemType> => {
|
||||
|
|
@ -613,11 +623,5 @@ export const getSystemToolById = async (id: string): Promise<SystemPluginTemplat
|
|||
};
|
||||
|
||||
declare global {
|
||||
var systemToolsCache:
|
||||
| {
|
||||
expires: number;
|
||||
data: SystemPluginTemplateItemType[];
|
||||
}
|
||||
| undefined;
|
||||
var systemToolsTypeCache: Record<string, 1>;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ const getAppTemplates = async () => {
|
|||
|
||||
const res = [
|
||||
...communityTemplateConfig,
|
||||
...dbTemplates.filter((t) => !isCommunityTemplate(t.templateId))
|
||||
...dbTemplates.filter((t) => isCommercialTemaplte(t.templateId))
|
||||
].sort((a, b) => (a.order ?? 0) - (b.order ?? 0));
|
||||
|
||||
return res;
|
||||
|
|
@ -60,8 +60,8 @@ export const getAppTemplatesAndLoadThem = async (refresh = false) => {
|
|||
}
|
||||
};
|
||||
|
||||
export const isCommunityTemplate = (templateId: string) => {
|
||||
return templateId.startsWith(PluginSourceEnum.community);
|
||||
export const isCommercialTemaplte = (templateId: string) => {
|
||||
return templateId.startsWith(PluginSourceEnum.commercial);
|
||||
};
|
||||
|
||||
declare global {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import type { I18nStringStrictType, ToolTypeEnum } from '@fastgpt/global/sdk/fastgpt-plugin';
|
||||
import { RunToolWithStream } from '@fastgpt/global/sdk/fastgpt-plugin';
|
||||
import { PluginSourceEnum } from '@fastgpt/global/core/app/plugin/constants';
|
||||
import { pluginClient, BASE_URL, TOKEN } from '../../../thirdProvider/fastgptPlugin';
|
||||
import { pluginClient, PLUGIN_BASE_URL, PLUGIN_TOKEN } from '../../../thirdProvider/fastgptPlugin';
|
||||
import { addLog } from '../../../common/system/log';
|
||||
import { retryFn } from '@fastgpt/global/common/system/utils';
|
||||
|
||||
|
|
@ -26,8 +26,8 @@ export async function APIGetSystemToolList() {
|
|||
}
|
||||
|
||||
const runToolInstance = new RunToolWithStream({
|
||||
baseUrl: BASE_URL,
|
||||
token: TOKEN
|
||||
baseUrl: PLUGIN_BASE_URL,
|
||||
token: PLUGIN_TOKEN
|
||||
});
|
||||
export const APIRunSystemTool = runToolInstance.run.bind(runToolInstance);
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,47 @@
|
|||
import { connectionMongo, getMongoModel } from '../../common/mongo';
|
||||
const { Schema } = connectionMongo;
|
||||
import type { ChatItemResponseSchemaType } from '@fastgpt/global/core/chat/type';
|
||||
import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
|
||||
import { AppCollectionName } from '../app/schema';
|
||||
import { ChatItemResponseCollectionName } from './constants';
|
||||
|
||||
const ChatItemResponseSchema = new Schema({
|
||||
teamId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: TeamCollectionName,
|
||||
required: true
|
||||
},
|
||||
appId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: AppCollectionName,
|
||||
required: true
|
||||
},
|
||||
chatId: {
|
||||
type: String,
|
||||
require: true
|
||||
},
|
||||
chatItemDataId: {
|
||||
type: String,
|
||||
require: true
|
||||
},
|
||||
data: {
|
||||
type: Object,
|
||||
default: {}
|
||||
},
|
||||
|
||||
time: {
|
||||
type: Date,
|
||||
default: () => new Date()
|
||||
}
|
||||
});
|
||||
|
||||
// Get response/Delete
|
||||
ChatItemResponseSchema.index({ appId: 1, chatId: 1, chatItemDataId: 1 });
|
||||
|
||||
// Clear expired response
|
||||
ChatItemResponseSchema.index({ teamId: 1, time: -1 });
|
||||
|
||||
export const MongoChatItemResponse = getMongoModel<ChatItemResponseSchemaType>(
|
||||
ChatItemResponseCollectionName,
|
||||
ChatItemResponseSchema
|
||||
);
|
||||
|
|
@ -10,8 +10,7 @@ import {
|
|||
import { AppCollectionName } from '../app/schema';
|
||||
import { userCollectionName } from '../../support/user/schema';
|
||||
import { DispatchNodeResponseKeyEnum } from '@fastgpt/global/core/workflow/runtime/constants';
|
||||
|
||||
export const ChatItemCollectionName = 'chatitems';
|
||||
import { ChatItemCollectionName } from './constants';
|
||||
|
||||
const ChatItemSchema = new Schema({
|
||||
teamId: {
|
||||
|
|
@ -35,7 +34,7 @@ const ChatItemSchema = new Schema({
|
|||
dataId: {
|
||||
type: String,
|
||||
require: true,
|
||||
default: () => getNanoid(22)
|
||||
default: () => getNanoid(24)
|
||||
},
|
||||
appId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
|
|
@ -61,6 +60,8 @@ const ChatItemSchema = new Schema({
|
|||
type: Array,
|
||||
default: []
|
||||
},
|
||||
|
||||
// Field memory
|
||||
memories: Object,
|
||||
errorMsg: String,
|
||||
userGoodFeedback: String,
|
||||
|
|
@ -75,29 +76,25 @@ const ChatItemSchema = new Schema({
|
|||
a: String
|
||||
}
|
||||
},
|
||||
[DispatchNodeResponseKeyEnum.nodeResponse]: {
|
||||
type: Array,
|
||||
default: []
|
||||
},
|
||||
durationSeconds: Number
|
||||
durationSeconds: Number,
|
||||
citeCollectionIds: [String],
|
||||
|
||||
// @deprecated
|
||||
[DispatchNodeResponseKeyEnum.nodeResponse]: Array
|
||||
});
|
||||
|
||||
try {
|
||||
ChatItemSchema.index({ dataId: 1 });
|
||||
/* delete by app;
|
||||
delete by chat id;
|
||||
get chat list;
|
||||
get chat logs;
|
||||
close custom feedback;
|
||||
*/
|
||||
ChatItemSchema.index({ appId: 1, chatId: 1, dataId: 1 });
|
||||
// timer, clear history
|
||||
ChatItemSchema.index({ teamId: 1, time: -1 });
|
||||
/*
|
||||
delete by app;
|
||||
delete by chat id;
|
||||
get chat list;
|
||||
get chat logs;
|
||||
close custom feedback;
|
||||
*/
|
||||
ChatItemSchema.index({ appId: 1, chatId: 1, dataId: 1 });
|
||||
// timer, clear history
|
||||
ChatItemSchema.index({ teamId: 1, time: -1 });
|
||||
|
||||
// Admin charts
|
||||
ChatItemSchema.index({ obj: 1, time: -1 }, { partialFilterExpression: { obj: 'Human' } });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
// Admin charts
|
||||
ChatItemSchema.index({ obj: 1, time: -1 }, { partialFilterExpression: { obj: 'Human' } });
|
||||
|
||||
export const MongoChatItem = getMongoModel<ChatItemType>(ChatItemCollectionName, ChatItemSchema);
|
||||
|
|
|
|||
|
|
@ -7,8 +7,7 @@ import {
|
|||
TeamMemberCollectionName
|
||||
} from '@fastgpt/global/support/user/team/constant';
|
||||
import { AppCollectionName } from '../app/schema';
|
||||
|
||||
export const chatCollectionName = 'chat';
|
||||
import { chatCollectionName } from './constants';
|
||||
|
||||
const ChatSchema = new Schema({
|
||||
chatId: {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
export const chatCollectionName = 'chats';
|
||||
export const ChatItemResponseCollectionName = 'chat_item_responses';
|
||||
export const ChatItemCollectionName = 'chatitems';
|
||||
|
|
@ -1,10 +1,13 @@
|
|||
import type { ChatItemType } from '@fastgpt/global/core/chat/type';
|
||||
import type { ChatHistoryItemResType, ChatItemType } from '@fastgpt/global/core/chat/type';
|
||||
import { MongoChatItem } from './chatItemSchema';
|
||||
import { addLog } from '../../common/system/log';
|
||||
import { delFileByFileIdList, getGFSCollection } from '../../common/file/gridfs/controller';
|
||||
import { BucketNameEnum } from '@fastgpt/global/common/file/constants';
|
||||
import { MongoChat } from './chatSchema';
|
||||
import { UserError } from '@fastgpt/global/common/error/utils';
|
||||
import { DispatchNodeResponseKeyEnum } from '@fastgpt/global/core/workflow/runtime/constants';
|
||||
import { ChatRoleEnum } from '@fastgpt/global/core/chat/constants';
|
||||
import { MongoChatItemResponse } from './chatItemResponseSchema';
|
||||
|
||||
export async function getChatItems({
|
||||
appId,
|
||||
|
|
@ -23,12 +26,44 @@ export async function getChatItems({
|
|||
return { histories: [], total: 0 };
|
||||
}
|
||||
|
||||
// Extend dataId
|
||||
field = `dataId ${field}`;
|
||||
|
||||
const [histories, total] = await Promise.all([
|
||||
MongoChatItem.find({ chatId, appId }, field).sort({ _id: -1 }).skip(offset).limit(limit).lean(),
|
||||
MongoChatItem.countDocuments({ chatId, appId })
|
||||
MongoChatItem.find({ appId, chatId }, field).sort({ _id: -1 }).skip(offset).limit(limit).lean(),
|
||||
MongoChatItem.countDocuments({ appId, chatId })
|
||||
]);
|
||||
histories.reverse();
|
||||
|
||||
// Add node responses field
|
||||
if (field.includes(DispatchNodeResponseKeyEnum.nodeResponse)) {
|
||||
const chatItemDataIds = histories
|
||||
.filter((item) => item.obj === ChatRoleEnum.AI && !item.responseData?.length)
|
||||
.map((item) => item.dataId);
|
||||
|
||||
const chatItemResponsesMap = await MongoChatItemResponse.find(
|
||||
{ appId, chatId, chatItemDataId: { $in: chatItemDataIds } },
|
||||
{ chatItemDataId: 1, data: 1 }
|
||||
)
|
||||
.lean()
|
||||
.then((res) => {
|
||||
const map = new Map<string, ChatHistoryItemResType[]>();
|
||||
res.forEach((item) => {
|
||||
const val = map.get(item.chatItemDataId) || [];
|
||||
val.push(item.data);
|
||||
map.set(item.chatItemDataId, val);
|
||||
});
|
||||
return map;
|
||||
});
|
||||
|
||||
histories.forEach((item) => {
|
||||
const val = chatItemResponsesMap.get(String(item.dataId));
|
||||
if (item.obj === ChatRoleEnum.AI && val) {
|
||||
item.responseData = val;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
return { histories, total };
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,11 +2,7 @@ import { addLog } from '../../common/system/log';
|
|||
import { MongoChatItem } from './chatItemSchema';
|
||||
import { MongoChat } from './chatSchema';
|
||||
import axios from 'axios';
|
||||
import {
|
||||
type AIChatItemType,
|
||||
type ChatItemType,
|
||||
type UserChatItemType
|
||||
} from '@fastgpt/global/core/chat/type';
|
||||
import { type AIChatItemType, type UserChatItemType } from '@fastgpt/global/core/chat/type';
|
||||
import { ChatItemValueTypeEnum } from '@fastgpt/global/core/chat/constants';
|
||||
|
||||
export type Metadata = {
|
||||
|
|
@ -43,18 +39,6 @@ export const pushChatLog = ({
|
|||
}
|
||||
};
|
||||
|
||||
type ChatItem = ChatItemType & {
|
||||
userGoodFeedback?: string;
|
||||
userBadFeedback?: string;
|
||||
chatId: string;
|
||||
responseData: {
|
||||
moduleType: string;
|
||||
runningTime: number; //s
|
||||
historyPreview: { obj: string; value: string }[];
|
||||
}[];
|
||||
time: Date;
|
||||
};
|
||||
|
||||
type ChatLog = {
|
||||
title: string;
|
||||
feedback: 'like' | 'dislike' | null;
|
||||
|
|
|
|||
|
|
@ -12,10 +12,11 @@ import { type AppChatConfigType } from '@fastgpt/global/core/app/type';
|
|||
import { mergeChatResponseData } from '@fastgpt/global/core/chat/utils';
|
||||
import { pushChatLog } from './pushChatLog';
|
||||
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
|
||||
import { DispatchNodeResponseKeyEnum } from '@fastgpt/global/core/workflow/runtime/constants';
|
||||
import { extractDeepestInteractive } from '@fastgpt/global/core/workflow/runtime/utils';
|
||||
import { MongoAppChatLog } from '../app/logs/chatLogsSchema';
|
||||
import { writePrimary } from '../../common/mongo/utils';
|
||||
import { MongoChatItemResponse } from './chatItemResponseSchema';
|
||||
import { chatValue2RuntimePrompt } from '@fastgpt/global/core/chat/adapt';
|
||||
|
||||
type Props = {
|
||||
chatId: string;
|
||||
|
|
@ -31,12 +32,59 @@ type Props = {
|
|||
sourceName?: string;
|
||||
shareId?: string;
|
||||
outLinkUid?: string;
|
||||
content: [UserChatItemType & { dataId?: string }, AIChatItemType & { dataId?: string }];
|
||||
userContent: UserChatItemType & { dataId?: string };
|
||||
aiContent: AIChatItemType & { dataId?: string };
|
||||
metadata?: Record<string, any>;
|
||||
durationSeconds: number; //s
|
||||
errorMsg?: string;
|
||||
};
|
||||
|
||||
const formatAiContent = ({
|
||||
aiContent,
|
||||
durationSeconds,
|
||||
errorMsg
|
||||
}: {
|
||||
aiContent: AIChatItemType & { dataId?: string };
|
||||
durationSeconds: number;
|
||||
errorMsg?: string;
|
||||
}) => {
|
||||
const { responseData, ...aiResponse } = aiContent;
|
||||
|
||||
const citeCollectionIds = new Set<string>();
|
||||
|
||||
const nodeResponses = responseData?.map((responseItem) => {
|
||||
if (responseItem.moduleType === FlowNodeTypeEnum.datasetSearchNode && responseItem.quoteList) {
|
||||
return {
|
||||
...responseItem,
|
||||
quoteList: responseItem.quoteList.map((quote) => {
|
||||
citeCollectionIds.add(quote.collectionId);
|
||||
return {
|
||||
id: quote.id,
|
||||
chunkIndex: quote.chunkIndex,
|
||||
datasetId: quote.datasetId,
|
||||
collectionId: quote.collectionId,
|
||||
sourceId: quote.sourceId,
|
||||
sourceName: quote.sourceName,
|
||||
score: quote.score
|
||||
};
|
||||
})
|
||||
};
|
||||
}
|
||||
return responseItem;
|
||||
});
|
||||
|
||||
return {
|
||||
aiResponse: {
|
||||
...aiResponse,
|
||||
durationSeconds,
|
||||
errorMsg,
|
||||
citeCollectionIds: Array.from(citeCollectionIds)
|
||||
},
|
||||
nodeResponses,
|
||||
citeCollectionIds
|
||||
};
|
||||
};
|
||||
|
||||
export async function saveChat({
|
||||
chatId,
|
||||
appId,
|
||||
|
|
@ -51,7 +99,8 @@ export async function saveChat({
|
|||
sourceName,
|
||||
shareId,
|
||||
outLinkUid,
|
||||
content,
|
||||
userContent,
|
||||
aiContent,
|
||||
durationSeconds,
|
||||
errorMsg,
|
||||
metadata = {}
|
||||
|
|
@ -81,42 +130,15 @@ export async function saveChat({
|
|||
)?.inputs;
|
||||
|
||||
// Format save chat content: Remove quote q/a
|
||||
const processedContent = content.map((item) => {
|
||||
if (item.obj === ChatRoleEnum.AI) {
|
||||
const nodeResponse = item[DispatchNodeResponseKeyEnum.nodeResponse]?.map((responseItem) => {
|
||||
if (
|
||||
responseItem.moduleType === FlowNodeTypeEnum.datasetSearchNode &&
|
||||
responseItem.quoteList
|
||||
) {
|
||||
return {
|
||||
...responseItem,
|
||||
quoteList: responseItem.quoteList.map((quote: any) => ({
|
||||
id: quote.id,
|
||||
chunkIndex: quote.chunkIndex,
|
||||
datasetId: quote.datasetId,
|
||||
collectionId: quote.collectionId,
|
||||
sourceId: quote.sourceId,
|
||||
sourceName: quote.sourceName,
|
||||
score: quote.score,
|
||||
tokens: quote.tokens
|
||||
}))
|
||||
};
|
||||
}
|
||||
return responseItem;
|
||||
});
|
||||
|
||||
return {
|
||||
...item,
|
||||
[DispatchNodeResponseKeyEnum.nodeResponse]: nodeResponse,
|
||||
durationSeconds,
|
||||
errorMsg
|
||||
};
|
||||
}
|
||||
return item;
|
||||
const { aiResponse, nodeResponses } = formatAiContent({
|
||||
aiContent,
|
||||
durationSeconds,
|
||||
errorMsg
|
||||
});
|
||||
const processedContent = [userContent, aiResponse];
|
||||
|
||||
await mongoSessionRun(async (session) => {
|
||||
const [{ _id: chatItemIdHuman }, { _id: chatItemIdAi }] = await MongoChatItem.create(
|
||||
const [{ _id: chatItemIdHuman }, { _id: chatItemIdAi, dataId }] = await MongoChatItem.create(
|
||||
processedContent.map((item) => ({
|
||||
chatId,
|
||||
teamId,
|
||||
|
|
@ -127,6 +149,20 @@ export async function saveChat({
|
|||
{ session, ordered: true, ...writePrimary }
|
||||
);
|
||||
|
||||
// Create chat item respones
|
||||
if (nodeResponses) {
|
||||
await MongoChatItemResponse.create(
|
||||
nodeResponses.map((item) => ({
|
||||
teamId,
|
||||
appId,
|
||||
chatId,
|
||||
chatItemDataId: dataId,
|
||||
data: item
|
||||
})),
|
||||
{ session, ordered: true, ...writePrimary }
|
||||
);
|
||||
}
|
||||
|
||||
await MongoChat.updateOne(
|
||||
{
|
||||
appId,
|
||||
|
|
@ -166,18 +202,15 @@ export async function saveChat({
|
|||
});
|
||||
});
|
||||
|
||||
// Create chat data log
|
||||
try {
|
||||
const userId = String(outLinkUid || tmbId);
|
||||
const now = new Date();
|
||||
const fifteenMinutesAgo = new Date(now.getTime() - 15 * 60 * 1000);
|
||||
|
||||
const aiResponse = processedContent.find((item) => item.obj === ChatRoleEnum.AI);
|
||||
const errorCount = aiResponse?.responseData?.some((item) => item.errorText) ? 1 : 0;
|
||||
const errorCount = nodeResponses?.some((item) => item.errorText) ? 1 : 0;
|
||||
const totalPoints =
|
||||
aiResponse?.responseData?.reduce(
|
||||
(sum: number, item: any) => sum + (item.totalPoints || 0),
|
||||
0
|
||||
) || 0;
|
||||
nodeResponses?.reduce((sum: number, item: any) => sum + (item.totalPoints || 0), 0) || 0;
|
||||
|
||||
const hasHistoryChat = await MongoAppChatLog.exists({
|
||||
teamId,
|
||||
|
|
@ -242,20 +275,16 @@ export async function saveChat({
|
|||
}
|
||||
|
||||
export const updateInteractiveChat = async ({
|
||||
teamId,
|
||||
chatId,
|
||||
|
||||
appId,
|
||||
userInteractiveVal,
|
||||
aiResponse,
|
||||
newVariables,
|
||||
durationSeconds
|
||||
}: {
|
||||
chatId: string;
|
||||
appId: string;
|
||||
userInteractiveVal: string;
|
||||
aiResponse: AIChatItemType & { dataId?: string };
|
||||
newVariables?: Record<string, any>;
|
||||
durationSeconds: number;
|
||||
}) => {
|
||||
userContent,
|
||||
aiContent,
|
||||
variables,
|
||||
durationSeconds,
|
||||
errorMsg
|
||||
}: Props) => {
|
||||
if (!chatId) return;
|
||||
|
||||
const chatItem = await MongoChatItem.findOne({ appId, chatId, obj: ChatRoleEnum.AI }).sort({
|
||||
|
|
@ -276,17 +305,23 @@ export const updateInteractiveChat = async ({
|
|||
}
|
||||
|
||||
const parsedUserInteractiveVal = (() => {
|
||||
const { text: userInteractiveVal } = chatValue2RuntimePrompt(userContent.value);
|
||||
try {
|
||||
return JSON.parse(userInteractiveVal);
|
||||
} catch (err) {
|
||||
return userInteractiveVal;
|
||||
}
|
||||
})();
|
||||
const { aiResponse, nodeResponses } = formatAiContent({
|
||||
aiContent,
|
||||
durationSeconds,
|
||||
errorMsg
|
||||
});
|
||||
|
||||
let finalInteractive = extractDeepestInteractive(interactiveValue.interactive);
|
||||
|
||||
if (finalInteractive.type === 'userSelect') {
|
||||
finalInteractive.params.userSelectedVal = userInteractiveVal;
|
||||
finalInteractive.params.userSelectedVal = parsedUserInteractiveVal;
|
||||
} else if (
|
||||
finalInteractive.type === 'userInput' &&
|
||||
typeof parsedUserInteractiveVal === 'object'
|
||||
|
|
@ -308,16 +343,14 @@ export const updateInteractiveChat = async ({
|
|||
? [...chatItem.customFeedbacks, ...aiResponse.customFeedbacks]
|
||||
: aiResponse.customFeedbacks;
|
||||
}
|
||||
|
||||
if (aiResponse.responseData) {
|
||||
chatItem.responseData = chatItem.responseData
|
||||
? mergeChatResponseData([...chatItem.responseData, ...aiResponse.responseData])
|
||||
: aiResponse.responseData;
|
||||
}
|
||||
|
||||
if (aiResponse.value) {
|
||||
chatItem.value = chatItem.value ? [...chatItem.value, ...aiResponse.value] : aiResponse.value;
|
||||
}
|
||||
if (aiResponse.citeCollectionIds) {
|
||||
chatItem.citeCollectionIds = chatItem.citeCollectionIds
|
||||
? [...chatItem.citeCollectionIds, ...aiResponse.citeCollectionIds]
|
||||
: aiResponse.citeCollectionIds;
|
||||
}
|
||||
|
||||
chatItem.durationSeconds = chatItem.durationSeconds
|
||||
? +(chatItem.durationSeconds + durationSeconds).toFixed(2)
|
||||
|
|
@ -332,7 +365,7 @@ export const updateInteractiveChat = async ({
|
|||
},
|
||||
{
|
||||
$set: {
|
||||
variables: newVariables,
|
||||
variables,
|
||||
updateTime: new Date()
|
||||
}
|
||||
},
|
||||
|
|
@ -340,5 +373,36 @@ export const updateInteractiveChat = async ({
|
|||
session
|
||||
}
|
||||
);
|
||||
|
||||
// Create chat item respones
|
||||
if (nodeResponses) {
|
||||
// Merge
|
||||
const lastResponse = await MongoChatItemResponse.findOneAndDelete({
|
||||
appId,
|
||||
chatId,
|
||||
chatItemDataId: chatItem.dataId
|
||||
})
|
||||
.sort({
|
||||
_id: -1
|
||||
})
|
||||
.lean()
|
||||
.session(session);
|
||||
|
||||
const newResponses = lastResponse?.data
|
||||
? // @ts-ignore
|
||||
mergeChatResponseData([lastResponse?.data, ...nodeResponses])
|
||||
: nodeResponses;
|
||||
|
||||
await MongoChatItemResponse.create(
|
||||
newResponses.map((item) => ({
|
||||
teamId,
|
||||
appId,
|
||||
chatId,
|
||||
chatItemDataId: chatItem.dataId,
|
||||
data: item
|
||||
})),
|
||||
{ session, ordered: true, ...writePrimary }
|
||||
);
|
||||
}
|
||||
});
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,8 +1,4 @@
|
|||
import {
|
||||
DatasetCollectionTypeEnum,
|
||||
DatasetCollectionDataProcessModeEnum,
|
||||
DatasetTypeEnum
|
||||
} from '@fastgpt/global/core/dataset/constants';
|
||||
import { DatasetCollectionDataProcessModeEnum } from '@fastgpt/global/core/dataset/constants';
|
||||
import type { CreateDatasetCollectionParams } from '@fastgpt/global/core/dataset/api.d';
|
||||
import { MongoDatasetCollection } from './schema';
|
||||
import type {
|
||||
|
|
@ -25,9 +21,7 @@ import { createTrainingUsage } from '../../../support/wallet/usage/controller';
|
|||
import { UsageSourceEnum } from '@fastgpt/global/support/wallet/usage/constants';
|
||||
import { getLLMModel, getEmbeddingModel, getVlmModel } from '../../ai/model';
|
||||
import { pushDataListToTrainingQueue, pushDatasetToParseQueue } from '../training/controller';
|
||||
import { MongoImage } from '../../../common/file/image/schema';
|
||||
import { hashStr } from '@fastgpt/global/common/string/tools';
|
||||
import { addDays } from 'date-fns';
|
||||
import { MongoDatasetDataText } from '../data/dataTextSchema';
|
||||
import { retryFn } from '@fastgpt/global/common/system/utils';
|
||||
import { getTrainingModeByCollection } from './utils';
|
||||
|
|
@ -184,9 +178,9 @@ export const createCollectionAndInsertData = async ({
|
|||
});
|
||||
|
||||
// 4. create training bill
|
||||
const traingBillId = await (async () => {
|
||||
const traingUsageId = await (async () => {
|
||||
if (billId) return billId;
|
||||
const { billId: newBillId } = await createTrainingUsage({
|
||||
const { usageId: newUsageId } = await createTrainingUsage({
|
||||
teamId,
|
||||
tmbId,
|
||||
appName: formatCreateCollectionParams.name,
|
||||
|
|
@ -196,7 +190,7 @@ export const createCollectionAndInsertData = async ({
|
|||
vllmModel: getVlmModel(dataset.vlmModel)?.name,
|
||||
session
|
||||
});
|
||||
return newBillId;
|
||||
return newUsageId;
|
||||
})();
|
||||
|
||||
// 5. insert to training queue
|
||||
|
|
@ -212,7 +206,7 @@ export const createCollectionAndInsertData = async ({
|
|||
vlmModel: dataset.vlmModel,
|
||||
indexSize,
|
||||
mode: trainingMode,
|
||||
billId: traingBillId,
|
||||
billId: traingUsageId,
|
||||
data: chunks.map((item, index) => ({
|
||||
...item,
|
||||
indexes: item.indexes?.map((text) => ({
|
||||
|
|
@ -229,7 +223,7 @@ export const createCollectionAndInsertData = async ({
|
|||
tmbId,
|
||||
datasetId: dataset._id,
|
||||
collectionId,
|
||||
billId: traingBillId,
|
||||
billId: traingUsageId,
|
||||
session
|
||||
});
|
||||
return {
|
||||
|
|
|
|||
|
|
@ -156,7 +156,8 @@ export const readDatasetSourceRawText = async ({
|
|||
externalFileId,
|
||||
apiDatasetServer,
|
||||
customPdfParse,
|
||||
getFormatText
|
||||
getFormatText,
|
||||
usageId
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
|
|
@ -168,6 +169,7 @@ export const readDatasetSourceRawText = async ({
|
|||
selector?: string; // link selector
|
||||
externalFileId?: string; // external file dataset
|
||||
apiDatasetServer?: ApiDatasetServerType; // api dataset
|
||||
usageId?: string;
|
||||
}): Promise<{
|
||||
title?: string;
|
||||
rawText: string;
|
||||
|
|
@ -178,8 +180,9 @@ export const readDatasetSourceRawText = async ({
|
|||
tmbId,
|
||||
bucketName: BucketNameEnum.dataset,
|
||||
fileId: sourceId,
|
||||
getFormatText,
|
||||
customPdfParse,
|
||||
getFormatText
|
||||
usageId
|
||||
});
|
||||
return {
|
||||
title: filename,
|
||||
|
|
|
|||
|
|
@ -63,6 +63,7 @@ export const dispatchAppRequest = async (props: Props): Promise<Response> => {
|
|||
...props,
|
||||
runningAppInfo: {
|
||||
id: String(appData._id),
|
||||
name: appData.name,
|
||||
teamId: String(appData.teamId),
|
||||
tmbId: String(appData.tmbId)
|
||||
},
|
||||
|
|
|
|||
|
|
@ -48,6 +48,7 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise<
|
|||
lastInteractive,
|
||||
runningUserInfo,
|
||||
externalProvider,
|
||||
usageId,
|
||||
params: {
|
||||
model,
|
||||
systemPrompt,
|
||||
|
|
@ -117,7 +118,8 @@ export const dispatchRunTools = async (props: DispatchToolModuleProps): Promise<
|
|||
customPdfParse: chatConfig?.fileSelectConfig?.customPdfParse,
|
||||
fileLinks,
|
||||
inputFiles: globalFiles,
|
||||
hasReadFilesTool
|
||||
hasReadFilesTool,
|
||||
usageId
|
||||
});
|
||||
|
||||
const concatenateSystemPrompt = [
|
||||
|
|
@ -273,7 +275,8 @@ const getMultiInput = async ({
|
|||
maxFiles,
|
||||
customPdfParse,
|
||||
inputFiles,
|
||||
hasReadFilesTool
|
||||
hasReadFilesTool,
|
||||
usageId
|
||||
}: {
|
||||
runningUserInfo: ChatDispatchProps['runningUserInfo'];
|
||||
histories: ChatItemType[];
|
||||
|
|
@ -283,6 +286,7 @@ const getMultiInput = async ({
|
|||
customPdfParse?: boolean;
|
||||
inputFiles: UserChatItemValueItemType['file'][];
|
||||
hasReadFilesTool: boolean;
|
||||
usageId?: string;
|
||||
}) => {
|
||||
// Not file quote
|
||||
if (!fileLinks || hasReadFilesTool) {
|
||||
|
|
@ -309,6 +313,7 @@ const getMultiInput = async ({
|
|||
requestOrigin,
|
||||
maxFiles,
|
||||
customPdfParse,
|
||||
usageId,
|
||||
teamId: runningUserInfo.teamId,
|
||||
tmbId: runningUserInfo.tmbId
|
||||
});
|
||||
|
|
|
|||
|
|
@ -74,6 +74,7 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
|
|||
runningUserInfo,
|
||||
workflowStreamResponse,
|
||||
chatConfig,
|
||||
usageId,
|
||||
params: {
|
||||
model,
|
||||
temperature,
|
||||
|
|
@ -131,6 +132,7 @@ export const dispatchChatCompletion = async (props: ChatProps): Promise<ChatResp
|
|||
requestOrigin,
|
||||
maxFiles: chatConfig?.fileSelectConfig?.maxFiles || 20,
|
||||
customPdfParse: chatConfig?.fileSelectConfig?.customPdfParse,
|
||||
usageId,
|
||||
runningUserInfo
|
||||
})
|
||||
]);
|
||||
|
|
@ -313,6 +315,7 @@ async function getMultiInput({
|
|||
requestOrigin,
|
||||
maxFiles,
|
||||
customPdfParse,
|
||||
usageId,
|
||||
runningUserInfo
|
||||
}: {
|
||||
histories: ChatItemType[];
|
||||
|
|
@ -322,6 +325,7 @@ async function getMultiInput({
|
|||
requestOrigin?: string;
|
||||
maxFiles: number;
|
||||
customPdfParse?: boolean;
|
||||
usageId?: string;
|
||||
runningUserInfo: ChatDispatchProps['runningUserInfo'];
|
||||
}) {
|
||||
// 旧版本适配====>
|
||||
|
|
@ -359,9 +363,10 @@ async function getMultiInput({
|
|||
urls,
|
||||
requestOrigin,
|
||||
maxFiles,
|
||||
customPdfParse,
|
||||
teamId: runningUserInfo.teamId,
|
||||
tmbId: runningUserInfo.tmbId
|
||||
tmbId: runningUserInfo.tmbId,
|
||||
customPdfParse,
|
||||
usageId
|
||||
});
|
||||
|
||||
return {
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ import { authAppByTmbId } from '../../../../support/permission/app/auth';
|
|||
import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant';
|
||||
import { getAppVersionById } from '../../../app/version/controller';
|
||||
import { parseUrlToFileType } from '@fastgpt/global/common/file/tools';
|
||||
import { getUserChatInfoAndAuthTeamPoints } from '../../../../support/permission/auth/team';
|
||||
import { getUserChatInfo } from '../../../../support/user/team/utils';
|
||||
import { getRunningUserInfoByTmbId } from '../../../../support/user/team/utils';
|
||||
|
||||
type Props = ModuleDispatchProps<{
|
||||
|
|
@ -99,7 +99,7 @@ export const dispatchRunAppNode = async (props: Props): Promise<Response> => {
|
|||
|
||||
// Rewrite children app variables
|
||||
const systemVariables = filterSystemVariables(variables);
|
||||
const { externalProvider } = await getUserChatInfoAndAuthTeamPoints(appData.tmbId);
|
||||
const { externalProvider } = await getUserChatInfo(appData.tmbId);
|
||||
const childrenRunVariables = {
|
||||
...systemVariables,
|
||||
...childrenAppVariables,
|
||||
|
|
@ -144,6 +144,7 @@ export const dispatchRunAppNode = async (props: Props): Promise<Response> => {
|
|||
: {}),
|
||||
runningAppInfo: {
|
||||
id: String(appData._id),
|
||||
name: appData.name,
|
||||
teamId: String(appData.teamId),
|
||||
tmbId: String(appData.tmbId),
|
||||
isChildApp: true
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants';
|
|||
import { MCPClient } from '../../../app/mcp';
|
||||
import { getSecretValue } from '../../../../common/secret/utils';
|
||||
import type { McpToolDataType } from '@fastgpt/global/core/app/mcpTools/type';
|
||||
import type { HttpToolConfigType } from '@fastgpt/global/core/app/type';
|
||||
import { APIRunSystemTool } from '../../../app/tool/api';
|
||||
import { MongoSystemPlugin } from '../../../app/plugin/systemPluginSchema';
|
||||
import { SystemToolInputTypeEnum } from '@fastgpt/global/core/app/systemTool/constants';
|
||||
|
|
@ -20,6 +21,7 @@ import { pushTrack } from '../../../../common/middle/tracks/utils';
|
|||
import { getNodeErrResponse } from '../utils';
|
||||
import { splitCombinePluginId } from '@fastgpt/global/core/app/plugin/utils';
|
||||
import { getAppVersionById } from '../../../../core/app/version/controller';
|
||||
import { runHTTPTool } from '../../../app/http';
|
||||
|
||||
type SystemInputConfigType = {
|
||||
type: SystemToolInputTypeEnum;
|
||||
|
|
@ -34,7 +36,7 @@ type RunToolProps = ModuleDispatchProps<{
|
|||
|
||||
type RunToolResponse = DispatchNodeResultType<
|
||||
{
|
||||
[NodeOutputKeyEnum.rawResponse]?: any; // MCP Tool
|
||||
[NodeOutputKeyEnum.rawResponse]?: any;
|
||||
[key: string]: any;
|
||||
},
|
||||
Record<string, any>
|
||||
|
|
@ -214,6 +216,60 @@ export const dispatchRunTool = async (props: RunToolProps): Promise<RunToolRespo
|
|||
},
|
||||
[DispatchNodeResponseKeyEnum.toolResponses]: result
|
||||
};
|
||||
} else if (toolConfig?.httpTool?.toolId) {
|
||||
const { pluginId } = splitCombinePluginId(toolConfig.httpTool.toolId);
|
||||
const [parentId, toolSetName, toolName] = pluginId.split('/');
|
||||
const toolset = await getAppVersionById({
|
||||
appId: parentId,
|
||||
versionId: version
|
||||
});
|
||||
const toolSetData = toolset.nodes[0].toolConfig?.httpToolSet;
|
||||
if (!toolSetData || typeof toolSetData !== 'object') {
|
||||
throw new Error('HTTP tool set not found');
|
||||
}
|
||||
|
||||
const { headerSecret, baseUrl, toolList, customHeaders } = toolSetData;
|
||||
|
||||
const httpTool = toolList?.find((tool: HttpToolConfigType) => tool.name === toolName);
|
||||
if (!httpTool) {
|
||||
throw new Error(`HTTP tool ${toolName} not found`);
|
||||
}
|
||||
|
||||
const { data, errorMsg } = await runHTTPTool({
|
||||
baseUrl: baseUrl,
|
||||
toolPath: httpTool.path,
|
||||
method: httpTool.method,
|
||||
params,
|
||||
headerSecret,
|
||||
customHeaders: customHeaders
|
||||
? typeof customHeaders === 'string'
|
||||
? JSON.parse(customHeaders)
|
||||
: customHeaders
|
||||
: undefined
|
||||
});
|
||||
|
||||
if (errorMsg) {
|
||||
if (catchError) {
|
||||
return {
|
||||
error: { [NodeOutputKeyEnum.errorText]: errorMsg },
|
||||
[DispatchNodeResponseKeyEnum.nodeResponse]: {
|
||||
toolRes: errorMsg,
|
||||
moduleLogo: avatar
|
||||
},
|
||||
[DispatchNodeResponseKeyEnum.toolResponses]: errorMsg
|
||||
};
|
||||
}
|
||||
throw new Error(errorMsg);
|
||||
}
|
||||
|
||||
return {
|
||||
data: { [NodeOutputKeyEnum.rawResponse]: data, ...(typeof data === 'object' ? data : {}) },
|
||||
[DispatchNodeResponseKeyEnum.nodeResponse]: {
|
||||
toolRes: data,
|
||||
moduleLogo: avatar
|
||||
},
|
||||
[DispatchNodeResponseKeyEnum.toolResponses]: data
|
||||
};
|
||||
} else {
|
||||
// mcp tool (old version compatible)
|
||||
const { toolData, system_toolData, ...restParams } = params;
|
||||
|
|
|
|||
|
|
@ -15,7 +15,6 @@ import { type ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type
|
|||
import { MongoDataset } from '../../../dataset/schema';
|
||||
import { i18nT } from '../../../../../web/i18n/utils';
|
||||
import { filterDatasetsByTmbId } from '../../../dataset/utils';
|
||||
import { ModelTypeEnum } from '@fastgpt/global/core/ai/model';
|
||||
import { getDatasetSearchToolResponsePrompt } from '../../../../../global/core/ai/prompt/dataset';
|
||||
import { getNodeErrResponse } from '../utils';
|
||||
|
||||
|
|
|
|||
|
|
@ -47,8 +47,13 @@ import { rewriteRuntimeWorkFlow, removeSystemVariable } from './utils';
|
|||
import { getHandleId } from '@fastgpt/global/core/workflow/utils';
|
||||
import { callbackMap } from './constants';
|
||||
import { anyValueDecrypt } from '../../../common/secret/utils';
|
||||
import { getUserChatInfo } from '../../../support/user/team/utils';
|
||||
import { checkTeamAIPoints } from '../../../support/permission/teamLimit';
|
||||
import type { UsageSourceEnum } from '@fastgpt/global/support/wallet/usage/constants';
|
||||
import { createChatUsageRecord, pushChatItemUsage } from '../../../support/wallet/usage/controller';
|
||||
import type { RequireOnlyOne } from '@fastgpt/global/common/type/utils';
|
||||
|
||||
type Props = Omit<ChatDispatchProps, 'workflowDispatchDeep'> & {
|
||||
type Props = Omit<ChatDispatchProps, 'workflowDispatchDeep' | 'timezone' | 'externalProvider'> & {
|
||||
runtimeNodes: RuntimeNodeItemType[];
|
||||
runtimeEdges: RuntimeEdgeItemType[];
|
||||
defaultSkipNodeQueue?: WorkflowDebugResponse['skipNodeQueue'];
|
||||
|
|
@ -61,8 +66,38 @@ type NodeResponseCompleteType = Omit<NodeResponseType, 'responseData'> & {
|
|||
};
|
||||
|
||||
// Run workflow
|
||||
export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowResponse> {
|
||||
const { res, stream, externalProvider } = data;
|
||||
type WorkflowUsageProps = RequireOnlyOne<{
|
||||
usageSource: UsageSourceEnum;
|
||||
concatUsage: (points: number) => any;
|
||||
usageId: string;
|
||||
}>;
|
||||
export async function dispatchWorkFlow({
|
||||
usageSource,
|
||||
usageId,
|
||||
concatUsage,
|
||||
...data
|
||||
}: Props & WorkflowUsageProps): Promise<DispatchFlowResponse> {
|
||||
const { res, stream, runningUserInfo, runningAppInfo, lastInteractive } = data;
|
||||
|
||||
await checkTeamAIPoints(runningUserInfo.teamId);
|
||||
const [{ timezone, externalProvider }, newUsageId] = await Promise.all([
|
||||
getUserChatInfo(runningUserInfo.tmbId),
|
||||
(() => {
|
||||
if (lastInteractive?.usageId) {
|
||||
return lastInteractive.usageId;
|
||||
}
|
||||
if (usageSource) {
|
||||
return createChatUsageRecord({
|
||||
appName: runningAppInfo.name,
|
||||
appId: runningAppInfo.id,
|
||||
teamId: runningUserInfo.teamId,
|
||||
tmbId: runningUserInfo.tmbId,
|
||||
source: usageSource
|
||||
});
|
||||
}
|
||||
return usageId;
|
||||
})()
|
||||
]);
|
||||
|
||||
let streamCheckTimer: NodeJS.Timeout | null = null;
|
||||
|
||||
|
|
@ -96,15 +131,22 @@ export async function dispatchWorkFlow(data: Props): Promise<DispatchFlowRespons
|
|||
// Get default variables
|
||||
const defaultVariables = {
|
||||
...externalProvider.externalWorkflowVariables,
|
||||
...getSystemVariables(data)
|
||||
...getSystemVariables({
|
||||
...data,
|
||||
timezone
|
||||
})
|
||||
};
|
||||
|
||||
// Init some props
|
||||
return runWorkflow({
|
||||
...data,
|
||||
timezone,
|
||||
externalProvider,
|
||||
defaultSkipNodeQueue: data.lastInteractive?.skipNodeQueue || data.defaultSkipNodeQueue,
|
||||
variables: defaultVariables,
|
||||
workflowDispatchDeep: 0
|
||||
workflowDispatchDeep: 0,
|
||||
usageId: newUsageId,
|
||||
concatUsage
|
||||
}).finally(() => {
|
||||
if (streamCheckTimer) {
|
||||
clearInterval(streamCheckTimer);
|
||||
|
|
@ -116,6 +158,7 @@ type RunWorkflowProps = ChatDispatchProps & {
|
|||
runtimeNodes: RuntimeNodeItemType[];
|
||||
runtimeEdges: RuntimeEdgeItemType[];
|
||||
defaultSkipNodeQueue?: WorkflowDebugResponse['skipNodeQueue'];
|
||||
concatUsage?: (points: number) => any;
|
||||
};
|
||||
export const runWorkflow = async (data: RunWorkflowProps): Promise<DispatchFlowResponse> => {
|
||||
let {
|
||||
|
|
@ -129,7 +172,10 @@ export const runWorkflow = async (data: RunWorkflowProps): Promise<DispatchFlowR
|
|||
retainDatasetCite = true,
|
||||
version = 'v1',
|
||||
responseDetail = true,
|
||||
responseAllData = true
|
||||
responseAllData = true,
|
||||
usageId,
|
||||
concatUsage,
|
||||
runningUserInfo: { teamId }
|
||||
} = data;
|
||||
|
||||
// Over max depth
|
||||
|
|
@ -487,7 +533,7 @@ export const runWorkflow = async (data: RunWorkflowProps): Promise<DispatchFlowR
|
|||
data: responseAllData
|
||||
? formatResponseData
|
||||
: filterPublicNodeResponseData({
|
||||
flowResponses: [formatResponseData],
|
||||
nodeRespones: [formatResponseData],
|
||||
responseDetail
|
||||
})[0]
|
||||
});
|
||||
|
|
@ -573,6 +619,17 @@ export const runWorkflow = async (data: RunWorkflowProps): Promise<DispatchFlowR
|
|||
}
|
||||
|
||||
if (nodeDispatchUsages) {
|
||||
if (usageId) {
|
||||
pushChatItemUsage({
|
||||
teamId,
|
||||
usageId,
|
||||
nodeUsages: nodeDispatchUsages
|
||||
});
|
||||
}
|
||||
if (concatUsage) {
|
||||
concatUsage(nodeDispatchUsages.reduce((sum, item) => sum + (item.totalPoints || 0), 0));
|
||||
}
|
||||
|
||||
this.chatNodeUsages = this.chatNodeUsages.concat(nodeDispatchUsages);
|
||||
}
|
||||
|
||||
|
|
@ -827,7 +884,8 @@ export const runWorkflow = async (data: RunWorkflowProps): Promise<DispatchFlowR
|
|||
...edge,
|
||||
status: entryNodeIds.includes(edge.target) ? 'active' : edge.status
|
||||
})),
|
||||
nodeOutputs
|
||||
nodeOutputs,
|
||||
usageId
|
||||
};
|
||||
|
||||
// Tool call, not need interactive response
|
||||
|
|
@ -945,7 +1003,9 @@ const getSystemVariables = ({
|
|||
uid,
|
||||
chatConfig,
|
||||
variables
|
||||
}: Props): SystemVariablesType => {
|
||||
}: Props & {
|
||||
timezone: string;
|
||||
}): SystemVariablesType => {
|
||||
// Get global variables(Label -> key; Key -> key)
|
||||
const variablesConfig = chatConfig?.variables || [];
|
||||
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ import { getPluginRunUserQuery } from '@fastgpt/global/core/workflow/utils';
|
|||
import type { NodeInputKeyEnum, NodeOutputKeyEnum } from '@fastgpt/global/core/workflow/constants';
|
||||
import { getChildAppRuntimeById } from '../../../app/plugin/controller';
|
||||
import { runWorkflow } from '../index';
|
||||
import { getUserChatInfoAndAuthTeamPoints } from '../../../../support/permission/auth/team';
|
||||
import { getUserChatInfo } from '../../../../support/user/team/utils';
|
||||
import { dispatchRunTool } from '../child/runTool';
|
||||
import type { PluginRuntimeType } from '@fastgpt/global/core/app/plugin/type';
|
||||
|
||||
|
|
@ -111,7 +111,7 @@ export const dispatchRunPlugin = async (props: RunPluginProps): Promise<RunPlugi
|
|||
};
|
||||
});
|
||||
|
||||
const { externalProvider } = await getUserChatInfoAndAuthTeamPoints(runningAppInfo.tmbId);
|
||||
const { externalProvider } = await getUserChatInfo(runningAppInfo.tmbId);
|
||||
const runtimeVariables = {
|
||||
...filterSystemVariables(props.variables),
|
||||
appId: String(plugin.id),
|
||||
|
|
@ -129,6 +129,7 @@ export const dispatchRunPlugin = async (props: RunPluginProps): Promise<RunPlugi
|
|||
: {}),
|
||||
runningAppInfo: {
|
||||
id: String(plugin.id),
|
||||
name: plugin.name,
|
||||
// 如果系统插件有 teamId 和 tmbId，则使用系统插件的 teamId 和 tmbId（管理员指定了插件作为系统插件）
|
||||
teamId: plugin.teamId || runningAppInfo.teamId,
|
||||
tmbId: plugin.tmbId || runningAppInfo.tmbId,
|
||||
|
|
|
|||
|
|
@ -52,7 +52,8 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => {
|
|||
histories,
|
||||
chatConfig,
|
||||
node: { version },
|
||||
params: { fileUrlList = [] }
|
||||
params: { fileUrlList = [] },
|
||||
usageId
|
||||
} = props;
|
||||
const maxFiles = chatConfig?.fileSelectConfig?.maxFiles || 20;
|
||||
const customPdfParse = chatConfig?.fileSelectConfig?.customPdfParse || false;
|
||||
|
|
@ -68,7 +69,8 @@ export const dispatchReadFiles = async (props: Props): Promise<Response> => {
|
|||
maxFiles,
|
||||
teamId,
|
||||
tmbId,
|
||||
customPdfParse
|
||||
customPdfParse,
|
||||
usageId
|
||||
});
|
||||
|
||||
return {
|
||||
|
|
@ -119,7 +121,8 @@ export const getFileContentFromLinks = async ({
|
|||
maxFiles,
|
||||
teamId,
|
||||
tmbId,
|
||||
customPdfParse
|
||||
customPdfParse,
|
||||
usageId
|
||||
}: {
|
||||
urls: string[];
|
||||
requestOrigin?: string;
|
||||
|
|
@ -127,6 +130,7 @@ export const getFileContentFromLinks = async ({
|
|||
teamId: string;
|
||||
tmbId: string;
|
||||
customPdfParse?: boolean;
|
||||
usageId?: string;
|
||||
}) => {
|
||||
const parseUrlList = urls
|
||||
// Remove invalid urls
|
||||
|
|
@ -225,7 +229,8 @@ export const getFileContentFromLinks = async ({
|
|||
buffer,
|
||||
encoding,
|
||||
customPdfParse,
|
||||
getFormatText: true
|
||||
getFormatText: true,
|
||||
usageId
|
||||
});
|
||||
|
||||
// Add to buffer
|
||||
|
|
|
|||
|
|
@ -18,11 +18,13 @@ import {
|
|||
import { getNanoid } from '@fastgpt/global/common/string/tools';
|
||||
import { type SearchDataResponseItemType } from '@fastgpt/global/core/dataset/type';
|
||||
import { getMCPToolRuntimeNode } from '@fastgpt/global/core/app/mcpTools/utils';
|
||||
import { getHTTPToolRuntimeNode } from '@fastgpt/global/core/app/httpTools/utils';
|
||||
import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant';
|
||||
import { MongoApp } from '../../../core/app/schema';
|
||||
import { getMCPChildren } from '../../../core/app/mcp';
|
||||
import { getSystemToolRunTimeNodeFromSystemToolset } from '../utils';
|
||||
import type { localeType } from '@fastgpt/global/common/i18n/type';
|
||||
import type { HttpToolConfigType } from '@fastgpt/global/core/app/type';
|
||||
|
||||
export const getWorkflowResponseWrite = ({
|
||||
res,
|
||||
|
|
@ -197,7 +199,8 @@ export const rewriteRuntimeWorkFlow = async ({
|
|||
for (const toolSetNode of toolSetNodes) {
|
||||
nodeIdsToRemove.add(toolSetNode.nodeId);
|
||||
const systemToolId = toolSetNode.toolConfig?.systemToolSet?.toolId;
|
||||
const mcpToolsetVal = toolSetNode.toolConfig?.mcpToolSet ?? toolSetNode.inputs[0].value;
|
||||
const mcpToolsetVal = toolSetNode.toolConfig?.mcpToolSet ?? toolSetNode.inputs?.[0]?.value;
|
||||
const httpToolsetVal = toolSetNode.toolConfig?.httpToolSet;
|
||||
|
||||
const incomingEdges = edges.filter((edge) => edge.target === toolSetNode.nodeId);
|
||||
const pushEdges = (nodeId: string) => {
|
||||
|
|
@ -243,6 +246,22 @@ export const rewriteRuntimeWorkFlow = async ({
|
|||
});
|
||||
pushEdges(newToolNode.nodeId);
|
||||
});
|
||||
} else if (httpToolsetVal) {
|
||||
const parentId = toolSetNode.pluginId || '';
|
||||
httpToolsetVal.toolList.forEach((tool: HttpToolConfigType, index: number) => {
|
||||
const newToolNode = getHTTPToolRuntimeNode({
|
||||
tool: {
|
||||
...tool,
|
||||
name: `${toolSetNode.name}/${tool.name}`
|
||||
},
|
||||
nodeId: `${parentId}${index}`,
|
||||
avatar: toolSetNode.avatar,
|
||||
parentId
|
||||
});
|
||||
|
||||
nodes.push(newToolNode);
|
||||
pushEdges(newToolNode.nodeId);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -27,6 +27,8 @@
|
|||
"encoding": "^0.1.13",
|
||||
"file-type": "^19.0.0",
|
||||
"form-data": "^4.0.4",
|
||||
"http-proxy-agent": "^7.0.2",
|
||||
"https-proxy-agent": "^7.0.6",
|
||||
"iconv-lite": "^0.6.3",
|
||||
"ioredis": "^5.6.0",
|
||||
"joplin-turndown-plugin-gfm": "^1.0.12",
|
||||
|
|
@ -35,6 +37,7 @@
|
|||
"jsonwebtoken": "^9.0.2",
|
||||
"lodash": "^4.17.21",
|
||||
"mammoth": "^1.6.0",
|
||||
"minio": "^8.0.5",
|
||||
"mongoose": "^8.10.1",
|
||||
"multer": "2.0.2",
|
||||
"mysql2": "^3.11.3",
|
||||
|
|
|
|||
|
|
@ -1,32 +0,0 @@
|
|||
import { MongoTeamMember } from '../../user/team/teamMemberSchema';
|
||||
import { checkTeamAIPoints } from '../teamLimit';
|
||||
import { type UserModelSchema } from '@fastgpt/global/support/user/type';
|
||||
import { type TeamSchema } from '@fastgpt/global/support/user/team/type';
|
||||
import { TeamErrEnum } from '@fastgpt/global/common/error/code/team';
|
||||
|
||||
export async function getUserChatInfoAndAuthTeamPoints(tmbId: string) {
|
||||
const tmb = await MongoTeamMember.findById(tmbId, 'userId teamId')
|
||||
.populate<{ user: UserModelSchema; team: TeamSchema }>([
|
||||
{
|
||||
path: 'user',
|
||||
select: 'timezone'
|
||||
},
|
||||
{
|
||||
path: 'team',
|
||||
select: 'openaiAccount externalWorkflowVariables'
|
||||
}
|
||||
])
|
||||
.lean();
|
||||
|
||||
if (!tmb) return Promise.reject(TeamErrEnum.notUser);
|
||||
|
||||
await checkTeamAIPoints(tmb.team._id);
|
||||
|
||||
return {
|
||||
timezone: tmb.user.timezone,
|
||||
externalProvider: {
|
||||
openaiAccount: tmb.team.openaiAccount,
|
||||
externalWorkflowVariables: tmb.team.externalWorkflowVariables
|
||||
}
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant';
|
||||
import { getGroupsByTmbId } from '../memberGroup/controllers';
|
||||
import { getOrgsByTmbId } from '../org/controllers';
|
||||
import { MongoResourcePermission } from '../schema';
|
||||
import { getCollaboratorId } from '@fastgpt/global/support/permission/utils';
|
||||
import { isProVersion } from '../../../common/system/constants';
|
||||
|
||||
export const getMyModels = async ({
|
||||
teamId,
|
||||
tmbId,
|
||||
isTeamOwner
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
isTeamOwner: boolean;
|
||||
}) => {
|
||||
if (isTeamOwner || !isProVersion()) {
|
||||
return global.systemModelList.map((m) => m.model);
|
||||
}
|
||||
const [groups, orgs] = await Promise.all([
|
||||
getGroupsByTmbId({
|
||||
teamId,
|
||||
tmbId
|
||||
}),
|
||||
getOrgsByTmbId({
|
||||
teamId,
|
||||
tmbId
|
||||
})
|
||||
]);
|
||||
|
||||
const myIdSet = new Set([tmbId, ...groups.map((g) => g._id), ...orgs.map((o) => o._id)]);
|
||||
|
||||
const rps = await MongoResourcePermission.find({
|
||||
teamId,
|
||||
resourceType: PerResourceTypeEnum.model
|
||||
}).lean();
|
||||
|
||||
const permissionConfiguredModelSet = new Set(rps.map((rp) => rp.resourceName));
|
||||
const unconfiguredModels = global.systemModelList.filter(
|
||||
(model) => !permissionConfiguredModelSet.has(model.model)
|
||||
);
|
||||
|
||||
const myModels = rps.filter((rp) => myIdSet.has(getCollaboratorId(rp)));
|
||||
|
||||
return [...unconfiguredModels.map((m) => m.model), ...myModels.map((m) => m.resourceName)];
|
||||
};
|
||||
|
|
@ -41,14 +41,20 @@ export const ResourcePermissionSchema = new Schema({
|
|||
type: Number,
|
||||
required: true
|
||||
},
|
||||
|
||||
/**
|
||||
* Optional. Only be set when the resource is *inherited* from the parent resource.
|
||||
* For recording the self permission. When cancel the inheritance, it will overwrite the permission property and set to `unset`.
|
||||
*/
|
||||
// Resource ID: App or DataSet or any other resource type.
|
||||
// It is null if the resourceType is team.
|
||||
resourceId: {
|
||||
type: Schema.Types.ObjectId
|
||||
},
|
||||
|
||||
/**
|
||||
* Optional, For some resources, which do not have resourceId, the resourceName is required.
|
||||
*/
|
||||
resourceName: {
|
||||
type: String
|
||||
}
|
||||
});
|
||||
|
||||
|
|
@ -72,6 +78,7 @@ ResourcePermissionSchema.virtual('org', {
|
|||
});
|
||||
|
||||
try {
|
||||
// Indexes for resourceId-based resources
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
|
|
@ -84,6 +91,9 @@ try {
|
|||
partialFilterExpression: {
|
||||
groupId: {
|
||||
$exists: true
|
||||
},
|
||||
resourceId: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -101,6 +111,9 @@ try {
|
|||
partialFilterExpression: {
|
||||
orgId: {
|
||||
$exists: true
|
||||
},
|
||||
resourceId: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -118,17 +131,106 @@ try {
|
|||
partialFilterExpression: {
|
||||
tmbId: {
|
||||
$exists: true
|
||||
},
|
||||
resourceId: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Delete tmb permission
|
||||
ResourcePermissionSchema.index({
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
resourceId: 1
|
||||
});
|
||||
// General index for resourceId-based resources
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
resourceId: 1
|
||||
},
|
||||
{
|
||||
partialFilterExpression: {
|
||||
resourceId: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Indexes for resourceName-based resources
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
resourceName: 1,
|
||||
groupId: 1
|
||||
},
|
||||
{
|
||||
unique: true,
|
||||
partialFilterExpression: {
|
||||
groupId: {
|
||||
$exists: true
|
||||
},
|
||||
resourceName: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
resourceName: 1,
|
||||
orgId: 1
|
||||
},
|
||||
{
|
||||
unique: true,
|
||||
partialFilterExpression: {
|
||||
orgId: {
|
||||
$exists: true
|
||||
},
|
||||
resourceName: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
resourceName: 1,
|
||||
tmbId: 1
|
||||
},
|
||||
{
|
||||
unique: true,
|
||||
partialFilterExpression: {
|
||||
tmbId: {
|
||||
$exists: true
|
||||
},
|
||||
resourceName: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// General index for resourceName-based resources
|
||||
ResourcePermissionSchema.index(
|
||||
{
|
||||
resourceType: 1,
|
||||
teamId: 1,
|
||||
resourceName: 1
|
||||
},
|
||||
{
|
||||
partialFilterExpression: {
|
||||
resourceName: {
|
||||
$exists: true
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -46,7 +46,13 @@ export const checkTeamAppLimit = async (teamId: string, amount = 1) => {
|
|||
MongoApp.countDocuments({
|
||||
teamId,
|
||||
type: {
|
||||
$in: [AppTypeEnum.simple, AppTypeEnum.workflow, AppTypeEnum.plugin, AppTypeEnum.toolSet]
|
||||
$in: [
|
||||
AppTypeEnum.simple,
|
||||
AppTypeEnum.workflow,
|
||||
AppTypeEnum.plugin,
|
||||
AppTypeEnum.toolSet,
|
||||
AppTypeEnum.httpToolSet
|
||||
]
|
||||
}
|
||||
})
|
||||
]);
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ export function getI18nAppType(type: AppTypeEnum): string {
|
|||
if (type === AppTypeEnum.workflow) return i18nT('account_team:type.Workflow bot');
|
||||
if (type === AppTypeEnum.plugin) return i18nT('account_team:type.Plugin');
|
||||
if (type === AppTypeEnum.httpPlugin) return i18nT('account_team:type.Http plugin');
|
||||
if (type === AppTypeEnum.httpToolSet) return i18nT('account_team:type.Http tool set');
|
||||
if (type === AppTypeEnum.toolSet) return i18nT('account_team:type.Tool set');
|
||||
if (type === AppTypeEnum.tool) return i18nT('account_team:type.Tool');
|
||||
return i18nT('common:UnKnow');
|
||||
|
|
|
|||
|
|
@ -33,3 +33,28 @@ export async function getRunningUserInfoByTmbId(tmbId: string) {
|
|||
|
||||
return Promise.reject(TeamErrEnum.notUser);
|
||||
}
|
||||
|
||||
export async function getUserChatInfo(tmbId: string) {
|
||||
const tmb = await MongoTeamMember.findById(tmbId, 'userId teamId')
|
||||
.populate<{ user: UserModelSchema; team: TeamSchema }>([
|
||||
{
|
||||
path: 'user',
|
||||
select: 'timezone'
|
||||
},
|
||||
{
|
||||
path: 'team',
|
||||
select: 'openaiAccount externalWorkflowVariables'
|
||||
}
|
||||
])
|
||||
.lean();
|
||||
|
||||
if (!tmb) return Promise.reject(TeamErrEnum.notUser);
|
||||
|
||||
return {
|
||||
timezone: tmb.user.timezone,
|
||||
externalProvider: {
|
||||
openaiAccount: tmb.team.openaiAccount,
|
||||
externalWorkflowVariables: tmb.team.externalWorkflowVariables
|
||||
}
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
export const UsageCollectionName = 'usages';
|
||||
export const UsageItemCollectionName = 'usage_items';
|
||||
|
|
@ -1,19 +1,21 @@
|
|||
import { UsageSourceEnum } from '@fastgpt/global/support/wallet/usage/constants';
|
||||
import { UsageItemTypeEnum, UsageSourceEnum } from '@fastgpt/global/support/wallet/usage/constants';
|
||||
import { MongoUsage } from './schema';
|
||||
import { type ClientSession } from '../../../common/mongo';
|
||||
import { addLog } from '../../../common/system/log';
|
||||
import { type ChatNodeUsageType } from '@fastgpt/global/support/wallet/bill/type';
|
||||
import {
|
||||
type ConcatUsageProps,
|
||||
type CreateUsageProps
|
||||
import type {
|
||||
PushUsageItemsProps,
|
||||
ConcatUsageProps,
|
||||
CreateUsageProps
|
||||
} from '@fastgpt/global/support/wallet/usage/api';
|
||||
import { i18nT } from '../../../../web/i18n/utils';
|
||||
import { formatModelChars2Points } from './utils';
|
||||
import { ModelTypeEnum } from '@fastgpt/global/core/ai/model';
|
||||
import { mongoSessionRun } from '../../../common/mongo/sessionRun';
|
||||
import { MongoUsageItem } from './usageItemSchema';
|
||||
|
||||
export async function createUsage(data: CreateUsageProps) {
|
||||
try {
|
||||
await global.createUsageHandler(data);
|
||||
return await global.createUsageHandler(data);
|
||||
} catch (error) {
|
||||
addLog.error('createUsage error', error);
|
||||
}
|
||||
|
|
@ -25,15 +27,94 @@ export async function concatUsage(data: ConcatUsageProps) {
|
|||
addLog.error('concatUsage error', error);
|
||||
}
|
||||
}
|
||||
export async function pushUsageItems(data: PushUsageItemsProps) {
|
||||
try {
|
||||
await global.pushUsageItemsHandler(data);
|
||||
} catch (error) {
|
||||
addLog.error('pushUsageItems error', error);
|
||||
}
|
||||
}
|
||||
|
||||
export const createChatUsage = ({
|
||||
export const createPdfParseUsage = async ({
|
||||
teamId,
|
||||
tmbId,
|
||||
pages,
|
||||
usageId
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
pages: number;
|
||||
usageId?: string;
|
||||
}) => {
|
||||
const unitPrice = global.systemEnv?.customPdfParse?.price || 0;
|
||||
const totalPoints = pages * unitPrice;
|
||||
|
||||
if (usageId) {
|
||||
pushUsageItems({
|
||||
teamId,
|
||||
usageId,
|
||||
list: [{ moduleName: i18nT('account_usage:pdf_enhanced_parse'), amount: totalPoints, pages }]
|
||||
});
|
||||
} else {
|
||||
createUsage({
|
||||
teamId,
|
||||
tmbId,
|
||||
appName: i18nT('account_usage:pdf_enhanced_parse'),
|
||||
totalPoints,
|
||||
source: UsageSourceEnum.pdfParse,
|
||||
list: [
|
||||
{
|
||||
moduleName: i18nT('account_usage:pdf_enhanced_parse'),
|
||||
amount: totalPoints,
|
||||
pages
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
};
|
||||
export const pushLLMTrainingUsage = async ({
|
||||
teamId,
|
||||
model,
|
||||
inputTokens,
|
||||
outputTokens,
|
||||
usageId,
|
||||
type
|
||||
}: {
|
||||
teamId: string;
|
||||
model: string;
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
usageId: string;
|
||||
type: UsageItemTypeEnum;
|
||||
}) => {
|
||||
// Compute points
|
||||
const { totalPoints } = formatModelChars2Points({
|
||||
model,
|
||||
inputTokens,
|
||||
outputTokens
|
||||
});
|
||||
|
||||
concatUsage({
|
||||
usageId,
|
||||
teamId,
|
||||
itemType: type,
|
||||
totalPoints,
|
||||
inputTokens,
|
||||
outputTokens
|
||||
});
|
||||
|
||||
return { totalPoints };
|
||||
};
|
||||
|
||||
/* Create usage, and return usageId */
|
||||
// Chat
|
||||
export const createChatUsageRecord = async ({
|
||||
appName,
|
||||
appId,
|
||||
pluginId,
|
||||
teamId,
|
||||
tmbId,
|
||||
source,
|
||||
flowUsages
|
||||
source
|
||||
}: {
|
||||
appName: string;
|
||||
appId?: string;
|
||||
|
|
@ -41,42 +122,46 @@ export const createChatUsage = ({
|
|||
teamId: string;
|
||||
tmbId: string;
|
||||
source: UsageSourceEnum;
|
||||
flowUsages: ChatNodeUsageType[];
|
||||
}) => {
|
||||
const totalPoints = flowUsages.reduce((sum, item) => sum + (item.totalPoints || 0), 0);
|
||||
|
||||
createUsage({
|
||||
const [{ _id: usageId }] = await MongoUsage.create(
|
||||
[
|
||||
{
|
||||
teamId,
|
||||
tmbId,
|
||||
appId,
|
||||
pluginId,
|
||||
appName,
|
||||
source,
|
||||
totalPoints: 0
|
||||
}
|
||||
],
|
||||
{ ordered: true }
|
||||
);
|
||||
return String(usageId);
|
||||
};
|
||||
export const pushChatItemUsage = ({
|
||||
teamId,
|
||||
usageId,
|
||||
nodeUsages
|
||||
}: {
|
||||
teamId: string;
|
||||
usageId: string;
|
||||
nodeUsages: ChatNodeUsageType[];
|
||||
}) => {
|
||||
pushUsageItems({
|
||||
teamId,
|
||||
tmbId,
|
||||
appName,
|
||||
appId,
|
||||
pluginId,
|
||||
totalPoints,
|
||||
source,
|
||||
list: flowUsages.map((item) => ({
|
||||
usageId,
|
||||
list: nodeUsages.map((item) => ({
|
||||
moduleName: item.moduleName,
|
||||
amount: item.totalPoints || 0,
|
||||
amount: item.totalPoints,
|
||||
model: item.model,
|
||||
inputTokens: item.inputTokens,
|
||||
outputTokens: item.outputTokens
|
||||
}))
|
||||
});
|
||||
addLog.debug(`Create chat usage`, {
|
||||
source,
|
||||
teamId,
|
||||
totalPoints
|
||||
});
|
||||
return { totalPoints };
|
||||
};
|
||||
|
||||
export type DatasetTrainingMode = 'paragraph' | 'qa' | 'autoIndex' | 'imageIndex' | 'imageParse';
|
||||
export const datasetTrainingUsageIndexMap: Record<DatasetTrainingMode, number> = {
|
||||
paragraph: 1,
|
||||
qa: 2,
|
||||
autoIndex: 3,
|
||||
imageIndex: 4,
|
||||
imageParse: 5
|
||||
};
|
||||
// Dataset training
|
||||
export const createTrainingUsage = async ({
|
||||
teamId,
|
||||
tmbId,
|
||||
|
|
@ -91,189 +176,161 @@ export const createTrainingUsage = async ({
|
|||
tmbId: string;
|
||||
appName: string;
|
||||
billSource: UsageSourceEnum;
|
||||
vectorModel?: string;
|
||||
|
||||
vectorModel: string;
|
||||
agentModel?: string;
|
||||
vllmModel?: string;
|
||||
session?: ClientSession;
|
||||
}) => {
|
||||
const [{ _id }] = await MongoUsage.create(
|
||||
[
|
||||
const create = async (session: ClientSession) => {
|
||||
const [result] = await MongoUsage.create(
|
||||
[
|
||||
{
|
||||
teamId,
|
||||
tmbId,
|
||||
source: billSource,
|
||||
appName,
|
||||
totalPoints: 0
|
||||
}
|
||||
],
|
||||
{ session, ordered: true }
|
||||
);
|
||||
await MongoUsageItem.create(
|
||||
[
|
||||
{
|
||||
teamId,
|
||||
usageId: result._id,
|
||||
itemType: UsageItemTypeEnum.training_vector,
|
||||
name: i18nT('account_usage:embedding_index'),
|
||||
model: vectorModel,
|
||||
amount: 0,
|
||||
inputTokens: 0
|
||||
},
|
||||
...(agentModel
|
||||
? [
|
||||
{
|
||||
teamId,
|
||||
usageId: result._id,
|
||||
itemType: UsageItemTypeEnum.training_paragraph,
|
||||
name: i18nT('account_usage:llm_paragraph'),
|
||||
model: agentModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
},
|
||||
{
|
||||
teamId,
|
||||
usageId: result._id,
|
||||
itemType: UsageItemTypeEnum.training_qa,
|
||||
name: i18nT('account_usage:qa'),
|
||||
model: agentModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
},
|
||||
{
|
||||
teamId,
|
||||
usageId: result._id,
|
||||
itemType: UsageItemTypeEnum.training_autoIndex,
|
||||
name: i18nT('account_usage:auto_index'),
|
||||
model: agentModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
}
|
||||
]
|
||||
: []),
|
||||
...(vllmModel
|
||||
? [
|
||||
{
|
||||
teamId,
|
||||
usageId: result._id,
|
||||
itemType: UsageItemTypeEnum.training_imageIndex,
|
||||
name: i18nT('account_usage:image_index'),
|
||||
model: vllmModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
},
|
||||
{
|
||||
teamId,
|
||||
usageId: result._id,
|
||||
itemType: UsageItemTypeEnum.training_imageParse,
|
||||
name: i18nT('account_usage:image_parse'),
|
||||
model: vllmModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
}
|
||||
]
|
||||
: [])
|
||||
],
|
||||
{
|
||||
teamId,
|
||||
tmbId,
|
||||
appName,
|
||||
source: billSource,
|
||||
totalPoints: 0,
|
||||
list: [
|
||||
...(vectorModel
|
||||
? [
|
||||
{
|
||||
moduleName: i18nT('account_usage:embedding_index'),
|
||||
model: vectorModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
}
|
||||
]
|
||||
: []),
|
||||
...(agentModel
|
||||
? [
|
||||
{
|
||||
moduleName: i18nT('account_usage:llm_paragraph'),
|
||||
model: agentModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
},
|
||||
{
|
||||
moduleName: i18nT('account_usage:qa'),
|
||||
model: agentModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
},
|
||||
{
|
||||
moduleName: i18nT('account_usage:auto_index'),
|
||||
model: agentModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
}
|
||||
]
|
||||
: []),
|
||||
...(vllmModel
|
||||
? [
|
||||
{
|
||||
moduleName: i18nT('account_usage:image_index'),
|
||||
model: vllmModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
},
|
||||
{
|
||||
moduleName: i18nT('account_usage:image_parse'),
|
||||
model: vllmModel,
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0
|
||||
}
|
||||
]
|
||||
: [])
|
||||
]
|
||||
session,
|
||||
ordered: true
|
||||
}
|
||||
],
|
||||
{ session, ordered: true }
|
||||
);
|
||||
);
|
||||
|
||||
return { billId: String(_id) };
|
||||
};
|
||||
|
||||
export const createPdfParseUsage = async ({
|
||||
teamId,
|
||||
tmbId,
|
||||
pages
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
pages: number;
|
||||
}) => {
|
||||
const unitPrice = global.systemEnv?.customPdfParse?.price || 0;
|
||||
const totalPoints = pages * unitPrice;
|
||||
|
||||
createUsage({
|
||||
teamId,
|
||||
tmbId,
|
||||
appName: i18nT('account_usage:pdf_enhanced_parse'),
|
||||
totalPoints,
|
||||
source: UsageSourceEnum.pdfParse,
|
||||
list: [
|
||||
{
|
||||
moduleName: i18nT('account_usage:pdf_enhanced_parse'),
|
||||
amount: totalPoints,
|
||||
pages
|
||||
}
|
||||
]
|
||||
});
|
||||
};
|
||||
|
||||
export const pushLLMTrainingUsage = async ({
|
||||
teamId,
|
||||
tmbId,
|
||||
model,
|
||||
inputTokens,
|
||||
outputTokens,
|
||||
billId,
|
||||
mode
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
model: string;
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
billId: string;
|
||||
mode: DatasetTrainingMode;
|
||||
}) => {
|
||||
const index = datasetTrainingUsageIndexMap[mode];
|
||||
|
||||
// Compute points
|
||||
const { totalPoints } = formatModelChars2Points({
|
||||
model,
|
||||
inputTokens,
|
||||
outputTokens
|
||||
});
|
||||
|
||||
concatUsage({
|
||||
billId,
|
||||
teamId,
|
||||
tmbId,
|
||||
totalPoints,
|
||||
inputTokens,
|
||||
outputTokens,
|
||||
listIndex: index
|
||||
});
|
||||
|
||||
return { totalPoints };
|
||||
return { usageId: String(result._id) };
|
||||
};
|
||||
if (session) return create(session);
|
||||
return mongoSessionRun(create);
|
||||
};
|
||||
|
||||
// Evaluation
|
||||
export const createEvaluationUsage = async ({
|
||||
teamId,
|
||||
tmbId,
|
||||
appName,
|
||||
model,
|
||||
session
|
||||
model
|
||||
}: {
|
||||
teamId: string;
|
||||
tmbId: string;
|
||||
appName: string;
|
||||
model: string;
|
||||
session?: ClientSession;
|
||||
}) => {
|
||||
const [{ _id: usageId }] = await MongoUsage.create(
|
||||
[
|
||||
const { usageId } = await mongoSessionRun(async (session) => {
|
||||
const [{ _id: usageId }] = await MongoUsage.create(
|
||||
[
|
||||
{
|
||||
teamId,
|
||||
tmbId,
|
||||
appName,
|
||||
source: UsageSourceEnum.evaluation,
|
||||
totalPoints: 0
|
||||
}
|
||||
],
|
||||
{ session, ordered: true }
|
||||
);
|
||||
await MongoUsageItem.create(
|
||||
[
|
||||
{
|
||||
teamId,
|
||||
usageId,
|
||||
itemType: UsageItemTypeEnum.evaluation_generateAnswer,
|
||||
name: i18nT('account_usage:generate_answer'),
|
||||
amount: 0,
|
||||
count: 0
|
||||
},
|
||||
{
|
||||
teamId,
|
||||
usageId,
|
||||
itemType: UsageItemTypeEnum.evaluation_answerAccuracy,
|
||||
name: i18nT('account_usage:answer_accuracy'),
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
model
|
||||
}
|
||||
],
|
||||
{
|
||||
teamId,
|
||||
tmbId,
|
||||
appName,
|
||||
source: UsageSourceEnum.evaluation,
|
||||
totalPoints: 0,
|
||||
list: [
|
||||
{
|
||||
moduleName: i18nT('account_usage:generate_answer'),
|
||||
amount: 0,
|
||||
count: 0
|
||||
},
|
||||
{
|
||||
moduleName: i18nT('account_usage:answer_accuracy'),
|
||||
amount: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
model
|
||||
}
|
||||
]
|
||||
session,
|
||||
ordered: true
|
||||
}
|
||||
],
|
||||
{ session, ordered: true }
|
||||
);
|
||||
);
|
||||
|
||||
return { usageId: String(usageId) };
|
||||
});
|
||||
|
||||
return { usageId };
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
import { connectionMongo, getMongoModel, type Model } from '../../../common/mongo';
|
||||
import { connectionMongo, getMongoModel } from '../../../common/mongo';
|
||||
const { Schema } = connectionMongo;
|
||||
import { type UsageSchemaType } from '@fastgpt/global/support/wallet/usage/type';
|
||||
import { UsageSourceEnum } from '@fastgpt/global/support/wallet/usage/constants';
|
||||
|
|
@ -6,8 +6,7 @@ import {
|
|||
TeamCollectionName,
|
||||
TeamMemberCollectionName
|
||||
} from '@fastgpt/global/support/user/team/constant';
|
||||
|
||||
export const UsageCollectionName = 'usages';
|
||||
import { UsageCollectionName, UsageItemCollectionName } from './constants';
|
||||
|
||||
const UsageSchema = new Schema({
|
||||
teamId: {
|
||||
|
|
@ -30,6 +29,11 @@ const UsageSchema = new Schema({
|
|||
type: String,
|
||||
default: ''
|
||||
},
|
||||
totalPoints: {
|
||||
// total points
|
||||
type: Number,
|
||||
required: true
|
||||
},
|
||||
appId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: 'apps',
|
||||
|
|
@ -44,26 +48,21 @@ const UsageSchema = new Schema({
|
|||
type: Date,
|
||||
default: () => new Date()
|
||||
},
|
||||
totalPoints: {
|
||||
// total points
|
||||
type: Number,
|
||||
required: true
|
||||
},
|
||||
// total: {
|
||||
// // total points
|
||||
// type: Number,
|
||||
// required: true
|
||||
// },
|
||||
|
||||
// @description It will not be used again in the future.
|
||||
list: {
|
||||
type: Array,
|
||||
default: []
|
||||
type: Array
|
||||
}
|
||||
});
|
||||
|
||||
UsageSchema.virtual('usageItems', {
|
||||
ref: UsageItemCollectionName,
|
||||
localField: '_id',
|
||||
foreignField: 'usageId'
|
||||
});
|
||||
|
||||
try {
|
||||
UsageSchema.index({ teamId: 1, tmbId: 1, source: 1, time: 1, appName: 1, _id: -1 });
|
||||
// timer task. clear dead team
|
||||
// UsageSchema.index({ teamId: 1, time: -1 });
|
||||
|
||||
UsageSchema.index({ time: 1 }, { expireAfterSeconds: 360 * 24 * 60 * 60 });
|
||||
} catch (error) {
|
||||
|
|
|
|||
|
|
@ -1,16 +1,6 @@
|
|||
export type ConcatBillQueueItemType = {
|
||||
billId: string; // usageId
|
||||
listIndex?: number;
|
||||
totalPoints: number;
|
||||
|
||||
// Model usage
|
||||
inputTokens?: number;
|
||||
outputTokens?: number;
|
||||
// Times
|
||||
count?: number;
|
||||
};
|
||||
import type { ConcatUsageProps } from '@fastgpt/global/support/wallet/usage/api';
|
||||
|
||||
declare global {
|
||||
var reduceAiPointsQueue: { teamId: string; totalPoints: number }[];
|
||||
var concatBillQueue: ConcatBillQueueItemType[];
|
||||
var concatBillQueue: ConcatUsageProps[];
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,53 @@
|
|||
import { connectionMongo, getMongoModel } from '../../../common/mongo';
|
||||
const { Schema } = connectionMongo;
|
||||
import type { UsageItemSchemaType } from '@fastgpt/global/support/wallet/usage/type';
|
||||
import { UsageCollectionName, UsageItemCollectionName } from './constants';
|
||||
import { TeamCollectionName } from '@fastgpt/global/support/user/team/constant';
|
||||
|
||||
const UsageItemSchema = new Schema({
|
||||
teamId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: TeamCollectionName,
|
||||
required: true
|
||||
},
|
||||
usageId: {
|
||||
type: Schema.Types.ObjectId,
|
||||
ref: UsageCollectionName,
|
||||
required: true
|
||||
},
|
||||
name: {
|
||||
// usage name
|
||||
type: String,
|
||||
required: true
|
||||
},
|
||||
amount: {
|
||||
type: Number,
|
||||
default: 0
|
||||
},
|
||||
itemType: Number,
|
||||
time: {
|
||||
type: Date,
|
||||
default: () => new Date()
|
||||
},
|
||||
|
||||
// Params
|
||||
inputTokens: Number,
|
||||
outputTokens: Number,
|
||||
charsLength: Number,
|
||||
duration: Number,
|
||||
pages: Number,
|
||||
count: Number,
|
||||
model: String
|
||||
});
|
||||
|
||||
try {
|
||||
UsageItemSchema.index({ usageId: 'hashed' });
|
||||
UsageItemSchema.index({ time: 1 }, { expireAfterSeconds: 360 * 24 * 60 * 60 });
|
||||
} catch (error) {
|
||||
console.log(error);
|
||||
}
|
||||
|
||||
export const MongoUsageItem = getMongoModel<UsageItemSchemaType>(
|
||||
UsageItemCollectionName,
|
||||
UsageItemSchema
|
||||
);
|
||||
|
|
@ -1,9 +1,9 @@
|
|||
import { createClient } from '@fastgpt/global/sdk/fastgpt-plugin';
|
||||
|
||||
export const BASE_URL = process.env.PLUGIN_BASE_URL || '';
|
||||
export const TOKEN = process.env.PLUGIN_TOKEN || '';
|
||||
export const PLUGIN_BASE_URL = process.env.PLUGIN_BASE_URL || '';
|
||||
export const PLUGIN_TOKEN = process.env.PLUGIN_TOKEN || '';
|
||||
|
||||
export const pluginClient = createClient({
|
||||
baseUrl: BASE_URL,
|
||||
token: TOKEN
|
||||
baseUrl: PLUGIN_BASE_URL,
|
||||
token: PLUGIN_TOKEN
|
||||
});
|
||||
|
|
|
|||
|
|
@ -43,9 +43,18 @@ const parsePowerPoint = async ({
|
|||
return Promise.reject('解析 PPT 失败');
|
||||
}
|
||||
|
||||
// Sort files by slide number to ensure correct order
|
||||
const sortedFiles = files.sort((a, b) => {
|
||||
const getSlideNumber = (path: string) => {
|
||||
const match = path.match(/\d+/);
|
||||
return match ? parseInt(match[0]) : 0;
|
||||
};
|
||||
return getSlideNumber(a.path) - getSlideNumber(b.path);
|
||||
});
|
||||
|
||||
// Returning an array of all the xml contents read using fs.readFileSync
|
||||
const xmlContentArray = await Promise.all(
|
||||
files.map(async (file) => {
|
||||
sortedFiles.map(async (file) => {
|
||||
try {
|
||||
return await fs.promises.readFile(`${decompressPath}/${file.path}`, encoding);
|
||||
} catch (err) {
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import React, { useState, useMemo, useRef, useEffect } from 'react';
|
||||
import type { BoxProps } from '@chakra-ui/react';
|
||||
import { Box, Card, Flex, useOutsideClick, Button } from '@chakra-ui/react';
|
||||
import { Box, Card, Flex, useTheme, useOutsideClick, Button } from '@chakra-ui/react';
|
||||
import { addDays, format } from 'date-fns';
|
||||
import { DayPicker } from 'react-day-picker';
|
||||
import 'react-day-picker/dist/style.css';
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ export const iconPaths = {
|
|||
close: () => import('./icons/close.svg'),
|
||||
closeSolid: () => import('./icons/closeSolid.svg'),
|
||||
code: () => import('./icons/code.svg'),
|
||||
codeCopilot: () => import('./icons/codeCopilot.svg'),
|
||||
collectionLight: () => import('./icons/collectionLight.svg'),
|
||||
collectionSolid: () => import('./icons/collectionSolid.svg'),
|
||||
comment: () => import('./icons/comment.svg'),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,46 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" fill="none">
|
||||
<path d="M4.65137 13.7275C4.98754 13.7275 5.26074 14.0128 5.26074 14.3643C5.2606 14.7156 4.98745 15 4.65137 15C4.31537 14.9999 4.04311 14.7155 4.04297 14.3643C4.04297 14.0129 4.31528 13.7276 4.65137 13.7275Z" fill="url(#paint0_linear_26054_96715)"/>
|
||||
<path d="M2.01855 11.3457C2.06906 11.1279 2.36551 11.1279 2.41602 11.3457L2.55469 11.9443C2.57269 12.0219 2.63092 12.0826 2.70508 12.1016L3.27832 12.2471C3.48661 12.2999 3.48661 12.6093 3.27832 12.6621L2.70508 12.8076C2.63091 12.8265 2.57273 12.8873 2.55469 12.9648L2.41602 13.5645C2.36538 13.782 2.06915 13.782 2.01855 13.5645L1.87988 12.9648C1.86181 12.8872 1.80374 12.8265 1.72949 12.8076L1.15625 12.6621C0.948018 12.6093 0.94798 12.2999 1.15625 12.2471L1.72949 12.1016C1.80375 12.0826 1.86187 12.022 1.87988 11.9443L2.01855 11.3457Z" fill="url(#paint1_linear_26054_96715)"/>
|
||||
<path d="M8.82227 3.50586C9.19608 3.61202 9.41664 4.00027 9.31445 4.37402C9.02199 5.44297 8.64314 7.12446 8.33301 8.57617C8.17854 9.29922 8.04241 9.96146 7.94336 10.4541C7.89386 10.7003 7.85325 10.9038 7.8252 11.0508C7.7996 11.1849 7.78871 11.2467 7.78613 11.2598C7.74665 11.6461 7.40153 11.9249 7.01562 11.8818C6.62991 11.8387 6.34941 11.4906 6.38867 11.1045C6.39858 11.0085 6.47241 10.6316 6.56641 10.1641C6.66658 9.66585 6.80339 8.99884 6.95898 8.27051C7.26915 6.81866 7.65498 5.10146 7.95898 3.99023C8.06148 3.61653 8.44823 3.39994 8.82227 3.50586Z" fill="url(#paint2_linear_26054_96715)"/>
|
||||
<path d="M4.34766 4.2959C4.63622 4.03649 5.08015 4.06029 5.33984 4.34863C5.59948 4.63715 5.57641 5.08103 5.28809 5.34082L2.68555 7.68262L5.28809 10.0234C5.57655 10.2832 5.59955 10.728 5.33984 11.0166C5.08006 11.3046 4.63608 11.3278 4.34766 11.0684L1.41309 8.42773L1.33496 8.34961C0.996541 7.97076 0.996493 7.39347 1.33496 7.01465L1.41309 6.93652L4.34766 4.2959Z" fill="url(#paint3_linear_26054_96715)"/>
|
||||
<path d="M10.6592 4.34863C10.9188 4.06017 11.3627 4.03659 11.6514 4.2959L14.5859 6.93652L14.6641 7.01465C15.0028 7.39353 15.0027 7.97067 14.6641 8.34961L14.5859 8.42773L11.6514 11.0684C11.3629 11.3276 10.9189 11.3048 10.6592 11.0166C10.3997 10.7281 10.4227 10.2832 10.7109 10.0234L13.3135 7.68262L10.7109 5.34082C10.4228 5.08107 10.3998 4.6371 10.6592 4.34863Z" fill="url(#paint4_linear_26054_96715)"/>
|
||||
<path d="M13.584 2.43652C13.6345 2.21877 13.93 2.21877 13.9805 2.43652L14.1201 3.03516C14.1381 3.1129 14.1961 3.17451 14.2705 3.19336L14.8438 3.33789C15.0518 3.39083 15.0518 3.701 14.8438 3.75391L14.2705 3.89844C14.1963 3.91726 14.1383 3.97812 14.1201 4.05566L13.9805 4.65527C13.9299 4.87285 13.6346 4.87287 13.584 4.65527L13.4443 4.05566C13.4262 3.97814 13.3682 3.91725 13.2939 3.89844L12.7207 3.75391C12.5128 3.70092 12.5128 3.39091 12.7207 3.33789L13.2939 3.19336C13.3683 3.17451 13.4263 3.1129 13.4443 3.03516L13.584 2.43652Z" fill="url(#paint5_linear_26054_96715)"/>
|
||||
<path d="M11.3477 1C11.6838 1 11.957 1.28526 11.957 1.63672C11.957 1.98816 11.6838 2.27344 11.3477 2.27344C11.0116 2.27326 10.7393 1.98806 10.7393 1.63672C10.7393 1.28537 11.0116 1.00017 11.3477 1Z" fill="url(#paint6_linear_26054_96715)"/>
|
||||
<defs>
|
||||
<linearGradient id="paint0_linear_26054_96715" x1="7.99986" y1="1.45455" x2="7.99986" y2="14.5455" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#499DFF"/>
|
||||
<stop offset="0.432292" stop-color="#2770FF"/>
|
||||
<stop offset="1" stop-color="#6E80FF"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint1_linear_26054_96715" x1="7.99986" y1="1.45455" x2="7.99986" y2="14.5455" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#499DFF"/>
|
||||
<stop offset="0.432292" stop-color="#2770FF"/>
|
||||
<stop offset="1" stop-color="#6E80FF"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint2_linear_26054_96715" x1="7.99986" y1="1.45455" x2="7.99986" y2="14.5455" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#499DFF"/>
|
||||
<stop offset="0.432292" stop-color="#2770FF"/>
|
||||
<stop offset="1" stop-color="#6E80FF"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint3_linear_26054_96715" x1="7.99986" y1="1.45455" x2="7.99986" y2="14.5455" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#499DFF"/>
|
||||
<stop offset="0.432292" stop-color="#2770FF"/>
|
||||
<stop offset="1" stop-color="#6E80FF"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint4_linear_26054_96715" x1="7.99986" y1="1.45455" x2="7.99986" y2="14.5455" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#499DFF"/>
|
||||
<stop offset="0.432292" stop-color="#2770FF"/>
|
||||
<stop offset="1" stop-color="#6E80FF"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint5_linear_26054_96715" x1="7.99986" y1="1.45455" x2="7.99986" y2="14.5455" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#499DFF"/>
|
||||
<stop offset="0.432292" stop-color="#2770FF"/>
|
||||
<stop offset="1" stop-color="#6E80FF"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint6_linear_26054_96715" x1="7.99986" y1="1.45455" x2="7.99986" y2="14.5455" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#499DFF"/>
|
||||
<stop offset="0.432292" stop-color="#2770FF"/>
|
||||
<stop offset="1" stop-color="#6E80FF"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 5.1 KiB |
|
|
@ -225,7 +225,7 @@ const MyMenu = ({
|
|||
autoSelect={false}
|
||||
direction={'ltr'}
|
||||
isLazy
|
||||
lazyBehavior={'keepMounted'}
|
||||
lazyBehavior={'unmount'}
|
||||
placement={placement}
|
||||
computePositionOnMount
|
||||
>
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@ const PopoverConfirm = ({
|
|||
openDelay={100}
|
||||
closeDelay={100}
|
||||
isLazy
|
||||
lazyBehavior="keepMounted"
|
||||
lazyBehavior="unmount"
|
||||
arrowSize={10}
|
||||
strategy={'fixed'}
|
||||
computePositionOnMount={true}
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@ const MyPopover = ({
|
|||
openDelay={100}
|
||||
closeDelay={100}
|
||||
isLazy
|
||||
lazyBehavior="keepMounted"
|
||||
lazyBehavior="unmount"
|
||||
autoFocus={false}
|
||||
>
|
||||
<PopoverTrigger>{Trigger}</PopoverTrigger>
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
import type { FlexProps } from '@chakra-ui/react';
|
||||
import {
|
||||
Box,
|
||||
Button,
|
||||
type ButtonProps,
|
||||
Checkbox,
|
||||
Flex,
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
*
|
||||
*/
|
||||
|
||||
import type { CSSProperties } from 'react';
|
||||
import { useMemo, useState, useTransition } from 'react';
|
||||
import { LexicalComposer } from '@lexical/react/LexicalComposer';
|
||||
import { PlainTextPlugin } from '@lexical/react/LexicalPlainTextPlugin';
|
||||
|
|
@ -40,18 +41,17 @@ import { useDeepCompareEffect } from 'ahooks';
|
|||
import VariablePickerPlugin from './plugins/VariablePickerPlugin';
|
||||
import MarkdownPlugin from './plugins/MarkdownPlugin';
|
||||
import MyIcon from '../../Icon';
|
||||
import TabToSpacesPlugin from './plugins/TabToSpacesPlugin';
|
||||
import ListExitPlugin from './plugins/ListExitPlugin';
|
||||
import KeyDownPlugin from './plugins/KeyDownPlugin';
|
||||
|
||||
const Placeholder = ({ children }: { children: React.ReactNode }) => (
|
||||
const Placeholder = ({ children, padding }: { children: React.ReactNode; padding: string }) => (
|
||||
<Box
|
||||
position={'absolute'}
|
||||
top={0}
|
||||
left={0}
|
||||
right={0}
|
||||
bottom={0}
|
||||
py={3}
|
||||
px={3.5}
|
||||
p={padding}
|
||||
pointerEvents={'none'}
|
||||
overflow={'hidden'}
|
||||
>
|
||||
|
|
@ -78,12 +78,14 @@ export type EditorProps = {
|
|||
maxH?: number;
|
||||
maxLength?: number;
|
||||
placeholder?: string;
|
||||
placeholderPadding?: string;
|
||||
isInvalid?: boolean;
|
||||
|
||||
onKeyDown?: (e: React.KeyboardEvent) => void;
|
||||
ExtensionPopover?: ((e: {
|
||||
onChangeText: (text: string) => void;
|
||||
iconButtonStyle: Record<string, any>;
|
||||
}) => React.ReactNode)[];
|
||||
boxStyle?: CSSProperties;
|
||||
};
|
||||
|
||||
export default function Editor({
|
||||
|
|
@ -100,10 +102,12 @@ export default function Editor({
|
|||
onBlur,
|
||||
value,
|
||||
placeholder = '',
|
||||
placeholderPadding = '12px 14px',
|
||||
bg = 'white',
|
||||
isInvalid,
|
||||
|
||||
ExtensionPopover
|
||||
onKeyDown,
|
||||
ExtensionPopover,
|
||||
boxStyle
|
||||
}: EditorProps &
|
||||
FormPropsType & {
|
||||
onOpenModal?: () => void;
|
||||
|
|
@ -180,13 +184,14 @@ export default function Editor({
|
|||
className={`${isInvalid ? styles.contentEditable_invalid : styles.contentEditable} ${styles.richText}`}
|
||||
style={{
|
||||
minHeight: `${minH}px`,
|
||||
maxHeight: `${maxH}px`
|
||||
maxHeight: `${maxH}px`,
|
||||
...boxStyle
|
||||
}}
|
||||
onFocus={() => setFocus(true)}
|
||||
onBlur={() => setFocus(false)}
|
||||
/>
|
||||
}
|
||||
placeholder={<Placeholder>{placeholder}</Placeholder>}
|
||||
placeholder={<Placeholder padding={placeholderPadding}>{placeholder}</Placeholder>}
|
||||
ErrorBoundary={LexicalErrorBoundary}
|
||||
/>
|
||||
) : (
|
||||
|
|
@ -196,11 +201,12 @@ export default function Editor({
|
|||
className={isInvalid ? styles.contentEditable_invalid : styles.contentEditable}
|
||||
style={{
|
||||
minHeight: `${minH}px`,
|
||||
maxHeight: `${maxH}px`
|
||||
maxHeight: `${maxH}px`,
|
||||
...boxStyle
|
||||
}}
|
||||
/>
|
||||
}
|
||||
placeholder={<Placeholder>{placeholder}</Placeholder>}
|
||||
placeholder={<Placeholder padding={placeholderPadding}>{placeholder}</Placeholder>}
|
||||
ErrorBoundary={LexicalErrorBoundary}
|
||||
/>
|
||||
)}
|
||||
|
|
@ -210,6 +216,7 @@ export default function Editor({
|
|||
<HistoryPlugin />
|
||||
<MaxLengthPlugin maxLength={maxLength || 999999} />
|
||||
<FocusPlugin focus={focus} setFocus={setFocus} />
|
||||
<KeyDownPlugin onKeyDown={onKeyDown} />
|
||||
|
||||
<VariablePlugin variables={variables} />
|
||||
{variableLabels.length > 0 && (
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue