Commit Graph

108 Commits

Author SHA1 Message Date
CaptainB e1a12c8684 refactor: optimize image Excel ID extraction logic in common_handle.py
--bug=1064254 --user=刘瑞斌 【github#4430】Excel里带图片的数据超过5行上传后不显示图片 https://www.tapd.cn/62980211/s/1807666
2025-12-08 19:13:51 +08:00
CaptainB dc0d9e29ce feat: enhance image path handling in markdown processing
--bug=1064136 --user=刘瑞斌 【工作流知识库】上传zip文件,文件中的图片内容没有写入知识库 https://www.tapd.cn/62980211/s/1806941
2025-12-08 11:16:38 +08:00
CaptainB ce6b41d78e fix: handle None case for cell_value in xlsx_parse_table_handle
--bug=1064042 --user=刘瑞斌 【工作流知识库】QA问答对分段时,分段的问题为空,导入后关联问题为 None https://www.tapd.cn/62980211/s/1806084
2025-12-05 10:47:24 +08:00
CaptainB 2ce4f9af92 feat: enhance Markdown parsing by splitting content into sheets and adding sheet titles 2025-12-04 19:10:26 +08:00
CaptainB a197cfc69d feat: implement CSV, XLS, and XLSX content extraction with Markdown formatting 2025-12-04 14:33:19 +08:00
CaptainB 9255089d8b feat: enhance PDF content extraction with font size analysis
Some checks are pending
sync2gitee / repo-sync (push) Waiting to run
Typos Check / Spell Check with Typos (push) Waiting to run
2025-12-02 19:22:56 +08:00
CaptainB 8c802c3d01 feat: add Markdown parsing support for QA handling 2025-12-02 18:20:01 +08:00
CaptainB 582fb99b9e feat: add Markdown parsing support for QA handling 2025-12-02 17:39:28 +08:00
wxg0103 33da607302 feat: implement AWS Bedrock Vision-Language and Reranker models with credential validation 2025-12-02 14:27:38 +08:00
shaohuzhang1 bfae088df6
feat: knowledge workflow (#4399)
Some checks are pending
sync2gitee / repo-sync (push) Waiting to run
Typos Check / Spell Check with Typos (push) Waiting to run
* feat: init knowledge workflow

* feat: add knowledge workflow and version models, serializers, and API views

* feat: knowledge workflow

* feat: knowledge workflow

* feat: add KnowledgeWorkflowModelSerializer and Operate class for workflow management

* fix: route

* feat: knowledge workflow

* feat: Knowledge workflow permission

* feat: knowledge workflow

* feat: knowledge workflow

* feat: knowledge workflow

* feat: knowledge workflow

* feat: Data source web node

* fix: Back route

* feat: knowledge workflow

* feat: knowledge workflow

* feat: Knowledge write node

* feat: add Data Source tool functionality and localization

* feat: add Data Source tool functionality and localization

* feat: knowledge workflow

* feat: knowledge workflow

* fix: simplify export tool permission check in ToolListContainer.vue

* fix: simplify export condition in ToolResourceIndex.vue

* fix: simplify condition for copying tool in ToolListContainer

* feat: knowledge workflow

* fix: Upload local files and add output fields

* feat: Knowledge write

* feat: add Document Split Node functionality and localization

* feat: add Document Split Node functionality and localization

* feat: Knowledge write

* feat: enhance Document Split Node with result processing and problem list generation

* fix: Allow problem be blank

* feat: enhance Document Split Node with result processing and problem list generation

* feat: tool datasource

* fix: Optimization of knowledge base workflow execution logic

* refactor: streamline image handling by updating application and knowledge ID management

* refactor: streamline image handling by updating application and knowledge ID management

* feat: extend support modes in variable aggregation node to include knowledge workflows

* feat: Chunks stored

* refactor: simplify file handling in document extraction by removing unnecessary byte conversion and enhancing file saving logic

* refactor: update file ID assignment in document extraction to use provided metadata

* feat: Workflow menu that distinguishes between applications and knowledge bases

* refactor: update file ID assignment in document extraction to use provided metadata

* fix: Add workspace ID as workflow execution parameter

* feat: add code template for Data Source tool form functionality

* refactor: remove unused sys import and improve module handling

* feat: Execution details support loading status

* refactor: update tool type handling and improve category merging logic

* feat: Alter fork depth

* fix: ensure filterList is properly initialized and updated in getList function

* refactor: simplify ToolStoreDialog by removing unused toolType logic

* perf: Optimize the style

* style: adjust div width for improved layout in Tree component

* refactor: improve polling mechanism for knowledge workflow action

* fix: Get workspace_id from workflow params

* fix: filter out 'file_bytes' from result in get_details method

* feat: add recursive filtering for file_bytes in context data

* fix: append results to paragraph_list instead of replacing it

* perf: Optimize translation files

* fix: include document name in bytes_to_uploaded_file call for better file handling

* refactor: optimize buffer retrieval in document processing

* refactor: remove redundant parameter from bytes_to_uploaded_file call

* fix: Page style optimization

* feat: add slider for setting limit in document rules form

* feat: add workflow knowledge management endpoints and related functionality

* fix: swap file size and file count limits in form inputs

* refactor: update tool_config args to use list format for improved readability

* feat: Node supports knowledge base workflow

* feat: Node supports knowledge base workflow

* fix: Basic node data cannot be obtained in the workflow

* style: Knowledge base workflow debugging page style adjustment

* fix: Loop nodes cannot be used in the knowledge base workflow

* fix: Knowledge base workflow variable assignment node

* feat: add chunk size slider to form for custom split strategy

* fix: Workflow style optimization

---------

Co-authored-by: CaptainB <bin@fit2cloud.com>
Co-authored-by: zhangzhanwei <zhanwei.zhang@fit2cloud.com>
Co-authored-by: wangdan-fit2cloud <dan.wang@fit2cloud.com>
2025-11-28 15:38:20 +08:00
liqiang-fit2cloud 8c0836627a refactor: remove print.
Some checks are pending
sync2gitee / repo-sync (push) Waiting to run
Typos Check / Spell Check with Typos (push) Waiting to run
2025-11-19 17:16:22 +08:00
shaohuzhang1 a8d0729e65
perf: Memory optimization (#4318) 2025-11-05 19:05:26 +08:00
CaptainB d147b794ce chore: replace split_text with smart_split_paragraph in pdf_split_handle.py 2025-10-27 14:23:42 +08:00
shaohuzhang1 d92dcd722b
fix: Add file name to prompt when processing images with doc (#4114)
Some checks are pending
sync2gitee / repo-sync (push) Waiting to run
Typos Check / Spell Check with Typos (push) Waiting to run
2025-09-25 18:51:21 +08:00
shaohuzhang1 7264545ab6
feat: Support loop node (#4045) 2025-09-16 15:49:49 +08:00
CaptainB 75c461f385 chore: replace datetime.now() with timezone.now() for consistent time handling 2025-08-29 10:16:53 +08:00
CaptainB 4c9756839a chore: normalize with_filter parameter to boolean in split handle files
Some checks are pending
sync2gitee / repo-sync (push) Waiting to run
--bug=1057879 --user=刘瑞斌 【知识库】高级分段中自动清洗功能未生效 https://www.tapd.cn/62980211/s/1727744
2025-07-10 15:06:19 +08:00
CaptainB cb40d62162 refactor: allow loading of truncated images and increase max pixel limit in common_handle.py
--bug=1057749 --user=刘瑞斌 【知识库】qa问答对文档中带图片,导入后图片未显示 https://www.tapd.cn/62980211/s/1723700
2025-07-04 15:53:37 +08:00
CaptainB aa901c7fc7 fix: update file URL paths to use relative references 2025-07-02 22:45:11 +08:00
CaptainB 089915f488 refactor: improve error logging for image reading and enhance image handling logic
--bug=1057749 --user=刘瑞斌 【知识库】qa问答对文档中带图片,导入后图片未显示 https://www.tapd.cn/62980211/s/1720856
2025-07-01 14:17:10 +08:00
CaptainB 0f1d57f0cb feat: enhance error logging for file processing in CSV, XLS, and DOC handlers 2025-06-30 12:49:50 +08:00
CaptainB 82a2203be6 fix: handle string type for limit and improve error logging in pdf_split_handle
--bug=1057493 --user=刘瑞斌 【知识库】上传文档,使用高级分段报错 https://www.tapd.cn/62980211/s/1720110
2025-06-30 12:47:47 +08:00
CaptainB d49f448a5f fix: correct image path replacement logic in zip_split_handle 2025-06-26 17:02:34 +08:00
CaptainB 37ac79dc5a feat: import File model in zip_split_handle for enhanced functionality
--bug=1057478 --user=刘瑞斌 【知识库】通用知识库上传ZIP文件,分段失败 https://www.tapd.cn/62980211/s/1719181
2025-06-26 16:56:28 +08:00
CaptainB e24a2001c5 feat: refine regex patterns in text_split_handle for improved comment detection
--bug=1057526 --user=刘瑞斌 【知识库】markdown文件导入知识库,分段详情中代码块展示异常 https://www.tapd.cn/62980211/s/1719131
2025-06-26 16:23:32 +08:00
CaptainB a73e0b10f9 refactor: replace logging with maxkb_logger for consistent logging across modules 2025-06-25 17:00:18 +08:00
CaptainB fe8f87834d refactor: replace logging with maxkb_logger for consistent logging across modules 2025-06-25 16:46:50 +08:00
CaptainB 3aa0847506 refactor: replace print statements with logging for improved error tracking 2025-06-25 16:18:19 +08:00
wxg0103 c253e8b696 refactor: remove print 2025-06-24 15:30:42 +08:00
CaptainB 45908b91ff refactor: update dataset_id to knowledge_id in zip_split_handle.py and tools.py 2025-06-18 21:28:33 +08:00
CaptainB c0b770f41e refactor: update dataset_id to knowledge_id in zip_split_handle.py and tools.py 2025-06-18 21:15:53 +08:00
CaptainB 9a7281212d fix: update image URL paths to use OSS endpoints 2025-06-12 15:49:54 +08:00
wxg0103 b8b14884bd refactor: add application settings 2025-06-07 17:57:11 +08:00
wxg0103 93833849c1 refactor: file to oss
Some checks are pending
sync2gitee / repo-sync (push) Waiting to run
2025-06-06 11:42:31 +08:00
CaptainB c3581be9bd fix: rename image_name to file_name in zip_split_handle and remove workspace_id assignment in document 2025-05-13 12:47:59 +08:00
CaptainB e702af8c2b feat: enhance Document API with workspace ID support for get, put, and delete operations 2025-05-06 15:24:36 +08:00
CaptainB 43bef216d5 refactor: reorganize file handling imports into a structured directory 2025-04-30 16:08:17 +08:00
CaptainB 48297d81e5 feat: add initial implementations of various file handling classes for CSV, XLS, and XLSX formats 2025-04-30 15:52:58 +08:00
CaptainB c78a6babb6 ci: v2 2025-04-11 15:47:59 +08:00
CaptainB 560890f717 fix: limit chapter title length to 256 characters in pdf_split_handle.py
--bug=1054363 --user=刘瑞斌 【知识库】导入PDF文档,分段标题长度超长时,没有自动截断 https://www.tapd.cn/57709429/s/1681044
2025-04-07 10:54:59 +08:00
CaptainB 675adeeb63 fix: exclude macOS specific files from zip processing
--bug=1054264 --user=刘瑞斌 【知识库】QA问答对模式,导入在mac上压缩的zip文件,会出现2个乱码文档 https://www.tapd.cn/57709429/s/1681034
2025-04-07 10:37:06 +08:00
CaptainB 27bc01d442 fix: skip macOS specific metadata directories and files in zip parsing
--bug=1054264 --user=刘瑞斌 【知识库】QA问答对模式,导入在mac上压缩的zip文件,会出现2个乱码文档 https://www.tapd.cn/57709429/s/1679674
2025-04-02 16:06:36 +08:00
shaohuzhang1 9750c6d605
fix: garbled zip import file names (#2747) 2025-03-31 16:22:39 +08:00
shaohuzhang1 55cdd0a708
fix: Zip with title cannot be parsed (#2683) 2025-03-26 10:31:31 +08:00
shaohuzhang1 5ec94860b2
perf: Enhance Word parsing (#2612) 2025-03-19 12:04:43 +08:00
shaohuzhang1 e420a01e0d
fix: Enterprise WeChat docking sub application cannot output thinking process (#2489) 2025-03-04 19:31:49 +08:00
shaohuzhang1 8c45e92ee4
feat: The OpenAI interface supports the thought process (#2392) 2025-02-25 14:22:51 +08:00
CaptainB c524fbc0e4 fix: Fix excel merge cells header 2025-02-14 10:26:18 +08:00
CaptainB 89c08b4bb0 fix: Filter blank sheet
--bug=1052097 --user=刘瑞斌 【github#2196】【应用编排】应用对话的时候上传带空白sheet的表格会报错 https://www.tapd.cn/57709429/s/1653414
2025-02-11 15:17:24 +08:00
shaohuzhang1 f16f417bd5
fix: The knowledge base table file upload is missing a header (#2185) 2025-02-10 10:22:23 +08:00