This commit is contained in:
liqiang-fit2cloud 2024-04-24 14:49:59 +08:00
commit 5a0d423156
68 changed files with 1750 additions and 257 deletions

128
CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,128 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
support@fit2cloud.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

25
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,25 @@
# Contributing
## Create pull request
PR are always welcome, even if they only contain small fixes like typos or a few lines of code. If there will be a significant effort, please document it as an issue and get a discussion going before starting to work on it.
Please submit a PR broken down into small changes bit by bit. A PR consisting of a lot of features and code changes may be hard to review. It is recommended to submit PRs in an incremental fashion.
This [development guideline](https://github.com/1Panel-dev/MaxKB/wiki/3-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA) contains information about repository structure, how to set up development environment, how to run it, and more.
Note: If you split your pull request to small changes, please make sure any of the changes goes to master will not break anything. Otherwise, it can not be merged until this feature complete.
## Report issues
It is a great way to contribute by reporting an issue. Well-written and complete bug reports are always welcome! Please open an issue and follow the template to fill in required information.
Before opening any issue, please look up the existing issues to avoid submitting a duplication.
If you find a match, you can "subscribe" to it to get notified on updates. If you have additional helpful information about the issue, please leave a comment.
When reporting issues, always include:
* Which version you are using.
* Steps to reproduce the issue.
* Snapshots or log files if needed
Because the issues are open to the public, when submitting files, be sure to remove any sensitive information, e.g. user name, password, IP address, and company name. You can
replace those parts with "REDACTED" or other strings like "****".

View File

@ -13,7 +13,7 @@ MaxKB 是一款基于 LLM 大语言模型的知识库问答系统。MaxKB = Max
- **开箱即用**:支持直接上传文档、自动爬取在线文档,支持文本自动拆分、向量化,智能问答交互体验好;
- **无缝嵌入**:支持零编码快速嵌入到第三方业务系统;
- **多模型支持**:支持对接主流的大模型,包括本地私有大模型(如 Llama 2、Azure OpenAI 和百度千帆大模型等。
- **多模型支持**:支持对接主流的大模型,包括本地私有大模型(如 Llama 2、Llama 3)、通义千问、OpenAI、Azure OpenAI、Kimi 和百度千帆大模型等。
## 快速开始
@ -33,6 +33,9 @@ docker run -d --name=maxkb -p 8080:8080 -v ~/.maxkb:/var/lib/postgresql/data 1pa
- [使用手册](https://github.com/1Panel-dev/MaxKB/wiki/1-%E5%AE%89%E8%A3%85%E9%83%A8%E7%BD%B2)
- [论坛求助](https://bbs.fit2cloud.com/c/mk/11)
- [演示视频](https://www.bilibili.com/video/BV1BE421M7YM/)
- 技术交流群
<image height="100px" width="100px" src="https://github.com/1Panel-dev/MaxKB/assets/52996290/a4f6303d-9667-4be0-bc2d-0110af782f67"/>
## UI 展示
@ -53,7 +56,7 @@ docker run -d --name=maxkb -p 8080:8080 -v ~/.maxkb:/var/lib/postgresql/data 1pa
- 后端:[Python / Django](https://www.djangoproject.com/)
- LangChain[LangChain](https://www.langchain.com/)
- 向量数据库:[PostgreSQL / pgvector](https://www.postgresql.org/)
- 大模型Azure OpenAI、百度千帆大模型、[Ollama](https://github.com/ollama/ollama)
- 大模型Azure OpenAI、OpenAI、百度千帆大模型、[Ollama](https://github.com/ollama/ollama)、通义千问、Kimi
## Star History

17
SECURITY.md Normal file
View File

@ -0,0 +1,17 @@
# 安全说明
如果您发现安全问题,请直接联系我们:
- support@fit2cloud.com
- 400-052-0755
感谢您的支持!
# Security Policy
All security bugs should be reported to the contact as below:
- support@fit2cloud.com
- 400-052-0755
Thanks for your support!

View File

@ -6,15 +6,16 @@
@date2024/1/9 18:10
@desc: 检索知识库
"""
import re
from abc import abstractmethod
from typing import List, Type
from django.core import validators
from rest_framework import serializers
from application.chat_pipeline.I_base_chat_pipeline import IBaseChatPipelineStep, ParagraphPipelineModel
from application.chat_pipeline.pipeline_manage import PipelineManage
from common.util.field_message import ErrMessage
from dataset.models import Paragraph
class ISearchDatasetStep(IBaseChatPipelineStep):
@ -38,6 +39,10 @@ class ISearchDatasetStep(IBaseChatPipelineStep):
# 相似度 0-1之间
similarity = serializers.FloatField(required=True, max_value=1, min_value=0,
error_messages=ErrMessage.float("引用分段数"))
search_mode = serializers.CharField(required=True, validators=[
validators.RegexValidator(regex=re.compile("^embedding|keywords|blend$"),
message="类型只支持register|reset_password", code=500)
], error_messages=ErrMessage.char("检索模式"))
def get_step_serializer(self, manage: PipelineManage) -> Type[InstanceSerializer]:
return self.InstanceSerializer
@ -50,6 +55,7 @@ class ISearchDatasetStep(IBaseChatPipelineStep):
@abstractmethod
def execute(self, problem_text: str, dataset_id_list: list[str], exclude_document_id_list: list[str],
exclude_paragraph_id_list: list[str], top_n: int, similarity: float, padding_problem_text: str = None,
search_mode: str = None,
**kwargs) -> List[ParagraphPipelineModel]:
"""
关于 用户和补全问题 说明: 补全问题如果有就使用补全问题去查询 反之就用用户原始问题查询
@ -60,6 +66,7 @@ class ISearchDatasetStep(IBaseChatPipelineStep):
:param exclude_document_id_list: 需要排除的文档id
:param exclude_paragraph_id_list: 需要排除段落id
:param padding_problem_text 补全问题
:param search_mode 检索模式
:return: 段落列表
"""
pass

View File

@ -17,6 +17,7 @@ from common.config.embedding_config import VectorStore, EmbeddingModel
from common.db.search import native_search
from common.util.file_util import get_file_content
from dataset.models import Paragraph
from embedding.models import SearchMode
from smartdoc.conf import PROJECT_DIR
@ -24,13 +25,14 @@ class BaseSearchDatasetStep(ISearchDatasetStep):
def execute(self, problem_text: str, dataset_id_list: list[str], exclude_document_id_list: list[str],
exclude_paragraph_id_list: list[str], top_n: int, similarity: float, padding_problem_text: str = None,
search_mode: str = None,
**kwargs) -> List[ParagraphPipelineModel]:
exec_problem_text = padding_problem_text if padding_problem_text is not None else problem_text
embedding_model = EmbeddingModel.get_embedding_model()
embedding_value = embedding_model.embed_query(exec_problem_text)
vector = VectorStore.get_embedding_vector()
embedding_list = vector.query(embedding_value, dataset_id_list, exclude_document_id_list,
exclude_paragraph_id_list, True, top_n, similarity)
embedding_list = vector.query(exec_problem_text, embedding_value, dataset_id_list, exclude_document_id_list,
exclude_paragraph_id_list, True, top_n, similarity, SearchMode(search_mode))
if embedding_list is None:
return []
paragraph_list = self.list_paragraph([row.get('paragraph_id') for row in embedding_list], vector)

View File

@ -0,0 +1,18 @@
# Generated by Django 4.1.13 on 2024-04-23 11:16
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('application', '0002_chat_client_id'),
]
operations = [
migrations.AddField(
model_name='application',
name='icon',
field=models.CharField(default='/ui/favicon.ico', max_length=256, verbose_name='应用icon'),
),
]

View File

@ -19,7 +19,7 @@ from users.models import User
def get_dataset_setting_dict():
return {'top_n': 3, 'similarity': 0.6, 'max_paragraph_char_number': 5000}
return {'top_n': 3, 'similarity': 0.6, 'max_paragraph_char_number': 5000, 'search_mode': 'embedding'}
def get_model_setting_dict():
@ -37,6 +37,7 @@ class Application(AppModelMixin):
dataset_setting = models.JSONField(verbose_name="数据集参数设置", default=get_dataset_setting_dict)
model_setting = models.JSONField(verbose_name="模型参数相关设置", default=get_model_setting_dict)
problem_optimization = models.BooleanField(verbose_name="问题优化", default=False)
icon = models.CharField(max_length=256, verbose_name="应用icon", default="/ui/favicon.ico")
@staticmethod
def get_default_model_prompt():

View File

@ -8,12 +8,13 @@
"""
import hashlib
import os
import re
import uuid
from functools import reduce
from typing import Dict
from django.contrib.postgres.fields import ArrayField
from django.core import cache
from django.core import cache, validators
from django.core import signing
from django.db import transaction, models
from django.db.models import QuerySet
@ -28,10 +29,12 @@ from common.constants.authentication_type import AuthenticationType
from common.db.search import get_dynamics_model, native_search, native_page_search
from common.db.sql_execute import select_list
from common.exception.app_exception import AppApiException, NotFound404
from common.field.common import UploadedImageField
from common.util.field_message import ErrMessage
from common.util.file_util import get_file_content
from dataset.models import DataSet, Document
from dataset.models import DataSet, Document, Image
from dataset.serializers.common_serializers import list_paragraph
from embedding.models import SearchMode
from setting.models import AuthOperate
from setting.models.model_management import Model
from setting.models_provider.constants.model_provider_constants import ModelProvideConstants
@ -77,6 +80,10 @@ class DatasetSettingSerializer(serializers.Serializer):
error_messages=ErrMessage.float("相识度"))
max_paragraph_char_number = serializers.IntegerField(required=True, min_value=500, max_value=10000,
error_messages=ErrMessage.integer("最多引用字符数"))
search_mode = serializers.CharField(required=True, validators=[
validators.RegexValidator(regex=re.compile("^embedding|keywords|blend$"),
message="类型只支持register|reset_password", code=500)
], error_messages=ErrMessage.char("检索模式"))
class ModelSettingSerializer(serializers.Serializer):
@ -245,6 +252,7 @@ class ApplicationSerializer(serializers.Serializer):
# 问题补全
problem_optimization = serializers.BooleanField(required=False, allow_null=True,
error_messages=ErrMessage.boolean("问题补全"))
icon = serializers.CharField(required=False, allow_null=True, error_messages=ErrMessage.char("icon图标"))
class Create(serializers.Serializer):
user_id = serializers.UUIDField(required=True, error_messages=ErrMessage.uuid("用户id"))
@ -291,6 +299,10 @@ class ApplicationSerializer(serializers.Serializer):
error_messages=ErrMessage.integer("topN"))
similarity = serializers.FloatField(required=True, max_value=1, min_value=0,
error_messages=ErrMessage.float("相关度"))
search_mode = serializers.CharField(required=True, validators=[
validators.RegexValidator(regex=re.compile("^embedding|keywords|blend$"),
message="类型只支持register|reset_password", code=500)
], error_messages=ErrMessage.char("检索模式"))
def is_valid(self, *, raise_exception=False):
super().is_valid(raise_exception=True)
@ -312,6 +324,7 @@ class ApplicationSerializer(serializers.Serializer):
hit_list = vector.hit_test(self.data.get('query_text'), dataset_id_list, exclude_document_id_list,
self.data.get('top_number'),
self.data.get('similarity'),
SearchMode(self.data.get('search_mode')),
EmbeddingModel.get_embedding_model())
hit_dict = reduce(lambda x, y: {**x, **y}, [{hit.get('paragraph_id'): hit} for hit in hit_list], {})
p_list = list_paragraph([h.get('paragraph_id') for h in hit_list])
@ -369,6 +382,8 @@ class ApplicationSerializer(serializers.Serializer):
def reset_application(application: Dict):
application['multiple_rounds_dialogue'] = True if application.get('dialogue_number') > 0 else False
del application['dialogue_number']
if 'dataset_setting' in application:
application['dataset_setting'] = {**application['dataset_setting'], 'search_mode': 'embedding'}
return application
def page(self, current_page: int, page_size: int, with_valid=True):
@ -381,7 +396,25 @@ class ApplicationSerializer(serializers.Serializer):
class ApplicationModel(serializers.ModelSerializer):
class Meta:
model = Application
fields = ['id', 'name', 'desc', 'prologue', 'dialogue_number']
fields = ['id', 'name', 'desc', 'prologue', 'dialogue_number', 'icon']
class IconOperate(serializers.Serializer):
application_id = serializers.UUIDField(required=True, error_messages=ErrMessage.uuid("应用id"))
user_id = serializers.UUIDField(required=True, error_messages=ErrMessage.uuid("用户id"))
image = UploadedImageField(required=True, error_messages=ErrMessage.image("图片"))
def edit(self, with_valid=True):
if with_valid:
self.is_valid(raise_exception=True)
application = QuerySet(Application).filter(id=self.data.get('application_id')).first()
if application is None:
raise AppApiException(500, '不存在的应用id')
image_id = uuid.uuid1()
image = Image(id=image_id, image=self.data.get('image').read(), image_name=self.data.get('image').name)
image.save()
application.icon = f'/api/image/{image_id}'
application.save()
return {**ApplicationSerializer.Query.reset_application(ApplicationSerializerModel(application).data)}
class Operate(serializers.Serializer):
application_id = serializers.UUIDField(required=True, error_messages=ErrMessage.uuid("应用id"))
@ -445,7 +478,7 @@ class ApplicationSerializer(serializers.Serializer):
update_keys = ['name', 'desc', 'model_id', 'multiple_rounds_dialogue', 'prologue', 'status',
'dataset_setting', 'model_setting', 'problem_optimization',
'api_key_is_active']
'api_key_is_active', 'icon']
for update_key in update_keys:
if update_key in instance and instance.get(update_key) is not None:
if update_key == 'multiple_rounds_dialogue':

View File

@ -77,6 +77,8 @@ class ChatInfo:
'model_id': self.application.model.id if self.application.model is not None else None,
'problem_optimization': self.application.problem_optimization,
'stream': True,
'search_mode': self.application.dataset_setting.get(
'search_mode') if 'search_mode' in self.application.dataset_setting else 'embedding'
}
@ -184,9 +186,9 @@ class ChatMessageSerializer(serializers.Serializer):
pipeline_manage_builder.append_step(BaseResetProblemStep)
# 构建流水线管理器
pipeline_message = (pipeline_manage_builder.append_step(BaseSearchDatasetStep)
.append_step(BaseGenerateHumanMessageStep)
.append_step(BaseChatStep)
.build())
.append_step(BaseGenerateHumanMessageStep)
.append_step(BaseChatStep)
.build())
exclude_paragraph_id_list = []
# 相同问题是否需要排除已经查询到的段落
if re_chat:

View File

@ -12,6 +12,17 @@ from common.mixins.api_mixin import ApiMixin
class ApplicationApi(ApiMixin):
class EditApplicationIcon(ApiMixin):
@staticmethod
def get_request_params_api():
return [
openapi.Parameter(name='file',
in_=openapi.IN_FORM,
type=openapi.TYPE_FILE,
required=True,
description='上传文件')
]
class Authentication(ApiMixin):
@staticmethod
def get_request_body_api():
@ -143,7 +154,9 @@ class ApplicationApi(ApiMixin):
'dataset_setting': ApplicationApi.DatasetSetting.get_request_body_api(),
'model_setting': ApplicationApi.ModelSetting.get_request_body_api(),
'problem_optimization': openapi.Schema(type=openapi.TYPE_BOOLEAN, title="问题优化",
description="是否开启问题优化", default=True)
description="是否开启问题优化", default=True),
'icon': openapi.Schema(type=openapi.TYPE_STRING, title="icon",
description="icon", default="/ui/favicon.ico")
}
)
@ -161,6 +174,8 @@ class ApplicationApi(ApiMixin):
default=0.6),
'max_paragraph_char_number': openapi.Schema(type=openapi.TYPE_NUMBER, title='最多引用字符数',
description="最多引用字符数", default=3000),
'search_mode': openapi.Schema(type=openapi.TYPE_STRING, title='检索模式',
description="embedding|keywords|blend", default='embedding'),
}
)

View File

@ -8,6 +8,7 @@ urlpatterns = [
path('application/profile', views.Application.Profile.as_view()),
path('application/embed', views.Application.Embed.as_view()),
path('application/authentication', views.Application.Authentication.as_view()),
path('application/<str:application_id>/edit_icon', views.Application.EditIcon.as_view()),
path('application/<str:application_id>/statistics/customer_count',
views.ApplicationStatistics.CustomerCount.as_view()),
path('application/<str:application_id>/statistics/customer_count_trend',

View File

@ -10,6 +10,7 @@
from django.http import HttpResponse
from drf_yasg.utils import swagger_auto_schema
from rest_framework.decorators import action
from rest_framework.parsers import MultiPartParser
from rest_framework.request import Request
from rest_framework.views import APIView
@ -131,6 +132,28 @@ class ApplicationStatistics(APIView):
class Application(APIView):
authentication_classes = [TokenAuth]
class EditIcon(APIView):
authentication_classes = [TokenAuth]
parser_classes = [MultiPartParser]
@action(methods=['PUT'], detail=False)
@swagger_auto_schema(operation_summary="修改应用icon",
operation_id="修改应用icon",
tags=['应用'],
manual_parameters=ApplicationApi.EditApplicationIcon.get_request_params_api(),
request_body=ApplicationApi.Operate.get_request_body_api())
@has_permissions(ViewPermission(
[RoleConstants.ADMIN, RoleConstants.USER],
[lambda r, keywords: Permission(group=Group.APPLICATION, operate=Operate.MANAGE,
dynamic_tag=keywords.get('application_id'))],
compare=CompareConstants.AND), PermissionConstants.APPLICATION_EDIT,
compare=CompareConstants.AND)
def put(self, request: Request, application_id: str):
return result.success(
ApplicationSerializer.IconOperate(
data={'application_id': application_id, 'user_id': request.user.id,
'image': request.FILES.get('file')}).edit(request.data))
class Embed(APIView):
@action(methods=["GET"], detail=False)
@swagger_auto_schema(operation_summary="获取嵌入js",
@ -343,7 +366,8 @@ class Application(APIView):
ApplicationSerializer.HitTest(data={'id': application_id, 'user_id': request.user.id,
"query_text": request.query_params.get("query_text"),
"top_number": request.query_params.get("top_number"),
'similarity': request.query_params.get('similarity')}).hit_test(
'similarity': request.query_params.get('similarity'),
'search_mode': request.query_params.get('search_mode')}).hit_test(
))
class Operate(APIView):

View File

@ -34,7 +34,7 @@ class ApplicationKey(AuthBaseHandle):
return application_api_key.user, Auth(role_list=[RoleConstants.APPLICATION_KEY],
permission_list=permission_list,
application_id=application_api_key.application_id,
client_id=token,
client_id=str(application_api_key.id),
client_type=AuthenticationType.API_KEY.value)
def support(self, request, token: str, get_token_details):

View File

@ -32,3 +32,11 @@ class FunctionField(serializers.Field):
def to_representation(self, value):
return value
class UploadedImageField(serializers.ImageField):
def __init__(self, **kwargs):
super().__init__(**kwargs)
def to_representation(self, value):
return value

View File

@ -17,7 +17,14 @@ class StaticHeadersMiddleware(MiddlewareMixin):
if request.path.startswith('/ui/chat/'):
access_token = request.path.replace('/ui/chat/', '')
application_access_token = QuerySet(ApplicationAccessToken).filter(access_token=access_token).first()
if application_access_token is not None and application_access_token.white_active:
# 添加自定义的响应头
response['Content-Security-Policy'] = f'frame-ancestors {" ".join(application_access_token.white_list)}'
if application_access_token is not None:
if application_access_token.white_active:
# 添加自定义的响应头
response[
'Content-Security-Policy'] = f'frame-ancestors {" ".join(application_access_token.white_list)}'
response.content = (response.content.decode('utf-8').replace(
'<link rel="icon" href="/ui/favicon.ico" />',
f'<link rel="icon" href="{application_access_token.application.icon}" />')
.replace('<title>MaxKB</title>', f'<title>{application_access_token.application.name}</title>').encode(
"utf-8"))
return response

View File

@ -33,6 +33,13 @@ class CommonApi:
default=0.6,
required=True,
description='相关性'),
openapi.Parameter(name='search_mode',
in_=openapi.IN_QUERY,
type=openapi.TYPE_STRING,
default="embedding",
required=True,
description='检索模式embedding|keywords|blend'
)
]
@staticmethod

View File

@ -95,3 +95,12 @@ class ErrMessage:
'invalid': gettext_lazy('%s】日期格式错误。请改用以下格式之一: {format}'),
'datetime': gettext_lazy('%s】应为日期,但得到的是日期时间。')
}
@staticmethod
def image(field: str):
return {
'required': gettext_lazy('%s】此字段必填。' % field),
'null': gettext_lazy('%s】此字段不能为null。' % field),
'invalid_image': gettext_lazy('%s】上载有效的图像。您上载的文件不是图像或图像已损坏。' % field),
'max_length': gettext_lazy('请确保此文件名最多包含 {max_length} 个字符(长度为 {length})。')
}

View File

@ -0,0 +1,107 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file ts_vecto_util.py
@date2024/4/16 15:26
@desc:
"""
import re
import uuid
from typing import List
import jieba
from jieba import analyse
from common.util.split_model import group_by
jieba_word_list_cache = [chr(item) for item in range(38, 84)]
for jieba_word in jieba_word_list_cache:
jieba.add_word('#' + jieba_word + '#')
# r"(?i)\b(?:https?|ftp|tcp|file)://[^\s]+\b",
# 某些不分词数据
# r'"([^"]*)"'
word_pattern_list = [r"v\d+.\d+.\d+",
r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}"]
remove_chars = '\n , :\'<>@#¥%……&*!@#$%^&*() /"./-'
def get_word_list(text: str):
result = []
for pattern in word_pattern_list:
word_list = re.findall(pattern, text)
for child_list in word_list:
for word in child_list if isinstance(child_list, tuple) else [child_list]:
# 不能有: 所以再使用: 进行分割
if word.__contains__(':'):
item_list = word.split(":")
for w in item_list:
result.append(w)
else:
result.append(word)
return result
def replace_word(word_dict, text: str):
for key in word_dict:
text = re.sub('(?<!#)' + word_dict[key] + '(?!#)', key, text)
return text
def get_word_key(text: str, use_word_list):
for j_word in jieba_word_list_cache:
if not text.__contains__(j_word) and not use_word_list.__contains__(j_word):
return j_word
j_word = str(uuid.uuid1())
jieba.add_word(j_word)
return j_word
def to_word_dict(word_list: List, text: str):
word_dict = {}
for word in word_list:
key = get_word_key(text, set(word_dict))
word_dict['#' + key + '#'] = word
return word_dict
def get_key_by_word_dict(key, word_dict):
v = word_dict.get(key)
if v is None:
return key
return v
def to_ts_vector(text: str):
# 获取不分词的数据
word_list = get_word_list(text)
# 获取关键词关系
word_dict = to_word_dict(word_list, text)
# 替换字符串
text = replace_word(word_dict, text)
# 分词
result = jieba.tokenize(text, mode='search')
result_ = [{'word': get_key_by_word_dict(item[0], word_dict), 'index': item[1]} for item in result]
result_group = group_by(result_, lambda r: r['word'])
return " ".join(
[f"{key.lower()}:{','.join([str(item['index'] + 1) for item in result_group[key]][:20])}" for key in
result_group if
not remove_chars.__contains__(key) and len(key.strip()) >= 0])
def to_query(text: str):
# 获取不分词的数据
word_list = get_word_list(text)
# 获取关键词关系
word_dict = to_word_dict(word_list, text)
# 替换字符串
text = replace_word(word_dict, text)
extract_tags = analyse.extract_tags(text, topK=5, withWeight=True, allowPOS=('ns', 'n', 'vn', 'v', 'eng'))
result = " ".join([get_key_by_word_dict(word, word_dict) for word, score in extract_tags if
not remove_chars.__contains__(word)])
# 删除词库
for word in word_list:
jieba.del_word(word)
return result

View File

@ -0,0 +1,27 @@
# Generated by Django 4.1.13 on 2024-04-22 19:31
from django.db import migrations, models
import uuid
class Migration(migrations.Migration):
dependencies = [
('dataset', '0001_initial'),
]
operations = [
migrations.CreateModel(
name='Image',
fields=[
('create_time', models.DateTimeField(auto_now_add=True, verbose_name='创建时间')),
('update_time', models.DateTimeField(auto_now=True, verbose_name='修改时间')),
('id', models.UUIDField(default=uuid.uuid1, editable=False, primary_key=True, serialize=False, verbose_name='主键id')),
('image', models.BinaryField(verbose_name='图片数据')),
('image_name', models.CharField(default='', max_length=256, verbose_name='图片名称')),
],
options={
'db_table': 'image',
},
),
]

View File

@ -105,3 +105,12 @@ class ProblemParagraphMapping(AppModelMixin):
class Meta:
db_table = "problem_paragraph_mapping"
class Image(AppModelMixin):
id = models.UUIDField(primary_key=True, max_length=128, default=uuid.uuid1, editable=False, verbose_name="主键id")
image = models.BinaryField(verbose_name="图片数据")
image_name = models.CharField(max_length=256, verbose_name="图片名称", default="")
class Meta:
db_table = "image"

View File

@ -37,6 +37,7 @@ from common.util.split_model import get_split_model
from dataset.models.data_set import DataSet, Document, Paragraph, Problem, Type, ProblemParagraphMapping
from dataset.serializers.common_serializers import list_paragraph, MetaSerializer
from dataset.serializers.document_serializers import DocumentSerializers, DocumentInstanceSerializer
from embedding.models import SearchMode
from setting.models import AuthOperate
from smartdoc.conf import PROJECT_DIR
@ -457,6 +458,10 @@ class DataSetSerializers(serializers.ModelSerializer):
error_messages=ErrMessage.char("响应Top"))
similarity = serializers.FloatField(required=True, max_value=1, min_value=0,
error_messages=ErrMessage.char("相似度"))
search_mode = serializers.CharField(required=True, validators=[
validators.RegexValidator(regex=re.compile("^embedding|keywords|blend$"),
message="类型只支持register|reset_password", code=500)
], error_messages=ErrMessage.char("检索模式"))
def is_valid(self, *, raise_exception=True):
super().is_valid(raise_exception=True)
@ -474,6 +479,7 @@ class DataSetSerializers(serializers.ModelSerializer):
hit_list = vector.hit_test(self.data.get('query_text'), [self.data.get('id')], exclude_document_id_list,
self.data.get('top_number'),
self.data.get('similarity'),
SearchMode(self.data.get('search_mode')),
EmbeddingModel.get_embedding_model())
hit_dict = reduce(lambda x, y: {**x, **y}, [{hit.get('paragraph_id'): hit} for hit in hit_list], {})
p_list = list_paragraph([h.get('paragraph_id') for h in hit_list])

View File

@ -0,0 +1,42 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file image_serializers.py
@date2024/4/22 16:36
@desc:
"""
import uuid
from django.db.models import QuerySet
from django.http import HttpResponse
from rest_framework import serializers
from common.exception.app_exception import NotFound404
from common.field.common import UploadedImageField
from common.util.field_message import ErrMessage
from dataset.models import Image
class ImageSerializer(serializers.Serializer):
image = UploadedImageField(required=True, error_messages=ErrMessage.image("图片"))
def upload(self, with_valid=True):
if with_valid:
self.is_valid(raise_exception=True)
image_id = uuid.uuid1()
image = Image(id=image_id, image=self.data.get('image').read(), image_name=self.data.get('image').name)
image.save()
return f'/api/image/{image_id}'
class Operate(serializers.Serializer):
id = serializers.UUIDField(required=True)
def get(self, with_valid=True):
if with_valid:
self.is_valid(raise_exception=True)
image_id = self.data.get('id')
image = QuerySet(Image).filter(id=image_id).first()
if image is None:
raise NotFound404(404, "不存在的图片")
return HttpResponse(image.image, status=200, headers={'Content-Type': 'image/png'})

View File

@ -0,0 +1,22 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file image_api.py
@date2024/4/23 11:23
@desc:
"""
from drf_yasg import openapi
from common.mixins.api_mixin import ApiMixin
class ImageApi(ApiMixin):
@staticmethod
def get_request_params_api():
return [openapi.Parameter(name='file',
in_=openapi.IN_FORM,
type=openapi.TYPE_FILE,
required=True,
description='上传图片文件')
]

View File

@ -40,5 +40,6 @@ urlpatterns = [
path('dataset/<str:dataset_id>/problem/<int:current_page>/<int:page_size>', views.Problem.Page.as_view()),
path('dataset/<str:dataset_id>/problem/<str:problem_id>', views.Problem.Operate.as_view()),
path('dataset/<str:dataset_id>/problem/<str:problem_id>/paragraph', views.Problem.Paragraph.as_view()),
path('image/<str:image_id>', views.Image.Operate.as_view()),
path('image', views.Image.as_view())
]

View File

@ -10,3 +10,4 @@ from .dataset import *
from .document import *
from .paragraph import *
from .problem import *
from .image import *

View File

@ -111,7 +111,8 @@ class Dataset(APIView):
DataSetSerializers.HitTest(data={'id': dataset_id, 'user_id': request.user.id,
"query_text": request.query_params.get("query_text"),
"top_number": request.query_params.get("top_number"),
'similarity': request.query_params.get('similarity')}).hit_test(
'similarity': request.query_params.get('similarity'),
'search_mode': request.query_params.get('search_mode')}).hit_test(
))
class Operate(APIView):

View File

@ -0,0 +1,43 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file image.py
@date2024/4/22 16:23
@desc:
"""
from drf_yasg import openapi
from drf_yasg.utils import swagger_auto_schema
from rest_framework.decorators import action
from rest_framework.parsers import MultiPartParser
from rest_framework.views import APIView
from rest_framework.views import Request
from common.auth import TokenAuth
from common.response import result
from dataset.serializers.image_serializers import ImageSerializer
class Image(APIView):
authentication_classes = [TokenAuth]
parser_classes = [MultiPartParser]
@action(methods=['POST'], detail=False)
@swagger_auto_schema(operation_summary="上传图片",
operation_id="上传图片",
manual_parameters=[openapi.Parameter(name='file',
in_=openapi.IN_FORM,
type=openapi.TYPE_FILE,
required=True,
description='上传文件')],
tags=["图片"])
def post(self, request: Request):
return result.success(ImageSerializer(data={'image': request.FILES.get('file')}).upload())
class Operate(APIView):
@action(methods=['GET'], detail=False)
@swagger_auto_schema(operation_summary="获取图片",
operation_id="获取图片",
tags=["图片"])
def get(self, request: Request, image_id: str):
return ImageSerializer.Operate(data={'id': image_id}).get()

View File

@ -0,0 +1,56 @@
# Generated by Django 4.1.13 on 2024-04-16 11:43
import django.contrib.postgres.search
from django.db import migrations
from common.util.common import sub_array
from common.util.ts_vecto_util import to_ts_vector
from dataset.models import Status
from embedding.models import Embedding
def update_embedding_search_vector(embedding, paragraph_list):
paragraphs = [paragraph for paragraph in paragraph_list if paragraph.id == embedding.get('paragraph')]
if len(paragraphs) > 0:
content = paragraphs[0].title + paragraphs[0].content
return Embedding(id=embedding.get('id'), search_vector=to_ts_vector(content))
return Embedding(id=embedding.get('id'), search_vector="")
def save_keywords(apps, schema_editor):
document = apps.get_model("dataset", "Document")
embedding = apps.get_model("embedding", "Embedding")
paragraph = apps.get_model('dataset', 'Paragraph')
db_alias = schema_editor.connection.alias
document_list = document.objects.using(db_alias).all()
for document in document_list:
document.status = Status.embedding
document.save()
paragraph_list = paragraph.objects.using(db_alias).filter(document=document).all()
embedding_list = embedding.objects.using(db_alias).filter(document=document).values('id', 'search_vector',
'paragraph')
embedding_update_list = [update_embedding_search_vector(embedding, paragraph_list) for embedding
in embedding_list]
child_array = sub_array(embedding_update_list, 50)
for c in child_array:
try:
embedding.objects.using(db_alias).bulk_update(c, ['search_vector'])
except Exception as e:
print(e)
document.status = Status.success
document.save()
class Migration(migrations.Migration):
dependencies = [
('embedding', '0001_initial'),
]
operations = [
migrations.AddField(
model_name='embedding',
name='search_vector',
field=django.contrib.postgres.search.SearchVectorField(default='', verbose_name='分词'),
),
migrations.RunPython(save_keywords)
]

View File

@ -10,6 +10,7 @@ from django.db import models
from common.field.vector_field import VectorField
from dataset.models.data_set import Document, Paragraph, DataSet
from django.contrib.postgres.search import SearchVectorField
class SourceType(models.TextChoices):
@ -19,6 +20,12 @@ class SourceType(models.TextChoices):
TITLE = 2, '标题'
class SearchMode(models.TextChoices):
embedding = 'embedding'
keywords = 'keywords'
blend = 'blend'
class Embedding(models.Model):
id = models.CharField(max_length=128, primary_key=True, verbose_name="主键id")
@ -37,6 +44,8 @@ class Embedding(models.Model):
embedding = VectorField(verbose_name="向量")
search_vector = SearchVectorField(verbose_name="分词", default="")
meta = models.JSONField(verbose_name="元数据", default=dict)
class Meta:

View File

@ -0,0 +1,26 @@
SELECT
paragraph_id,
comprehensive_score,
comprehensive_score AS similarity
FROM
(
SELECT DISTINCT ON
( "paragraph_id" ) ( similarity ),* ,
similarity AS comprehensive_score
FROM
(
SELECT
*,
(( 1 - ( embedding.embedding <=> %s ) )+ts_rank_cd( embedding.search_vector, websearch_to_tsquery('simple', %s ), 32 )) AS similarity
FROM
embedding ${embedding_query}
) TEMP
ORDER BY
paragraph_id,
similarity DESC
) DISTINCT_TEMP
WHERE
comprehensive_score >%s
ORDER BY
comprehensive_score DESC
LIMIT %s

View File

@ -0,0 +1,17 @@
SELECT
paragraph_id,
comprehensive_score,
comprehensive_score as similarity
FROM
(
SELECT DISTINCT ON
("paragraph_id") ( similarity ),* ,similarity AS comprehensive_score
FROM
( SELECT *,ts_rank_cd(embedding.search_vector,websearch_to_tsquery('simple',%s),32) AS similarity FROM embedding ${keywords_query}) TEMP
ORDER BY
paragraph_id,
similarity DESC
) DISTINCT_TEMP
WHERE comprehensive_score>%s
ORDER BY comprehensive_score DESC
LIMIT %s

View File

@ -14,7 +14,7 @@ from langchain_community.embeddings import HuggingFaceEmbeddings
from common.config.embedding_config import EmbeddingModel
from common.util.common import sub_array
from embedding.models import SourceType
from embedding.models import SourceType, SearchMode
lock = threading.Lock()
@ -113,13 +113,16 @@ class BaseVectorStore(ABC):
return result[0]
@abstractmethod
def query(self, query_embedding: List[float], dataset_id_list: list[str], exclude_document_id_list: list[str],
exclude_paragraph_list: list[str], is_active: bool, top_n: int, similarity: float):
def query(self, query_text:str,query_embedding: List[float], dataset_id_list: list[str],
exclude_document_id_list: list[str],
exclude_paragraph_list: list[str], is_active: bool, top_n: int, similarity: float,
search_mode: SearchMode):
pass
@abstractmethod
def hit_test(self, query_text, dataset_id: list[str], exclude_document_id_list: list[str], top_number: int,
similarity: float,
search_mode: SearchMode,
embedding: HuggingFaceEmbeddings):
pass

View File

@ -9,6 +9,7 @@
import json
import os
import uuid
from abc import ABC, abstractmethod
from typing import Dict, List
from django.db.models import QuerySet
@ -18,7 +19,8 @@ from common.config.embedding_config import EmbeddingModel
from common.db.search import generate_sql_by_query_dict
from common.db.sql_execute import select_list
from common.util.file_util import get_file_content
from embedding.models import Embedding, SourceType
from common.util.ts_vecto_util import to_ts_vector, to_query
from embedding.models import Embedding, SourceType, SearchMode
from embedding.vector.base_vector import BaseVectorStore
from smartdoc.conf import PROJECT_DIR
@ -57,7 +59,8 @@ class PGVector(BaseVectorStore):
paragraph_id=paragraph_id,
source_id=source_id,
embedding=text_embedding,
source_type=source_type)
source_type=source_type,
search_vector=to_ts_vector(text))
embedding.save()
return True
@ -71,13 +74,15 @@ class PGVector(BaseVectorStore):
is_active=text_list[index].get('is_active', True),
source_id=text_list[index].get('source_id'),
source_type=text_list[index].get('source_type'),
embedding=embeddings[index]) for index in
embedding=embeddings[index],
search_vector=to_ts_vector(text_list[index]['text'])) for index in
range(0, len(text_list))]
QuerySet(Embedding).bulk_create(embedding_list) if len(embedding_list) > 0 else None
return True
def hit_test(self, query_text, dataset_id_list: list[str], exclude_document_id_list: list[str], top_number: int,
similarity: float,
search_mode: SearchMode,
embedding: HuggingFaceEmbeddings):
if dataset_id_list is None or len(dataset_id_list) == 0:
return []
@ -87,17 +92,14 @@ class PGVector(BaseVectorStore):
if exclude_document_id_list is not None and len(exclude_document_id_list) > 0:
exclude_dict.__setitem__('document_id__in', exclude_document_id_list)
query_set = query_set.exclude(**exclude_dict)
exec_sql, exec_params = generate_sql_by_query_dict({'embedding_query': query_set},
select_string=get_file_content(
os.path.join(PROJECT_DIR, "apps", "embedding", 'sql',
'hit_test.sql')),
with_table_name=True)
embedding_model = select_list(exec_sql,
[json.dumps(embedding_query), *exec_params, similarity, top_number])
return embedding_model
for search_handle in search_handle_list:
if search_handle.support(search_mode):
return search_handle.handle(query_set, query_text, embedding_query, top_number, similarity, search_mode)
def query(self, query_embedding: List[float], dataset_id_list: list[str], exclude_document_id_list: list[str],
exclude_paragraph_list: list[str], is_active: bool, top_n: int, similarity: float):
def query(self, query_text: str, query_embedding: List[float], dataset_id_list: list[str],
exclude_document_id_list: list[str],
exclude_paragraph_list: list[str], is_active: bool, top_n: int, similarity: float,
search_mode: SearchMode):
exclude_dict = {}
if dataset_id_list is None or len(dataset_id_list) == 0:
return []
@ -107,14 +109,9 @@ class PGVector(BaseVectorStore):
if exclude_paragraph_list is not None and len(exclude_paragraph_list) > 0:
exclude_dict.__setitem__('paragraph_id__in', exclude_paragraph_list)
query_set = query_set.exclude(**exclude_dict)
exec_sql, exec_params = generate_sql_by_query_dict({'embedding_query': query_set},
select_string=get_file_content(
os.path.join(PROJECT_DIR, "apps", "embedding", 'sql',
'embedding_search.sql')),
with_table_name=True)
embedding_model = select_list(exec_sql,
[json.dumps(query_embedding), *exec_params, similarity, top_n])
return embedding_model
for search_handle in search_handle_list:
if search_handle.support(search_mode):
return search_handle.handle(query_set, query_text, query_embedding, top_n, similarity, search_mode)
def update_by_source_id(self, source_id: str, instance: Dict):
QuerySet(Embedding).filter(source_id=source_id).update(**instance)
@ -141,3 +138,81 @@ class PGVector(BaseVectorStore):
def delete_by_paragraph_id(self, paragraph_id: str):
QuerySet(Embedding).filter(paragraph_id=paragraph_id).delete()
class ISearch(ABC):
@abstractmethod
def support(self, search_mode: SearchMode):
pass
@abstractmethod
def handle(self, query_set, query_text, query_embedding, top_number: int,
similarity: float, search_mode: SearchMode):
pass
class EmbeddingSearch(ISearch):
def handle(self,
query_set,
query_text,
query_embedding,
top_number: int,
similarity: float,
search_mode: SearchMode):
exec_sql, exec_params = generate_sql_by_query_dict({'embedding_query': query_set},
select_string=get_file_content(
os.path.join(PROJECT_DIR, "apps", "embedding", 'sql',
'embedding_search.sql')),
with_table_name=True)
embedding_model = select_list(exec_sql,
[json.dumps(query_embedding), *exec_params, similarity, top_number])
return embedding_model
def support(self, search_mode: SearchMode):
return search_mode.value == SearchMode.embedding.value
class KeywordsSearch(ISearch):
def handle(self,
query_set,
query_text,
query_embedding,
top_number: int,
similarity: float,
search_mode: SearchMode):
exec_sql, exec_params = generate_sql_by_query_dict({'keywords_query': query_set},
select_string=get_file_content(
os.path.join(PROJECT_DIR, "apps", "embedding", 'sql',
'keywords_search.sql')),
with_table_name=True)
embedding_model = select_list(exec_sql,
[to_query(query_text), *exec_params, similarity, top_number])
return embedding_model
def support(self, search_mode: SearchMode):
return search_mode.value == SearchMode.keywords.value
class BlendSearch(ISearch):
def handle(self,
query_set,
query_text,
query_embedding,
top_number: int,
similarity: float,
search_mode: SearchMode):
exec_sql, exec_params = generate_sql_by_query_dict({'embedding_query': query_set},
select_string=get_file_content(
os.path.join(PROJECT_DIR, "apps", "embedding", 'sql',
'blend_search.sql')),
with_table_name=True)
embedding_model = select_list(exec_sql,
[json.dumps(query_embedding), to_query(query_text), *exec_params, similarity,
top_number])
return embedding_model
def support(self, search_mode: SearchMode):
return search_mode.value == SearchMode.blend.value
search_handle_list = [EmbeddingSearch(), KeywordsSearch(), BlendSearch()]

View File

@ -14,6 +14,8 @@ from setting.models_provider.impl.openai_model_provider.openai_model_provider im
from setting.models_provider.impl.qwen_model_provider.qwen_model_provider import QwenModelProvider
from setting.models_provider.impl.wenxin_model_provider.wenxin_model_provider import WenxinModelProvider
from setting.models_provider.impl.kimi_model_provider.kimi_model_provider import KimiModelProvider
from setting.models_provider.impl.xf_model_provider.xf_model_provider import XunFeiModelProvider
from setting.models_provider.impl.zhipu_model_provider.zhipu_model_provider import ZhiPuModelProvider
class ModelProvideConstants(Enum):
@ -23,3 +25,5 @@ class ModelProvideConstants(Enum):
model_openai_provider = OpenAIModelProvider()
model_kimi_provider = KimiModelProvider()
model_qwen_provider = QwenModelProvider()
model_zhipu_provider = ZhiPuModelProvider()
model_xf_provider = XunFeiModelProvider()

View File

@ -0,0 +1,8 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file __init__.py.py
@date2024/04/19 15:55
@desc:
"""

View File

@ -0,0 +1 @@
<svg t="1713509569091" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="4361" xmlns:xlink="http://www.w3.org/1999/xlink" width="100%" height="100%" ><path d="M500.1216 971.40736c-55.58272-4.28032-102.11328-16.7936-147.74272-39.7312-115.0976-57.83552-192.88064-168.5504-208.81408-297.12384-2.8672-23.18336-2.90816-69.18144-0.06144-93.3888a387.95264 387.95264 0 0 1 82.65728-196.46464c6.22592-7.7824 47.616-49.7664 94.49472-95.92832 112.8448-111.104 111.53408-109.85472 113.29536-108.09344 1.024 1.024 0.75776 5.59104-0.8192 14.45888-3.13344 17.57184-3.13344 55.99232 0.04096 77.0048 9.66656 64.49152 37.66272 124.3136 87.16288 186.32704 12.6976 15.91296 59.22816 63.24224 76.57472 77.88544 20.13184 16.9984 33.1776 37.53984 39.34208 61.8496 4.07552 16.0768 4.07552 42.10688 0 58.20416-10.24 40.57088-40.8576 72.58112-81.6128 85.38112-9.35936 2.92864-13.84448 3.39968-32.68608 3.39968s-23.3472-0.47104-32.768-3.39968c-29.02016-9.07264-56.40192-30.06464-32.52224-24.94464 12.94336 2.7648 29.65504-3.2768 37.49888-13.57824 10.81344-14.1312 12.57472-29.53216 5.09952-44.48256-3.76832-7.53664-6.8608-10.91584-19.12832-20.82816-33.1776-26.86976-65.7408-59.5968-88.8832-89.25184-11.81696-15.17568-28.8768-40.59136-33.95584-50.5856-1.92512-3.7888-4.15744-6.90176-4.95616-6.90176-2.00704 0-17.92 24.43264-24.73984 37.96992-7.84384 15.52384-15.33952 37.888-19.12832 57.0368-4.64896 23.3472-4.64896 59.14624-0.04096 82.16576 17.77664 88.63744 82.78016 154.74688 171.02848 173.8752 12.3904 2.70336 19.39456 3.23584 42.496 3.23584 23.08096 0 30.1056-0.53248 42.47552-3.23584 43.04896-9.3184 78.4384-28.672 109.6704-59.904 32.72704-32.72704 52.26496-69.44768 61.29664-115.3024 4.54656-23.06048 4.56704-56.36096 0.04096-79.29856-3.87072-19.5584-8.76544-35.16416-15.85152-50.3808-6.69696-14.35648-6.0416-15.21664 9.1136-12.0832 40.89856 8.45824 85.6064 31.41632 114.40128 58.75712 34.6112 32.84992 49.27488 65.45408 49.27488 109.71136 0 24.00256-3.4816 41.6768-13.35296 68.17792-20.54144 54.9888-50.54464 100.61824-93.7984 142.66368-51.26144 49.80736-116.8384 85.03296-183.95136 98.79552-30.45376 6.2464-76.53376 9.89184-101.1712 7.9872z m391.53664-433.93024c-17.05984-32.93184-41.75872-56.48384-76.8-73.19552-18.80064-8.97024-35.67616-14.52032-68.75136-22.58944-44.46208-10.8544-66.2528-18.2272-93.16352-31.62112-26.2144-13.04576-46.16192-27.3408-66.3552-47.5136-26.70592-26.74688-42.63936-52.4288-54.35392-87.67488-10.36288-31.27296-10.0352-27.2384-10.62912-128.96256-0.45056-78.37696-0.2048-92.0576 1.51552-92.0576 1.1264 0 45.8752 42.98752 99.40992 95.51872 190.95552 187.37152 194.58048 191.11936 216.7808 224.78848 20.13184 30.53568 39.26016 72.0896 48.76288 105.92256 4.7104 16.73216 11.0592 48.18944 11.81696 58.40896 0.7168 9.8304-2.84672 9.35936-8.23296-1.024z" fill="#3DC8F9" p-id="4362"></path><path d="M523.12064 53.8624c-1.7408 0-1.96608 13.68064-1.51552 92.0576 0.57344 101.74464 0.24576 97.6896 10.6496 128.96256 11.6736 35.2256 27.62752 60.928 54.33344 87.6544 20.19328 20.19328 40.1408 34.48832 66.3552 47.5136 26.91072 13.4144 48.70144 20.80768 93.14304 31.6416 33.09568 8.0896 49.9712 13.6192 68.75136 22.58944 35.04128 16.71168 59.74016 40.2432 76.8 73.19552 5.40672 10.38336 8.97024 10.8544 8.25344 1.024-0.75776-10.21952-7.12704-41.6768-11.81696-58.40896-9.50272-33.83296-28.63104-75.3664-48.76288-105.92256-22.20032-33.66912-25.82528-37.41696-216.7808-224.78848-53.5552-52.5312-98.304-95.51872-99.40992-95.51872z" fill="#EA0100" p-id="4363"></path><path d="M391.3728 762.30656s86.2208 88.41216 241.9712 100.06528c155.7504 11.63264 193.536-45.44512 193.536-45.44512s76.3904-100.864 65.08544-177.54112c-11.32544-76.67712-71.02464-131.21536-174.96064-154.89024 0 0 31.90784 80.2816 20.5824 128.14336-11.32544 47.86176-20.0704 138.93632-159.5392 186.7776 0 0-102.68672 30.208-186.6752-37.10976z" fill="#1652D8" p-id="4364"></path></svg>

After

Width:  |  Height:  |  Size: 3.8 KiB

View File

@ -0,0 +1,43 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file __init__.py.py
@date2024/04/19 15:55
@desc:
"""
from typing import List, Optional, Any, Iterator
from langchain_community.chat_models import ChatSparkLLM
from langchain_community.chat_models.sparkllm import _convert_message_to_dict, _convert_delta_to_message_chunk
from langchain_core.callbacks import CallbackManagerForLLMRun
from langchain_core.messages import BaseMessage, AIMessageChunk
from langchain_core.outputs import ChatGenerationChunk
class XFChatSparkLLM(ChatSparkLLM):
def _stream(
self,
messages: List[BaseMessage],
stop: Optional[List[str]] = None,
run_manager: Optional[CallbackManagerForLLMRun] = None,
**kwargs: Any,
) -> Iterator[ChatGenerationChunk]:
default_chunk_class = AIMessageChunk
self.client.arun(
[_convert_message_to_dict(m) for m in messages],
self.spark_user_id,
self.model_kwargs,
True,
)
for content in self.client.subscribe(timeout=self.request_timeout):
if "data" not in content:
continue
delta = content["data"]
chunk = _convert_delta_to_message_chunk(delta, default_chunk_class)
cg_chunk = ChatGenerationChunk(message=chunk)
if run_manager:
run_manager.on_llm_new_token(str(chunk.content), chunk=cg_chunk)
yield cg_chunk

View File

@ -0,0 +1,103 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file xf_model_provider.py
@date2024/04/19 14:47
@desc:
"""
import os
from typing import Dict
from langchain.schema import HumanMessage
from langchain_community.chat_models import ChatSparkLLM
from common import forms
from common.exception.app_exception import AppApiException
from common.forms import BaseForm
from common.util.file_util import get_file_content
from setting.models_provider.base_model_provider import ModelProvideInfo, ModelTypeConst, BaseModelCredential, \
ModelInfo, IModelProvider, ValidCode
from setting.models_provider.impl.xf_model_provider.model.xf_chat_model import XFChatSparkLLM
from smartdoc.conf import PROJECT_DIR
import ssl
ssl._create_default_https_context = ssl.create_default_context()
class XunFeiLLMModelCredential(BaseForm, BaseModelCredential):
def is_valid(self, model_type: str, model_name, model_credential: Dict[str, object], raise_exception=False):
model_type_list = XunFeiModelProvider().get_model_type_list()
if not any(list(filter(lambda mt: mt.get('value') == model_type, model_type_list))):
raise AppApiException(ValidCode.valid_error.value, f'{model_type} 模型类型不支持')
for key in ['spark_api_url', 'spark_app_id', 'spark_api_key', 'spark_api_secret']:
if key not in model_credential:
if raise_exception:
raise AppApiException(ValidCode.valid_error.value, f'{key} 字段为必填字段')
else:
return False
try:
model = XunFeiModelProvider().get_model(model_type, model_name, model_credential)
model.invoke([HumanMessage(content='你好')])
except Exception as e:
if isinstance(e, AppApiException):
raise e
if raise_exception:
raise AppApiException(ValidCode.valid_error.value, f'校验失败,请检查参数是否正确: {str(e)}')
else:
return False
return True
def encryption_dict(self, model: Dict[str, object]):
return {**model, 'spark_api_secret': super().encryption(model.get('spark_api_secret', ''))}
spark_api_url = forms.TextInputField('API 域名', required=True)
spark_app_id = forms.TextInputField('APP ID', required=True)
spark_api_key = forms.PasswordInputField("API Key", required=True)
spark_api_secret = forms.PasswordInputField('API Secret', required=True)
qwen_model_credential = XunFeiLLMModelCredential()
model_dict = {
'generalv3.5': ModelInfo('generalv3.5', '', ModelTypeConst.LLM, qwen_model_credential),
'generalv3': ModelInfo('generalv3', '', ModelTypeConst.LLM, qwen_model_credential),
'generalv2': ModelInfo('generalv2', '', ModelTypeConst.LLM, qwen_model_credential)
}
class XunFeiModelProvider(IModelProvider):
def get_dialogue_number(self):
return 3
def get_model(self, model_type, model_name, model_credential: Dict[str, object], **model_kwargs) -> XFChatSparkLLM:
zhipuai_chat = XFChatSparkLLM(
spark_app_id=model_credential.get('spark_app_id'),
spark_api_key=model_credential.get('spark_api_key'),
spark_api_secret=model_credential.get('spark_api_secret'),
spark_api_url=model_credential.get('spark_api_url'),
spark_llm_domain=model_name
)
return zhipuai_chat
def get_model_credential(self, model_type, model_name):
if model_name in model_dict:
return model_dict.get(model_name).model_credential
return qwen_model_credential
def get_model_provide_info(self):
return ModelProvideInfo(provider='model_xf_provider', name='讯飞星火', icon=get_file_content(
os.path.join(PROJECT_DIR, "apps", "setting", 'models_provider', 'impl', 'xf_model_provider', 'icon',
'xf_icon_svg')))
def get_model_list(self, model_type: str):
if model_type is None:
raise AppApiException(500, '模型类型不能为空')
return [model_dict.get(key).to_dict() for key in
list(filter(lambda key: model_dict.get(key).model_type == model_type, model_dict.keys()))]
def get_model_type_list(self):
return [{'key': "大语言模型", 'value': "LLM"}]

View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="100%" height="100%" fill="none" viewBox="0 0 30 32" id="icon-pure-logo"><g clip-path="url(#icon-pure-logo_a)" fill="#1665FF"><path d="M15 32a.377.377 0 0 0 .377-.375.377.377 0 0 0-.378-.375.377.377 0 0 0-.378.375c0 .207.17.375.378.375Zm3.907-20.375a2.26 2.26 0 0 0 2.27-2.25 2.26 2.26 0 0 0-2.27-2.25 2.26 2.26 0 0 0-2.269 2.25 2.26 2.26 0 0 0 2.27 2.25Zm3.908 6.625a2.26 2.26 0 0 0 2.27-2.25 2.26 2.26 0 0 0-2.27-2.25A2.26 2.26 0 0 0 20.546 16a2.26 2.26 0 0 0 2.27 2.25Zm-15.63 0A2.26 2.26 0 0 0 9.454 16a2.26 2.26 0 0 0-2.27-2.25A2.26 2.26 0 0 0 4.917 16a2.26 2.26 0 0 0 2.269 2.25Zm17.647-6.501a1.38 1.38 0 0 0 1.386-1.375A1.38 1.38 0 0 0 24.832 9a1.38 1.38 0 0 0-1.386 1.374c0 .76.621 1.375 1.386 1.375ZM15 6.25a1.38 1.38 0 0 0 1.385-1.375c0-.76-.62-1.375-1.386-1.375a1.38 1.38 0 0 0-1.386 1.375c0 .759.62 1.374 1.386 1.374Zm-9.833 5.499a1.38 1.38 0 0 0 1.386-1.375A1.38 1.38 0 0 0 5.167 9a1.38 1.38 0 0 0-1.386 1.374c0 .76.621 1.375 1.386 1.375Zm0 11.251a1.38 1.38 0 0 0 1.386-1.374c0-.76-.62-1.375-1.386-1.375a1.38 1.38 0 0 0-1.386 1.375A1.38 1.38 0 0 0 5.167 23ZM15 28.625a1.38 1.38 0 0 0 1.385-1.374c0-.76-.62-1.375-1.386-1.375a1.38 1.38 0 0 0-1.386 1.375c0 .759.62 1.374 1.386 1.374ZM24.832 23a1.38 1.38 0 0 0 1.386-1.374c0-.76-.62-1.375-1.386-1.375a1.38 1.38 0 0 0-1.386 1.375A1.38 1.38 0 0 0 24.832 23ZM22.059 4.751a.88.88 0 0 0 .883-.875.88.88 0 0 0-.883-.876.88.88 0 0 0-.883.876.88.88 0 0 0 .883.875Zm-14.118 0a.88.88 0 0 0 .883-.875A.88.88 0 0 0 7.94 3a.88.88 0 0 0-.883.876.88.88 0 0 0 .883.875ZM.883 16.876a.88.88 0 0 0 .883-.875.88.88 0 0 0-.883-.876.88.88 0 0 0-.883.876.88.88 0 0 0 .883.875ZM7.941 29a.88.88 0 0 0 .883-.875.88.88 0 0 0-.883-.876.88.88 0 0 0-.883.876.88.88 0 0 0 .883.875Zm14.118 0a.88.88 0 0 0 .883-.875.88.88 0 0 0-.883-.876.88.88 0 0 0-.883.876.88.88 0 0 0 .883.875Zm7.058-12.124a.88.88 0 0 0 .883-.875.88.88 0 0 0-.883-.876.88.88 0 0 0-.883.876.88.88 0 0 0 .883.875Zm-.503-8.251a.377.377 0 0 0 .378-.375.377.377 0 0 0-.378-.375.377.377 0 0 0-.378.375c0 .207.17.375.378.375ZM15 .75a.377.377 0 0 0 .377-.375A.377.377 0 0 0 15 0a.377.377 0 0 0-.378.375c0 .207.17.375.378.375ZM1.386 8.625a.377.377 0 0 0 .378-.375.377.377 0 0 0-.378-.375.377.377 0 0 0-.378.375c0 .207.17.375.378.375Zm0 15.625a.377.377 0 0 0 .378-.374.377.377 0 0 0-.378-.375.377.377 0 0 0-.378.375c0 .207.17.375.378.375Zm27.228-.125a.377.377 0 0 0 .378-.375.377.377 0 0 0-.378-.375.377.377 0 0 0-.378.375c0 .207.17.375.378.375Z"></path><path d="M19.538 20.5c-1.007-.374-2.142.126-2.646 1-.505.876-1.513 1.375-2.647 1-.63-.126-1.008-.625-1.261-1 0-.125-.127-.25-.127-.5 0-1 .756-1.75 1.64-1.875h.25c1.765.125 3.277-1.375 3.404-3.126.127-1.75-1.386-3.25-3.152-3.375h-.378c-1.008 0-1.64-.75-1.64-1.75 0-.249 0-.5.127-.624v-.125c0-.126.127-.126.127-.25.378-1.25-.252-2.5-1.512-2.874-1.135-.375-2.52.25-2.9 1.5-.377 1.125.252 2.375 1.387 2.874.168.084.336.126.505.126h.126c.883.125 1.64.875 1.64 1.75 0 .374-.127.624-.252.875-.252.5-.505 1-.505 1.625 0 .626.127 1.375.505 1.875.126.25.251.625.251.876 0 .875-.63 1.625-1.512 1.75h-.378c-1.261.249-2.018 1.5-1.764 2.75.253 1.25 1.512 2 2.646 1.75.63-.126 1.135-.501 1.386-1 .505-.876 1.64-1.375 2.647-1 .505.126 1.008.5 1.262 1 .251.375.756.75 1.26 1 1.262.374 2.396-.25 2.9-1.5.504-1.126-.127-2.376-1.387-2.751h-.002Z"></path></g><defs><clipPath id="icon-pure-logo_a"><path fill="#fff" d="M0 0h30v32H0z"></path></clipPath></defs></svg>

After

Width:  |  Height:  |  Size: 3.3 KiB

View File

@ -0,0 +1,93 @@
# coding=utf-8
"""
@project: maxkb
@Author
@file zhipu_model_provider.py
@date2024/04/19 13:5
@desc:
"""
import os
from typing import Dict
from langchain.schema import HumanMessage
from langchain_community.chat_models import ChatZhipuAI
from common import forms
from common.exception.app_exception import AppApiException
from common.forms import BaseForm
from common.util.file_util import get_file_content
from setting.models_provider.base_model_provider import ModelProvideInfo, ModelTypeConst, BaseModelCredential, \
ModelInfo, IModelProvider, ValidCode
from smartdoc.conf import PROJECT_DIR
class ZhiPuLLMModelCredential(BaseForm, BaseModelCredential):
def is_valid(self, model_type: str, model_name, model_credential: Dict[str, object], raise_exception=False):
model_type_list = ZhiPuModelProvider().get_model_type_list()
if not any(list(filter(lambda mt: mt.get('value') == model_type, model_type_list))):
raise AppApiException(ValidCode.valid_error.value, f'{model_type} 模型类型不支持')
for key in ['api_key']:
if key not in model_credential:
if raise_exception:
raise AppApiException(ValidCode.valid_error.value, f'{key} 字段为必填字段')
else:
return False
try:
model = ZhiPuModelProvider().get_model(model_type, model_name, model_credential)
model.invoke([HumanMessage(content='你好')])
except Exception as e:
if isinstance(e, AppApiException):
raise e
if raise_exception:
raise AppApiException(ValidCode.valid_error.value, f'校验失败,请检查参数是否正确: {str(e)}')
else:
return False
return True
def encryption_dict(self, model: Dict[str, object]):
return {**model, 'api_key': super().encryption(model.get('api_key', ''))}
api_key = forms.PasswordInputField('API Key', required=True)
qwen_model_credential = ZhiPuLLMModelCredential()
model_dict = {
'glm-4': ModelInfo('glm-4', '', ModelTypeConst.LLM, qwen_model_credential),
'glm-4v': ModelInfo('glm-4v', '', ModelTypeConst.LLM, qwen_model_credential),
'glm-3-turbo': ModelInfo('glm-3-turbo', '', ModelTypeConst.LLM, qwen_model_credential)
}
class ZhiPuModelProvider(IModelProvider):
def get_dialogue_number(self):
return 3
def get_model(self, model_type, model_name, model_credential: Dict[str, object], **model_kwargs) -> ChatZhipuAI:
zhipuai_chat = ChatZhipuAI(
temperature=0.5,
api_key=model_credential.get('api_key'),
model=model_name
)
return zhipuai_chat
def get_model_credential(self, model_type, model_name):
if model_name in model_dict:
return model_dict.get(model_name).model_credential
return qwen_model_credential
def get_model_provide_info(self):
return ModelProvideInfo(provider='model_zhipu_provider', name='智谱AI', icon=get_file_content(
os.path.join(PROJECT_DIR, "apps", "setting", 'models_provider', 'impl', 'zhipu_model_provider', 'icon',
'zhipuai_icon_svg')))
def get_model_list(self, model_type: str):
if model_type is None:
raise AppApiException(500, '模型类型不能为空')
return [model_dict.get(key).to_dict() for key in
list(filter(lambda key: model_dict.get(key).model_type == model_type, model_dict.keys()))]
def get_model_type_list(self):
return [{'key': "大语言模型", 'value': "LLM"}]

View File

@ -418,7 +418,8 @@ class UserProfile(ApiMixin):
permission_list = get_user_dynamics_permission(str(user.id))
permission_list += [p.value for p in get_permission_list_by_role(RoleConstants[user.role])]
return {'id': user.id, 'username': user.username, 'email': user.email, 'role': user.role,
'permissions': [str(p) for p in permission_list]}
'permissions': [str(p) for p in permission_list],
'is_edit_password': user.password == 'd880e722c47a34d8e9fce789fc62389d' if user.role == 'ADMIN' else False}
@staticmethod
def get_response_body_api():

View File

@ -33,6 +33,10 @@ pymupdf = "1.24.1"
python-docx = "^1.1.0"
xlwt = "^1.3.0"
dashscope = "^1.17.0"
zhipuai = "^2.0.1"
httpx = "^0.27.0"
httpx-sse = "^0.4.0"
websocket-client = "^1.7.0"
[build-system]
requires = ["poetry-core"]

View File

@ -67,10 +67,24 @@ const getStatistics: (
return get(`${prefix}/${application_id}/statistics/chat_record_aggregate_trend`, data, loading)
}
/**
* icon
* @param application_id
* data: file
*/
const putAppIcon: (
application_id: string,
data: any,
loading?: Ref<boolean>
) => Promise<Result<any>> = (application_id, data, loading) => {
return put(`${prefix}/${application_id}/edit_icon`, data, undefined, loading)
}
export default {
getAPIKey,
postAPIKey,
delAPIKey,
putAPIKey,
getStatistics
getStatistics,
putAppIcon
}

15
ui/src/api/image.ts Normal file
View File

@ -0,0 +1,15 @@
import { Result } from '@/request/Result'
import { get, post, del, put } from '@/request/index'
const prefix = '/image'
/**
*
* @param file:file
*/
const postImage: (data: any) => Promise<Result<any>> = (data) => {
return post(`${prefix}`, data)
}
export default {
postImage
}

View File

@ -10,6 +10,7 @@ interface ApplicationFormType {
dataset_setting?: any
model_setting?: any
problem_optimization?: boolean
icon?: string | undefined
}
interface chatType {
id: string

View File

@ -19,6 +19,10 @@ interface User {
*
*/
permissions: Array<string>
/**
*
*/
is_edit_password?: boolean
}
interface LoginRequest {

View File

@ -364,15 +364,15 @@ const getWrite = (chat: any, reader: any, stream: boolean) => {
let str = decoder.decode(value, { stream: true })
// start chunk chunkdata:{xxx}\n\n data:{ -> xxx}\n\n fetchchunkdata: \n\n
tempResult += str
if (tempResult.endsWith('\n\n')) {
str = tempResult
tempResult = ''
const split = tempResult.match(/data:.*}\n\n/g)
if (split) {
str = split.join('')
tempResult = tempResult.replace(str, '')
} else {
return reader.read().then(write_stream)
}
// end
if (str && str.startsWith('data:')) {
const split = str.match(/data:.*}\n\n/g)
if (split) {
for (const index in split) {
const chunk = JSON?.parse(split[index].replace('data:', ''))

View File

@ -21,6 +21,8 @@
autofocus
:maxlength="maxlength || '-'"
:show-word-limit="maxlength ? true : false"
@blur="isEdit = false"
@keyup.enter="submit"
></el-input>
</div>
@ -64,6 +66,10 @@ const loading = ref(false)
watch(isEdit, (bool) => {
if (!bool) {
writeValue.value = ''
} else {
nextTick(() => {
inputRef.value?.focus()
})
}
})
@ -80,10 +86,6 @@ function editNameHandle() {
isEdit.value = true
}
onMounted(() => {
nextTick(() => {
inputRef.value?.focus()
})
})
onMounted(() => {})
</script>
<style lang="scss" scoped></style>

View File

@ -18,6 +18,7 @@
class="mr-8"
:size="24"
/>
<AppAvatar
v-else-if="isDataset && currentType === '1'"
class="mr-8 avatar-purple"

View File

@ -30,16 +30,19 @@
</el-dropdown>
<ResetPassword ref="resetPasswordRef"></ResetPassword>
<AboutDialog ref="AboutDialogRef"></AboutDialog>
<UserPwdDialog ref="UserPwdDialogRef" />
</template>
<script setup lang="ts">
import { ref } from 'vue'
import { ref, onMounted } from 'vue'
import useStore from '@/stores'
import { useRouter } from 'vue-router'
import ResetPassword from './ResetPassword.vue'
import AboutDialog from './AboutDialog.vue'
import UserPwdDialog from '@/views/user-manage/component/UserPwdDialog.vue'
const { user } = useStore()
const router = useRouter()
const UserPwdDialogRef = ref()
const AboutDialogRef = ref()
const resetPasswordRef = ref<InstanceType<typeof ResetPassword>>()
@ -56,6 +59,12 @@ const logout = () => {
router.push({ name: 'login' })
})
}
onMounted(() => {
if (user.userInfo?.is_edit_password) {
UserPwdDialogRef.value.open(user.userInfo)
}
})
</script>
<style lang="scss" scoped>
.avatar-dropdown {

View File

@ -75,6 +75,19 @@ const useApplicationStore = defineStore({
},
async refreshAccessToken(token: string) {
this.asyncAppAuthentication(token)
},
// 修改应用
async asyncPutApplication(id: string, data: any, loading?: Ref<boolean>) {
return new Promise((resolve, reject) => {
applicationApi
.putApplication(id, data, loading)
.then((data) => {
resolve(data)
})
.catch((error) => {
reject(error)
})
})
}
}
})

View File

@ -306,6 +306,10 @@
.el-popover {
--el-popover-padding: 16px;
}
.el-radio {
--el-radio-font-weight: 400;
}
.el-radio__input.is-checked + .el-radio__label {
color: var(--app-text-color);
}

View File

@ -0,0 +1,6 @@
export const defaultIcon = '/ui/favicon.ico'
// 是否显示字母 / icon
export function isAppIcon(url: string | undefined) {
return url === defaultIcon ? '' : url
}

View File

@ -0,0 +1,141 @@
<template>
<el-dialog title="设置 Logo" v-model="dialogVisible">
<el-radio-group v-model="radioType" class="card__block mb-16">
<div>
<el-radio value="default">
<p>默认 Logo</p>
<AppAvatar
v-if="detail?.name"
:name="detail?.name"
pinyinColor
class="mt-8 mb-8"
shape="square"
:size="32"
/>
</el-radio>
</div>
<div>
<el-radio value="custom">
<p>自定义上传</p>
<div class="flex mt-8">
<AppAvatar
v-if="fileURL"
shape="square"
:size="32"
style="background: none"
class="mr-16"
>
<img :src="fileURL" alt="" />
</AppAvatar>
<el-upload
ref="uploadRef"
action="#"
:auto-upload="false"
:show-file-list="false"
:limit="1"
accept="image/*"
:on-change="onChange"
>
<el-button icon="Upload">上传</el-button>
</el-upload>
</div>
<div class="el-upload__tip info mt-16">
建议尺寸 32*32支持 icopng , 大小不超过200KB
</div>
</el-radio>
</div>
</el-radio-group>
<template #footer>
<span class="dialog-footer">
<el-button @click.prevent="dialogVisible = false"> 取消 </el-button>
<el-button type="primary" @click="submit" :loading="loading"> 保存 </el-button>
</span>
</template>
</el-dialog>
</template>
<script setup lang="ts">
import { ref, watch } from 'vue'
import { useRoute } from 'vue-router'
import overviewApi from '@/api/application-overview'
import { cloneDeep } from 'lodash'
import { MsgSuccess, MsgError } from '@/utils/message'
import { defaultIcon, isAppIcon } from '@/utils/application'
import useStore from '@/stores'
const { application } = useStore()
const route = useRoute()
const {
params: { id } //id
} = route
const emit = defineEmits(['refresh'])
const iconFile = ref<any>(null)
const fileURL = ref<any>(null)
const dialogVisible = ref<boolean>(false)
const loading = ref(false)
const detail = ref<any>(null)
const radioType = ref('default')
watch(dialogVisible, (bool) => {
if (!bool) {
iconFile.value = null
fileURL.value = null
}
})
const open = (data: any) => {
radioType.value = isAppIcon(data.icon) ? 'custom' : 'default'
fileURL.value = isAppIcon(data.icon) ? data.icon : null
detail.value = cloneDeep(data)
dialogVisible.value = true
}
const onChange = (file: any) => {
//1 200KB
const isLimit = file?.size / 1024 < 200
if (!isLimit) {
MsgError('文件大小超过 200KB')
return false
}
iconFile.value = file
fileURL.value = URL.createObjectURL(file.raw)
}
function submit() {
if (radioType.value === 'default') {
application.asyncPutApplication(id as string, { icon: defaultIcon }, loading).then((res) => {
emit('refresh')
MsgSuccess('设置成功')
dialogVisible.value = false
})
} else if (radioType.value === 'custom' && iconFile.value) {
let fd = new FormData()
fd.append('file', iconFile.value.raw)
overviewApi.putAppIcon(id as string, fd, loading).then((res: any) => {
emit('refresh')
MsgSuccess('设置成功')
dialogVisible.value = false
})
} else {
MsgError('请上传一张图片')
}
}
defineExpose({ open })
</script>
<style lang="scss" scope>
.card__block {
width: 100%;
display: block;
.el-radio {
align-items: flex-start;
height: 100%;
}
.el-radio__inner {
margin-top: 3px;
}
}
</style>

View File

@ -5,14 +5,37 @@
<h4 class="title-decoration-1 mb-16">应用信息</h4>
<el-card shadow="never" class="overview-card" v-loading="loading">
<div class="title flex align-center">
<AppAvatar
v-if="detail?.name"
:name="detail?.name"
pinyinColor
class="mr-12"
shape="square"
:size="32"
/>
<div
class="edit-avatar mr-12"
@mouseenter="showEditIcon = true"
@mouseleave="showEditIcon = false"
>
<AppAvatar
v-if="isAppIcon(detail?.icon)"
shape="square"
:size="32"
style="background: none"
>
<img :src="detail?.icon" alt="" />
</AppAvatar>
<AppAvatar
v-else-if="detail?.name"
:name="detail?.name"
pinyinColor
shape="square"
:size="32"
/>
<AppAvatar
v-if="showEditIcon"
shape="square"
class="edit-mask"
:size="32"
@click="openEditAvatar"
>
<el-icon><EditPen /></el-icon>
</AppAvatar>
</div>
<h4>{{ detail?.name }}</h4>
</div>
@ -102,6 +125,7 @@
<EmbedDialog ref="EmbedDialogRef" />
<APIKeyDialog ref="APIKeyDialogRef" />
<LimitDialog ref="LimitDialogRef" @refresh="refresh" />
<EditAvatarDialog ref="EditAvatarDialogRef" @refresh="refreshIcon" />
</LayoutContainer>
</template>
<script setup lang="ts">
@ -110,14 +134,14 @@ import { useRoute } from 'vue-router'
import EmbedDialog from './component/EmbedDialog.vue'
import APIKeyDialog from './component/APIKeyDialog.vue'
import LimitDialog from './component/LimitDialog.vue'
import EditAvatarDialog from './component/EditAvatarDialog.vue'
import StatisticsCharts from './component/StatisticsCharts.vue'
import applicationApi from '@/api/application'
import overviewApi from '@/api/application-overview'
import { nowDate, beforeDay } from '@/utils/time'
import { MsgSuccess, MsgConfirm } from '@/utils/message'
import { copyClick } from '@/utils/clipboard'
import { isAppIcon } from '@/utils/application'
import useStore from '@/stores'
const { application } = useStore()
const route = useRoute()
@ -127,6 +151,7 @@ const {
const apiUrl = window.location.origin + '/doc'
const EditAvatarDialogRef = ref()
const LimitDialogRef = ref()
const APIKeyDialogRef = ref()
const EmbedDialogRef = ref()
@ -175,6 +200,12 @@ const daterange = ref({
const statisticsLoading = ref(false)
const statisticsData = ref([])
const showEditIcon = ref(false)
function openEditAvatar() {
EditAvatarDialogRef.value.open(detail.value)
}
function changeDayHandle(val: number | string) {
if (val !== 'other') {
daterange.value.start_time = beforeDay(val)
@ -252,6 +283,11 @@ function getDetail() {
function refresh() {
getAccessToken()
}
function refreshIcon() {
getDetail()
}
onMounted(() => {
getDetail()
getAccessToken()
@ -266,5 +302,14 @@ onMounted(() => {
right: 16px;
top: 21px;
}
.edit-avatar {
position: relative;
.edit-mask {
position: absolute;
left: 0;
background: rgba(0, 0, 0, 0.4);
}
}
}
</style>

View File

@ -145,74 +145,12 @@
<div class="flex-between">
<span>关联知识库</span>
<div>
<el-popover :visible="popoverVisible" :width="214" trigger="click">
<template #reference>
<el-button type="primary" link @click="datasetSettingChange('open')">
<AppIcon iconName="app-operation" class="mr-4"></AppIcon>参数设置
</el-button>
</template>
<div class="dataset_setting">
<div class="form-item mb-16">
<div class="title flex align-center mb-8">
<span style="margin-right: 4px">相似度高于</span>
<el-tooltip
effect="dark"
content="相似度越高相关性越强。"
placement="right"
>
<AppIcon iconName="app-warning" class="app-warning-icon"></AppIcon>
</el-tooltip>
</div>
<div @click.stop>
<el-input-number
v-model="dataset_setting.similarity"
:min="0"
:max="1"
:precision="3"
:step="0.1"
controls-position="right"
style="width: 180px"
/>
</div>
</div>
<div class="form-item mb-16">
<div class="title mb-8">引用分段数 TOP</div>
<div @click.stop>
<el-input-number
v-model="dataset_setting.top_n"
:min="1"
:max="10"
controls-position="right"
style="width: 180px"
/>
</div>
</div>
<div class="form-item mb-16">
<div class="title mb-8">最多引用字符数</div>
<div class="flex align-center">
<el-slider
v-model="dataset_setting.max_paragraph_char_number"
show-input
:show-input-controls="false"
:min="500"
:max="10000"
style="width: 180px"
class="custom-slider"
/>
</div>
</div>
</div>
<div class="text-right">
<el-button @click="popoverVisible = false">取消</el-button>
<el-button type="primary" @click="datasetSettingChange('close')"
>确认</el-button
>
</div>
</el-popover>
<el-button type="primary" link @click="openDatasetDialog"
><el-icon class="mr-4"><Plus /></el-icon></el-button
>
<el-button type="primary" link @click="openParamSettingDialog">
<AppIcon iconName="app-operation" class="mr-4"></AppIcon>参数设置
</el-button>
<el-button type="primary" link @click="openDatasetDialog">
<el-icon class="mr-4"><Plus /></el-icon>
</el-button>
</div>
</div>
</template>
@ -221,13 +159,6 @@
>关联的知识库展示在这里</el-text
>
<el-row :gutter="12" v-else>
<!-- <el-col :xs="24" :sm="24" :md="12" :lg="12" :xl="12" class="mb-8">
<CardAdd
title="关联知识库"
@click="openDatasetDialog"
style="min-height: 50px; font-size: 14px"
/>
</el-col> -->
<el-col
:xs="24"
:sm="24"
@ -311,6 +242,7 @@
</el-col>
</el-row>
<ParamSettingDialog ref="ParamSettingDialogRef" @refresh="refreshParam" />
<AddDatasetDialog
ref="AddDatasetDialogRef"
@addData="addDataset"
@ -318,6 +250,8 @@
@refresh="refresh"
:loading="datasetLoading"
/>
<!-- 添加模版 -->
<CreateModelDialog
ref="createModelRef"
@submit="getModel"
@ -329,7 +263,8 @@
<script setup lang="ts">
import { reactive, ref, watch, onMounted } from 'vue'
import { useRouter, useRoute } from 'vue-router'
import { groupBy, cloneDeep } from 'lodash'
import { groupBy } from 'lodash'
import ParamSettingDialog from './components/ParamSettingDialog.vue'
import AddDatasetDialog from './components/AddDatasetDialog.vue'
import CreateModelDialog from '@/views/template/component/CreateModelDialog.vue'
import SelectProviderDialog from '@/views/template/component/SelectProviderDialog.vue'
@ -363,6 +298,8 @@ const defaultPrompt = `已知信息:
问题
{question}
`
const ParamSettingDialogRef = ref<InstanceType<typeof ParamSettingDialog>>()
const createModelRef = ref<InstanceType<typeof CreateModelDialog>>()
const selectProviderRef = ref<InstanceType<typeof SelectProviderDialog>>()
@ -384,7 +321,8 @@ const applicationForm = ref<ApplicationFormType>({
dataset_setting: {
top_n: 3,
similarity: 0.6,
max_paragraph_char_number: 5000
max_paragraph_char_number: 5000,
search_mode: 'embedding'
},
model_setting: {
prompt: defaultPrompt
@ -392,8 +330,6 @@ const applicationForm = ref<ApplicationFormType>({
problem_optimization: false
})
const popoverVisible = ref(false)
const rules = reactive<FormRules<ApplicationFormType>>({
name: [{ required: true, message: '请输入应用名称', trigger: 'blur' }],
model_id: [
@ -408,24 +344,13 @@ const rules = reactive<FormRules<ApplicationFormType>>({
const modelOptions = ref<any>(null)
const providerOptions = ref<Array<Provider>>([])
const datasetList = ref([])
const dataset_setting = ref<any>({})
function datasetSettingChange(val: string) {
if (val === 'open') {
popoverVisible.value = true
dataset_setting.value = cloneDeep(applicationForm.value.dataset_setting)
} else if (val === 'close') {
popoverVisible.value = false
applicationForm.value.dataset_setting = cloneDeep(dataset_setting.value)
}
}
const submit = async (formEl: FormInstance | undefined) => {
if (!formEl) return
await formEl.validate((valid, fields) => {
if (valid) {
if (id) {
applicationApi.putApplication(id, applicationForm.value, loading).then((res) => {
application.asyncPutApplication(id, applicationForm.value, loading).then((res) => {
MsgSuccess('保存成功')
})
} else {
@ -438,6 +363,14 @@ const submit = async (formEl: FormInstance | undefined) => {
})
}
const openParamSettingDialog = () => {
ParamSettingDialogRef.value?.open(applicationForm.value.dataset_setting)
}
function refreshParam(data: any) {
applicationForm.value.dataset_setting = data
}
const openCreateModel = (provider?: Provider) => {
if (provider && provider.provider) {
createModelRef.value?.open(provider)
@ -560,13 +493,4 @@ onMounted(() => {
.prologue-md-editor {
height: 150px;
}
.dataset_setting {
color: var(--el-text-color-regular);
font-weight: 400;
}
.custom-slider {
:deep(.el-input-number.is-without-controls .el-input__wrapper) {
padding: 0 !important;
}
}
</style>

View File

@ -0,0 +1,152 @@
<template>
<el-dialog title="参数设置" class="param-dialog" v-model="dialogVisible" style="width: 550px">
<div class="dialog-max-height">
<el-scrollbar>
<div class="p-16">
<el-form label-position="top" ref="paramFormRef" :model="form">
<el-form-item label="检索模式">
<el-radio-group v-model="form.search_mode" class="card__radio">
<el-card
shadow="never"
class="mb-16"
:class="form.search_mode === 'embedding' ? 'active' : ''"
>
<el-radio value="embedding" size="large">
<p class="mb-4">向量检索</p>
<el-text type="info">通过向量距离计算与用户问题最相似的文本分段</el-text>
</el-radio>
</el-card>
<el-card
shadow="never"
class="mb-16"
:class="form.search_mode === 'keywords' ? 'active' : ''"
>
<el-radio value="keywords" size="large">
<p class="mb-4">全文检索</p>
<el-text type="info">通过关键词检索返回包含关键词最多的文本分段</el-text>
</el-radio>
</el-card>
<el-card
shadow="never"
class="mb-16"
:class="form.search_mode === 'blend' ? 'active' : ''"
>
<el-radio value="blend" size="large">
<p class="mb-4">混合检索</p>
<el-text type="info"
>同时执行全文检索和向量检索再进行重排序从两类查询结果中选择匹配用户问题的最佳结果</el-text
>
</el-radio>
</el-card>
</el-radio-group>
</el-form-item>
<el-form-item>
<template #label>
<div class="flex align-center">
<span class="mr-4">相似度高于</span>
<el-tooltip effect="dark" content="相似度越高相关性越强。" placement="right">
<AppIcon iconName="app-warning" class="app-warning-icon"></AppIcon>
</el-tooltip>
</div>
</template>
<el-input-number
v-model="form.similarity"
:min="0"
:max="1"
:precision="3"
:step="0.1"
controls-position="right"
/>
</el-form-item>
<el-form-item label="引用分段数 TOP">
<el-input-number v-model="form.top_n" :min="1" :max="10" controls-position="right" />
</el-form-item>
<el-form-item label="最多引用字符数">
<el-slider
v-model="form.max_paragraph_char_number"
show-input
:show-input-controls="false"
:min="500"
:max="10000"
class="custom-slider"
/>
</el-form-item>
</el-form>
</div>
</el-scrollbar>
</div>
<template #footer>
<span class="dialog-footer">
<el-button @click.prevent="dialogVisible = false"> 取消 </el-button>
<el-button type="primary" @click="submit(paramFormRef)" :loading="loading">
保存
</el-button>
</span>
</template>
</el-dialog>
</template>
<script setup lang="ts">
import { ref, watch } from 'vue'
import type { FormInstance, FormRules } from 'element-plus'
const emit = defineEmits(['refresh'])
const paramFormRef = ref()
const form = ref<any>({
search_mode: 'embedding',
top_n: 3,
similarity: 0.6,
max_paragraph_char_number: 5000
})
const dialogVisible = ref<boolean>(false)
const loading = ref(false)
watch(dialogVisible, (bool) => {
if (!bool) {
form.value = {
search_mode: 'embedding',
top_n: 3,
similarity: 0.6,
max_paragraph_char_number: 5000
}
}
})
const open = (data: any) => {
form.value = { ...form.value, ...data }
dialogVisible.value = true
}
const submit = async (formEl: FormInstance | undefined) => {
if (!formEl) return
await formEl.validate((valid, fields) => {
if (valid) {
emit('refresh', form.value)
dialogVisible.value = false
}
})
}
defineExpose({ open })
</script>
<style lang="scss" scope>
.param-dialog {
padding: 8px;
.el-dialog__header {
padding: 16px 16px 0 16px;
}
.el-dialog__body {
padding: 0 !important;
}
.dialog-max-height {
height: calc(100vh - 260px);
}
.custom-slider {
.el-input-number.is-without-controls .el-input__wrapper {
padding: 0 !important;
}
}
}
</style>

View File

@ -41,12 +41,21 @@
>
<template #icon>
<AppAvatar
v-if="item.name"
:name="item.name"
pinyinColor
class="mr-12"
v-if="isAppIcon(item?.icon)"
shape="square"
:size="32"
style="background: none"
class="mr-8"
>
<img :src="item?.icon" alt="" />
</AppAvatar>
<AppAvatar
v-else-if="item?.name"
:name="item?.name"
pinyinColor
shape="square"
:size="32"
class="mr-8"
/>
</template>
@ -85,6 +94,7 @@
import { ref, onMounted, reactive, computed } from 'vue'
import applicationApi from '@/api/application'
import { MsgSuccess, MsgConfirm } from '@/utils/message'
import { isAppIcon } from '@/utils/application'
import { useRouter } from 'vue-router'
import useStore from '@/stores'
const { application } = useStore()

View File

@ -59,12 +59,21 @@
<CardCheckbox value-field="id" :data="item" v-model="application_id_list">
<template #icon>
<AppAvatar
v-if="item.name"
:name="item.name"
pinyinColor
class="mr-12"
v-if="isAppIcon(item?.icon)"
shape="square"
:size="32"
style="background: none"
class="mr-12"
>
<img :src="item?.icon" alt="" />
</AppAvatar>
<AppAvatar
v-else-if="item?.name"
:name="item?.name"
pinyinColor
shape="square"
:size="32"
class="mr-12"
/>
</template>
{{ item.name }}
@ -87,6 +96,7 @@ import BaseForm from '@/views/dataset/component/BaseForm.vue'
import datasetApi from '@/api/dataset'
import type { ApplicationFormType } from '@/api/type/application'
import { MsgSuccess } from '@/utils/message'
import { isAppIcon } from '@/utils/application'
import useStore from '@/stores'
const route = useRoute()
const {

View File

@ -54,7 +54,7 @@
<el-table-column prop="name" label="文件名称" min-width="280">
<template #default="{ row }">
<ReadWrite
@change="editName"
@change="editName($event, row.id)"
:data="row.name"
:showEditIcon="row.id === currentMouseId"
/>
@ -354,12 +354,12 @@ function changeState(bool: Boolean, row: any) {
currentMouseId.value && updateData(row.id, obj, str)
}
function editName(val: string) {
function editName(val: string, id: string) {
if (val) {
const obj = {
name: val
}
currentMouseId.value && updateData(currentMouseId.value, obj, '修改成功')
updateData(id, obj, '修改成功')
} else {
MsgError('文件名称不能为空!')
}

View File

@ -78,11 +78,47 @@
<ParagraphDialog ref="ParagraphDialogRef" :title="title" @refresh="refresh" />
</LayoutContainer>
<div class="hit-test__operate p-24 pt-0">
<el-popover :visible="popoverVisible" placement="right-end" :width="180" trigger="click">
<el-popover :visible="popoverVisible" placement="right-end" :width="500" trigger="click">
<template #reference>
<el-button icon="Setting" class="mb-8" @click="settingChange('open')">参数设置</el-button>
</template>
<div class="mb-16">
<div class="title mb-8">检索模式</div>
<el-radio-group v-model="cloneForm.search_mode" class="card__radio">
<el-card
shadow="never"
class="mb-16"
:class="cloneForm.search_mode === 'embedding' ? 'active' : ''"
>
<el-radio value="embedding" size="large">
<p class="mb-4">向量检索</p>
<el-text type="info">通过向量距离计算与用户问题最相似的文本分段</el-text>
</el-radio>
</el-card>
<el-card
shadow="never"
class="mb-16"
:class="cloneForm.search_mode === 'keywords' ? 'active' : ''"
>
<el-radio value="keywords" size="large">
<p class="mb-4">全文检索</p>
<el-text type="info">通过关键词检索返回包含关键词最多的文本分段</el-text>
</el-radio>
</el-card>
<el-card
shadow="never"
class="mb-16"
:class="cloneForm.search_mode === 'blend' ? 'active' : ''"
>
<el-radio value="blend" size="large">
<p class="mb-4">混合检索</p>
<el-text type="info"
>同时执行全文检索和向量检索再进行重排序从两类查询结果中选择匹配用户问题的最佳结果</el-text
>
</el-radio>
</el-card>
</el-radio-group>
</div>
<div class="mb-16">
<div class="title mb-8">相似度高于</div>
<el-input-number
@ -92,7 +128,6 @@
:precision="3"
:step="0.1"
controls-position="right"
style="width: 145px"
/>
</div>
<div class="mb-16">
@ -103,7 +138,6 @@
:min="1"
:max="10"
controls-position="right"
style="width: 145px"
/>
</div>
<div class="text-right">
@ -161,7 +195,8 @@ const title = ref('')
const inputValue = ref('')
const formInline = ref({
similarity: 0.6,
top_number: 5
top_number: 5,
search_mode: 'embedding'
})
//
@ -213,8 +248,7 @@ function sendChatHandle(event: any) {
function getHitTestList() {
const obj = {
query_text: inputValue.value,
similarity: formInline.value.similarity,
top_number: formInline.value.top_number
...formInline.value
}
if (isDataset.value) {
datasetApi.getDatasetHitTest(id, obj, loading).then((res) => {

View File

@ -26,17 +26,18 @@
<el-input v-model="form.title" placeholder="请给当前内容设置一个标题,以便管理查看">
</el-input>
</el-form-item>
<el-form-item label="保存至文档" prop="document">
<el-cascader
v-model="form.document"
:props="LoadDocument"
placeholder="请选择文档"
class="w-full"
<el-form-item label="选择知识库" prop="dataset_id">
<el-select
v-model="form.dataset_id"
filterable
placeholder="请选择知识库"
:loading="optionLoading"
@change="changeDataset"
>
<template #default="{ node, data }">
<el-option v-for="item in datasetList" :key="item.id" :label="item.name" :value="item.id">
<span class="flex align-center">
<AppAvatar
v-if="!data.dataset_id && data.type === '1'"
v-if="!item.dataset_id && item.type === '1'"
class="mr-12 avatar-purple"
shape="square"
:size="24"
@ -44,17 +45,34 @@
<img src="@/assets/icon_web.svg" style="width: 58%" alt="" />
</AppAvatar>
<AppAvatar
v-else-if="!data.dataset_id && data.type === '0'"
v-else-if="!item.dataset_id && item.type === '0'"
class="mr-12"
shape="square"
:size="24"
>
<img src="@/assets/icon_document.svg" style="width: 58%" alt="" />
</AppAvatar>
<span class="ellipsis"> {{ data.name }}</span>
{{ item.name }}
</span>
</template>
</el-cascader>
</el-option>
</el-select>
</el-form-item>
<el-form-item label="保存至文档" prop="document_id">
<el-select
v-model="form.document_id"
filterable
placeholder="请选择文档"
:loading="optionLoading"
>
<el-option
v-for="item in documentList"
:key="item.id"
:label="item.name"
:value="item.id"
>
{{ item.name }}
</el-option>
</el-select>
</el-form-item>
</el-form>
<template #footer>
@ -70,7 +88,6 @@ import { ref, watch, reactive } from 'vue'
import { useRoute } from 'vue-router'
import type { FormInstance, FormRules } from 'element-plus'
import logApi from '@/api/log'
import type { CascaderProps } from 'element-plus'
import useStore from '@/stores'
const { application, document } = useStore()
@ -99,15 +116,19 @@ const form = ref<any>({
problem_text: '',
title: '',
content: '',
document: []
dataset_id: '',
document_id: ''
})
const rules = reactive<FormRules>({
content: [{ required: true, message: '请输入内容', trigger: 'blur' }],
document: [{ type: 'array', required: true, message: '请选择文档', trigger: 'change' }]
dataset_id: [{ required: true, message: '请选择知识库', trigger: 'change' }],
document_id: [{ required: true, message: '请选择文档', trigger: 'change' }]
})
const datasetList = ref([])
const datasetList = ref<any[]>([])
const documentList = ref<any[]>([])
const optionLoading = ref(false)
watch(dialogVisible, (bool) => {
if (!bool) {
@ -117,45 +138,40 @@ watch(dialogVisible, (bool) => {
problem_text: '',
title: '',
content: '',
document: []
dataset_id: '',
document_id: ''
}
datasetList.value = []
documentList.value = []
formRef.value?.clearValidate()
}
})
const LoadDocument: CascaderProps = {
lazy: true,
value: 'id',
label: 'name',
leaf: 'dataset_id',
lazyLoad(node, resolve: any) {
const { level, data } = node
if (data?.id) {
getDocument(data?.id as string, resolve)
} else {
getDataset(resolve)
}
}
function changeDataset(id: string) {
form.value.document_id = ''
getDocument(id)
}
function getDocument(id: string, resolve: any) {
function getDocument(id: string) {
document.asyncGetAllDocument(id, loading).then((res: any) => {
datasetList.value = res.data
resolve(datasetList.value)
documentList.value = res.data
})
}
function getDataset(resolve: any) {
function getDataset() {
application.asyncGetApplicationDataset(id, loading).then((res: any) => {
datasetList.value = res.data
resolve(datasetList.value)
})
}
const open = (data: any) => {
getDataset()
form.value.chat_id = data.chat_id
form.value.record_id = data.id
form.value.problem_text = data.problem_text
form.value.content = data.answer_text
formRef.value?.clearValidate()
dialogVisible.value = true
}
const submitForm = async (formEl: FormInstance | undefined) => {
@ -171,8 +187,8 @@ const submitForm = async (formEl: FormInstance | undefined) => {
id,
form.value.chat_id,
form.value.record_id,
form.value.document[0],
form.value.document[1],
form.value.dataset_id,
form.value.document_id,
obj,
loading
)

View File

@ -30,6 +30,7 @@
:preview="false"
:toolbars="toolbars"
style="height: 300px"
@onUploadImg="onUploadImg"
/>
<MdPreview
v-else
@ -46,6 +47,7 @@
import { ref, reactive, onUnmounted, watch, nextTick } from 'vue'
import type { FormInstance, FormRules } from 'element-plus'
import { MdEditor, MdPreview } from 'md-editor-v3'
import imageApi from '@/api/image'
const props = defineProps({
data: {
type: Object,
@ -81,7 +83,7 @@ const toolbars = [
'=',
'pageFullscreen',
'preview',
'htmlPreview',
'htmlPreview'
] as any[]
const editorRef = ref()
@ -134,20 +136,25 @@ function validate() {
})
}
// const onHtmlChanged = () => {
// appendTarget()
// }
// const appendTarget = () => {
// nextTick(() => {
// var item = document.getElementsByClassName('maxkb-md')
// for (var j = 0; j < item.length; j++) {
// var aTags = item[j].getElementsByTagName('a')
// for (var i = 0; i < aTags.length; i++) {
// aTags[i].setAttribute('target', '_blank')
// }
// }
// })
// }
const onUploadImg = async (files: any, callback: any) => {
const res = await Promise.all(
files.map((file: any) => {
return new Promise((rev, rej) => {
const fd = new FormData()
fd.append('file', file)
imageApi
.postImage(fd)
.then((res: any) => {
rev(res)
})
.catch((error) => rej(error))
})
})
)
callback(res.map((item) => item.data))
}
onUnmounted(() => {
form.value = {

View File

@ -42,7 +42,7 @@
<el-table-column prop="content" label="问题" min-width="280">
<template #default="{ row }">
<ReadWrite
@change="editName"
@change="editName($event, row.id)"
:data="row.content"
:showEditIcon="row.id === currentMouseId"
:maxlength="256"
@ -121,7 +121,7 @@ import useStore from '@/stores'
const route = useRoute()
const {
params: { id }
params: { id } // id
} = route as any
const { problem } = useStore()
@ -216,12 +216,12 @@ function deleteProblem(row: any) {
.catch(() => {})
}
function editName(val: string) {
function editName(val: string, problemId: string) {
if (val) {
const obj = {
content: val
}
problemApi.putProblems(id, currentMouseId.value, obj, loading).then(() => {
problemApi.putProblems(id, problemId, obj, loading).then(() => {
getList()
MsgSuccess('修改成功')
})

View File

@ -30,9 +30,23 @@
label-position="top"
require-asterisk-position="right"
class="mb-24"
label-width="auto"
>
<template #default>
<el-form-item label="模型名称" prop="name" :rules="base_form_data_rule.name">
<el-form-item prop="name" :rules="base_form_data_rule.name">
<template #label>
<div class="flex align-center" style="display: inline-flex">
<div class="flex-between mr-4">
<span>模型名称 </span>
</div>
<el-tooltip effect="dark" placement="right">
<template #content>
<p>MaxKB 中自定义的模型名称</p>
</template>
<AppIcon iconName="app-warning" class="app-warning-icon"></AppIcon>
</el-tooltip>
</div>
</template>
<el-input
v-model="base_form_data.name"
maxlength="20"
@ -40,7 +54,10 @@
placeholder="请给基础模型设置一个名称"
/>
</el-form-item>
<el-form-item label="模型类型" prop="model_type" :rules="base_form_data_rule.model_type">
<el-form-item prop="model_type" :rules="base_form_data_rule.model_type">
<template #label>
<span>模型类型</span>
</template>
<el-select
v-loading="model_type_loading"
@change="list_base_model($event)"
@ -53,16 +70,33 @@
:key="item.value"
:label="item.key"
:value="item.value"
></el-option
></el-select>
></el-option>
</el-select>
</el-form-item>
<el-form-item label="基础模型" prop="model_name" :rules="base_form_data_rule.model_name">
<el-form-item prop="model_name" :rules="base_form_data_rule.model_name">
<template #label>
<div class="flex align-center" style="display: inline-flex">
<div class="flex-between mr-4">
<span>基础模型 </span>
</div>
<el-tooltip effect="dark" placement="right">
<template #content>
<p>为供应商的 LLM 模型支持自定义输入</p>
<p>
下拉选项是 OpenAI
常用的一些大语言模型如gpt-3.5-turbo-0613gpt-3.5-turbogpt-4
</p>
</template>
<AppIcon iconName="app-warning" class="app-warning-icon"></AppIcon>
</el-tooltip>
</div>
</template>
<el-select
@change="getModelForm($event)"
v-loading="base_model_loading"
v-model="base_form_data.model_name"
class="w-full m-2"
placeholder="请选择基础模型"
placeholder="自定义输入基础模型后回车即可"
filterable
allow-create
default-first-option
@ -72,8 +106,8 @@
:key="item.name"
:label="item.name"
:value="item.name"
></el-option
></el-select>
></el-option>
</el-select>
</el-form-item>
</template>
</DynamicsForm>
@ -94,6 +128,8 @@ import type { FormField } from '@/components/dynamics-form/type'
import DynamicsForm from '@/components/dynamics-form/index.vue'
import type { FormRules } from 'element-plus'
import { MsgSuccess } from '@/utils/message'
import { QuestionFilled } from '@element-plus/icons-vue'
const providerValue = ref<Provider>()
const dynamicsFormRef = ref<InstanceType<typeof DynamicsForm>>()
const emit = defineEmits(['change', 'submit'])
@ -204,10 +240,12 @@ defineExpose({ open, close })
font-weight: 400;
line-height: 24px;
cursor: pointer;
&:hover {
color: var(--el-color-primary);
}
}
.active-breadcrumb {
font-size: 16px;
color: rgba(31, 35, 41, 1);

View File

@ -27,7 +27,20 @@
require-asterisk-position="right"
>
<template #default>
<el-form-item label="模型名称" prop="name" :rules="base_form_data_rule.name">
<el-form-item prop="name" :rules="base_form_data_rule.name">
<template #label>
<div class="flex align-center" style="display: inline-flex">
<div class="flex-between mr-4">
<span>模型名称 </span>
</div>
<el-tooltip effect="dark" placement="right">
<template #content>
<p>MaxKB 中自定义的模型名称</p>
</template>
<AppIcon iconName="app-warning" class="app-warning-icon"></AppIcon>
</el-tooltip>
</div>
</template>
<el-input
v-model="base_form_data.name"
maxlength="20"
@ -35,7 +48,10 @@
placeholder="请给基础模型设置一个名称"
/>
</el-form-item>
<el-form-item label="模型类型" prop="model_type" :rules="base_form_data_rule.model_type">
<el-form-item prop="model_type" :rules="base_form_data_rule.model_type">
<template #label>
<span>模型类型</span>
</template>
<el-select
v-loading="model_type_loading"
@change="list_base_model($event)"
@ -48,10 +64,27 @@
:key="item.value"
:label="item.key"
:value="item.value"
></el-option
></el-select>
></el-option>
</el-select>
</el-form-item>
<el-form-item label="基础模型" prop="model_name" :rules="base_form_data_rule.model_name">
<el-form-item prop="model_name" :rules="base_form_data_rule.model_name">
<template #label>
<div class="flex align-center" style="display: inline-flex">
<div class="flex-between mr-4">
<span>基础模型 </span>
</div>
<el-tooltip effect="dark" placement="right">
<template #content>
<p>为供应商的 LLM 模型支持自定义输入</p>
<p>
下拉选项是 OpenAI
常用的一些大语言模型如gpt-3.5-turbo-0613gpt-3.5-turbogpt-4
</p>
</template>
<AppIcon iconName="app-warning" class="app-warning-icon"></AppIcon>
</el-tooltip>
</div>
</template>
<el-select
@change="getModelForm($event)"
v-loading="base_model_loading"
@ -67,8 +100,8 @@
:key="item.name"
:label="item.name"
:value="item.name"
></el-option
></el-select>
></el-option>
</el-select>
</el-form-item>
</template>
</DynamicsForm>
@ -89,6 +122,8 @@ import type { FormField } from '@/components/dynamics-form/type'
import DynamicsForm from '@/components/dynamics-form/index.vue'
import type { FormRules } from 'element-plus'
import { MsgSuccess } from '@/utils/message'
import { QuestionFilled } from '@element-plus/icons-vue'
import AppIcon from '@/components/icons/AppIcon.vue'
const providerValue = ref<Provider>()
const dynamicsFormRef = ref<InstanceType<typeof DynamicsForm>>()
@ -153,7 +188,7 @@ const list_base_model = (model_type: any) => {
}
}
const open = (provider: Provider, model: Model) => {
modelValue.value=model
modelValue.value = model
ModelApi.getModelById(model.id, formLoading).then((ok) => {
modelValue.value = ok.data
ModelApi.listModelType(model.provider, model_type_loading).then((ok) => {
@ -208,10 +243,12 @@ defineExpose({ open, close })
font-weight: 400;
line-height: 24px;
cursor: pointer;
&:hover {
color: var(--el-color-primary);
}
}
.active-breadcrumb {
font-size: 16px;
color: rgba(31, 35, 41, 1);