mirror of
https://github.com/Tencent/Metis.git
synced 2025-12-25 19:52:49 +00:00
feat(local): commit code
This commit is contained in:
dongdonglin(林向东)
commit
ee5235c6fe
|
|
@ -0,0 +1,22 @@
|
|||
language: python
|
||||
|
||||
python:
|
||||
- "2.7"
|
||||
|
||||
node_js: "8"
|
||||
|
||||
|
||||
install:
|
||||
- pip install django==1.11.13
|
||||
- pip install pylint
|
||||
- pip install mysql-python
|
||||
- pip install scikit-learn
|
||||
- npm install
|
||||
|
||||
services:
|
||||
- mysql
|
||||
|
||||
script:
|
||||
- export PATH=./node_modules/.bin:$PATH
|
||||
- commitlint-travis
|
||||
- make test
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
# Contributing to Metis
|
||||
我们欢迎[report Issues](https://github.com/Tencent/Metis/issues) 或者 [pull requests](https://github.com/Tencent/Metis/pulls)。 在贡献代码之前请阅读以下指引。
|
||||
|
||||
## 问题管理
|
||||
我们用 Github Issues 去跟踪 public bugs 和 feature requests。
|
||||
|
||||
### 查找已知的issue 优先
|
||||
请查找已存在或者相类似的issue，从而保证不存在冗余。
|
||||
|
||||
### 新建 Issues
|
||||
新建issues 时请提供详细的描述、截屏或者短视频来辅助我们定位问题
|
||||
|
||||
### 分支管理
|
||||
|
||||
有两个主分支：
|
||||
|
||||
1. `master` 分支
|
||||
1. **注意不要提交PR到此分支**
|
||||
2. `dev` 分支.
|
||||
1. **这是稳定的开发分支，经过完成测试后，`dev`分支的内容会在下次发布时合并到 `master`分支。**
|
||||
2. **建议提交PR到`dev`分支。**
|
||||
|
||||
### Pull Requests
|
||||
|
||||
我们欢迎大家贡献代码来使我们的Metis更加强大
|
||||
代码团队会监控pull request, 我们会做相应的代码检查和测试，测试通过之后我们就会接纳PR ，但是不会立即合并到master分支。
|
||||
|
||||
在完成一个pr之前请做一下确认:
|
||||
|
||||
1. 从 `master` fork 你自己的分支。
|
||||
2. 在修改了代码之后请修改对应的文档和注释。
|
||||
3. 在新建的文件中请加入licence 和copy right申明。
|
||||
4. 确保一致的代码风格。
|
||||
5. 做充分的测试。
|
||||
6. 然后，你可以提交你的代码到 `dev` 分支。
|
||||
|
||||
## 代码协议
|
||||
[BSD 3-Clause License](https://github.com/Tencent/Metis/master/LICENSE.TXT) 为Metis的开源协议，您贡献的代码也会受此协议保护。
|
||||
|
|
@ -0,0 +1,418 @@
|
|||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
If you have downloaded a copy of the Metis binary from Tencent, please note that the Metis binary is licensed under the BSD 3-Clause License.
|
||||
If you have downloaded a copy of the Metis source code from Tencent, please note that Metis source code is licensed under the BSD 3-Clause License, except for the third-party components listed below which are subject to different license terms. Your integration of Metis into your own projects may require compliance with the BSD 3-Clause License, as well as the other licenses applicable to the third-party components included within Metis.
|
||||
A copy of the BSD 3-Clause License is included in this file.
|
||||
|
||||
Other dependencies and licenses:
|
||||
|
||||
Open Source Software Licensed Under the Apache License, Version 2.0:
|
||||
----------------------------------------------------------------------------------------
|
||||
1. xgboost v0.71
|
||||
Copyright (c) 2016 by Contributors
|
||||
|
||||
|
||||
Terms of the Apache License, Version 2.0:
|
||||
--------------------------------------------------------------------
|
||||
Apache License
|
||||
|
||||
Version 2.0, January 2004
|
||||
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
“License” shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
“Licensor” shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
|
||||
|
||||
“Legal Entity” shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, “control” means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
“You” (or “Your”) shall mean an individual or Legal Entity exercising permissions granted by this License.
|
||||
|
||||
“Source” form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
|
||||
|
||||
“Object” form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
|
||||
|
||||
“Work” shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).
|
||||
|
||||
“Derivative Works” shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
|
||||
|
||||
“Contribution” shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, “submitted” means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as “Not a Contribution.”
|
||||
|
||||
“Contributor” shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
|
||||
|
||||
a) You must give any other recipients of the Work or Derivative Works a copy of this License; and
|
||||
|
||||
b) You must cause any modified files to carry prominent notices stating that You changed the files; and
|
||||
|
||||
c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
|
||||
|
||||
d) If the Work includes a “NOTICE” text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
|
||||
9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work
|
||||
To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
|
||||
|
||||
|
||||
Open Source Software Licensed Under the MIT License:
|
||||
----------------------------------------------------------------------------------------
|
||||
1. tsfresh v0.11.0
|
||||
Copyright (c) 2016 Maximilian Christ, Blue Yonder GmbH
|
||||
|
||||
2. react v16.4.1
|
||||
Copyright (c) 2013-present, Facebook, Inc.
|
||||
|
||||
3. react-router v4.3.1
|
||||
Copyright (c) React Training 2016-2018
|
||||
|
||||
4. lodash 4.17.5
|
||||
Copyright JS Foundation and other contributors <https://js.foundation/>
|
||||
|
||||
5. immutable-js v3.8.2
|
||||
Copyright (c) 2014-present, Facebook, Inc.
|
||||
|
||||
6. moment 2.17.1
|
||||
Copyright (c) JS Foundation and other contributors
|
||||
|
||||
7. Numeral-j 2.0.6
|
||||
Copyright (c) 2016 Adam Draper
|
||||
|
||||
8. shallowequal v1.1.0
|
||||
Copyright (c) 2017 Alberto Leal <mailforalberto@gmail.com> (github.com/dashed)
|
||||
|
||||
|
||||
Terms of the MIT License:
|
||||
---------------------------------------------------
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
|
||||
|
||||
Open Source Software Licensed Under the Python Software Foundation License Version 2:
|
||||
----------------------------------------------------------------------------------------
|
||||
1. Python 2.7.11
|
||||
Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011,
|
||||
2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved.
|
||||
|
||||
Copyright (c) 2000 BeOpen.com.
|
||||
All rights reserved.
|
||||
|
||||
Copyright (c) 1995-2001 Corporation for National Research Initiatives.
|
||||
All rights reserved.
|
||||
|
||||
Copyright (c) 1991-1995 Stichting Mathematisch Centrum.
|
||||
All rights reserved.
|
||||
|
||||
|
||||
Terms of the Python Software Foundation License Version 2:
|
||||
---------------------------------------------------
|
||||
A. HISTORY OF THE SOFTWARE
|
||||
==========================
|
||||
Python was created in the early 1990s by Guido van Rossum at Stichting
|
||||
Mathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands
|
||||
as a successor of a language called ABC. Guido remains Python's
|
||||
principal author, although it includes many contributions from others.
|
||||
|
||||
In 1995, Guido continued his work on Python at the Corporation for
|
||||
National Research Initiatives (CNRI, see http://www.cnri.reston.va.us)
|
||||
in Reston, Virginia where he released several versions of the
|
||||
software.
|
||||
|
||||
In May 2000, Guido and the Python core development team moved to
|
||||
BeOpen.com to form the BeOpen PythonLabs team. In October of the same
|
||||
year, the PythonLabs team moved to Digital Creations (now Zope
|
||||
Corporation, see http://www.zope.com). In 2001, the Python Software
|
||||
Foundation (PSF, see http://www.python.org/psf/) was formed, a
|
||||
non-profit organization created specifically to own Python-related
|
||||
Intellectual Property. Zope Corporation is a sponsoring member of
|
||||
the PSF.
|
||||
|
||||
All Python releases are Open Source (see http://www.opensource.org for
|
||||
the Open Source Definition). Historically, most, but not all, Python
|
||||
releases have also been GPL-compatible; the table below summarizes
|
||||
the various releases.
|
||||
|
||||
Release Derived Year Owner GPL-
|
||||
from compatible? (1)
|
||||
|
||||
0.9.0 thru 1.2 1991-1995 CWI yes
|
||||
1.3 thru 1.5.2 1.2 1995-1999 CNRI yes
|
||||
1.6 1.5.2 2000 CNRI no
|
||||
2.0 1.6 2000 BeOpen.com no
|
||||
1.6.1 1.6 2001 CNRI yes (2)
|
||||
2.1 2.0+1.6.1 2001 PSF no
|
||||
2.0.1 2.0+1.6.1 2001 PSF yes
|
||||
2.1.1 2.1+2.0.1 2001 PSF yes
|
||||
2.1.2 2.1.1 2002 PSF yes
|
||||
2.1.3 2.1.2 2002 PSF yes
|
||||
2.2 and above 2.1.1 2001-now PSF yes
|
||||
|
||||
Footnotes:
|
||||
|
||||
(1) GPL-compatible doesn't mean that we're distributing Python under
|
||||
the GPL. All Python licenses, unlike the GPL, let you distribute
|
||||
a modified version without making your changes open source. The
|
||||
GPL-compatible licenses make it possible to combine Python with
|
||||
other software that is released under the GPL; the others don't.
|
||||
|
||||
(2) According to Richard Stallman, 1.6.1 is not GPL-compatible,
|
||||
because its license has a choice of law clause. According to
|
||||
CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1
|
||||
is "not incompatible" with the GPL.
|
||||
|
||||
Thanks to the many outside volunteers who have worked under Guido's
|
||||
direction to make these releases possible.
|
||||
|
||||
B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
|
||||
===============================================================
|
||||
PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
|
||||
--------------------------------------------
|
||||
|
||||
1. This LICENSE AGREEMENT is between the Python Software Foundation
|
||||
("PSF"), and the Individual or Organization ("Licensee") accessing and
|
||||
otherwise using this software ("Python") in source or binary form and
|
||||
its associated documentation.
|
||||
|
||||
2. Subject to the terms and conditions of this License Agreement, PSF hereby
|
||||
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
|
||||
analyze, test, perform and/or display publicly, prepare derivative works,
|
||||
distribute, and otherwise use Python alone or in any derivative version,
|
||||
provided, however, that PSF's License Agreement and PSF's notice of copyright,
|
||||
i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
|
||||
2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved"
|
||||
are retained in Python alone or in any derivative version prepared by Licensee.
|
||||
|
||||
3. In the event Licensee prepares a derivative work that is based on
|
||||
or incorporates Python or any part thereof, and wants to make
|
||||
the derivative work available to others as provided herein, then
|
||||
Licensee hereby agrees to include in any such work a brief summary of
|
||||
the changes made to Python.
|
||||
|
||||
4. PSF is making Python available to Licensee on an "AS IS"
|
||||
basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
|
||||
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
|
||||
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR
|
||||
FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
|
||||
INFRINGE ANY THIRD PARTY RIGHTS.
|
||||
|
||||
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
|
||||
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
|
||||
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
|
||||
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
|
||||
|
||||
6. This License Agreement will automatically terminate upon a material
|
||||
breach of its terms and conditions.
|
||||
|
||||
7. Nothing in this License Agreement shall be deemed to create any
|
||||
relationship of agency, partnership, or joint venture between PSF and
|
||||
Licensee. This License Agreement does not grant permission to use PSF
|
||||
trademarks or trade name in a trademark sense to endorse or promote
|
||||
products or services of Licensee, or any third party.
|
||||
|
||||
8. By copying, installing or otherwise using Python, Licensee
|
||||
agrees to be bound by the terms and conditions of this License
|
||||
Agreement.
|
||||
|
||||
BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
|
||||
-------------------------------------------
|
||||
BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
|
||||
|
||||
1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
|
||||
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
|
||||
Individual or Organization ("Licensee") accessing and otherwise using
|
||||
this software in source or binary form and its associated
|
||||
documentation ("the Software").
|
||||
|
||||
2. Subject to the terms and conditions of this BeOpen Python License
|
||||
Agreement, BeOpen hereby grants Licensee a non-exclusive,
|
||||
royalty-free, world-wide license to reproduce, analyze, test, perform
|
||||
and/or display publicly, prepare derivative works, distribute, and
|
||||
otherwise use the Software alone or in any derivative version,
|
||||
provided, however, that the BeOpen Python License is retained in the
|
||||
Software, alone or in any derivative version prepared by Licensee.
|
||||
|
||||
3. BeOpen is making the Software available to Licensee on an "AS IS"
|
||||
basis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
|
||||
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
|
||||
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR
|
||||
FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE
|
||||
WILL NOT INFRINGE ANY THIRD PARTY RIGHTS.
|
||||
|
||||
4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
|
||||
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR
|
||||
LOSS AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR
|
||||
ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
|
||||
|
||||
5. This License Agreement will automatically terminate upon a material
|
||||
breach of its terms and conditions.
|
||||
|
||||
6. This License Agreement shall be governed by and interpreted in all
|
||||
respects by the law of the State of California, excluding conflict of
|
||||
law provisions. Nothing in this License Agreement shall be deemed to
|
||||
create any relationship of agency, partnership, or joint venture
|
||||
between BeOpen and Licensee. This License Agreement does not grant
|
||||
permission to use BeOpen trademarks or trade names in a trademark
|
||||
sense to endorse or promote products or services of Licensee, or any
|
||||
third party. As an exception, the "BeOpen Python" logos available at
|
||||
http://www.pythonlabs.com/logos.html may be used according to the
|
||||
permissions granted on that web page.
|
||||
|
||||
7. By copying, installing or otherwise using the software, Licensee
|
||||
agrees to be bound by the terms and conditions of this License
|
||||
Agreement.
|
||||
|
||||
|
||||
CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
|
||||
---------------------------------------
|
||||
1. This LICENSE AGREEMENT is between the Corporation for National
|
||||
Research Initiatives, having an office at 1895 Preston White Drive,
|
||||
Reston, VA 20191 ("CNRI"), and the Individual or Organization
|
||||
("Licensee") accessing and otherwise using Python 1.6.1 software in
|
||||
source or binary form and its associated documentation.
|
||||
|
||||
2. Subject to the terms and conditions of this License Agreement, CNRI
|
||||
hereby grants Licensee a nonexclusive, royalty-free, world-wide
|
||||
license to reproduce, analyze, test, perform and/or display publicly,
|
||||
prepare derivative works, distribute, and otherwise use Python 1.6.1
|
||||
alone or in any derivative version, provided, however, that CNRI's
|
||||
License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
|
||||
1995-2001 Corporation for National Research Initiatives; All Rights
|
||||
Reserved" are retained in Python 1.6.1 alone or in any derivative
|
||||
version prepared by Licensee. Alternately, in lieu of CNRI's License
|
||||
Agreement, Licensee may substitute the following text (omitting the
|
||||
quotes): "Python 1.6.1 is made available subject to the terms and
|
||||
conditions in CNRI's License Agreement. This Agreement together with
|
||||
Python 1.6.1 may be located on the Internet using the following
|
||||
unique, persistent identifier (known as a handle): 1895.22/1013. This
|
||||
Agreement may also be obtained from a proxy server on the Internet
|
||||
using the following URL: http://hdl.handle.net/1895.22/1013".
|
||||
|
||||
3. In the event Licensee prepares a derivative work that is based on
|
||||
or incorporates Python 1.6.1 or any part thereof, and wants to make
|
||||
the derivative work available to others as provided herein, then
|
||||
Licensee hereby agrees to include in any such work a brief summary of
|
||||
the changes made to Python 1.6.1.
|
||||
|
||||
4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
|
||||
basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
|
||||
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
|
||||
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR
|
||||
FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL
|
||||
NOT INFRINGE ANY THIRD PARTY RIGHTS.
|
||||
|
||||
5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
|
||||
1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
|
||||
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
|
||||
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
|
||||
|
||||
6. This License Agreement will automatically terminate upon a material
|
||||
breach of its terms and conditions.
|
||||
|
||||
7. This License Agreement shall be governed by the federal
|
||||
intellectual property law of the United States, including without
|
||||
limitation the federal copyright law, and, to the extent such
|
||||
U.S. federal law does not apply, by the law of the Commonwealth of
|
||||
Virginia, excluding Virginia's conflict of law provisions.
|
||||
Notwithstanding the foregoing, with regard to derivative works based
|
||||
on Python 1.6.1 that incorporate non-separable material that was
|
||||
previously distributed under the GNU General Public License (GPL), the
|
||||
law of the Commonwealth of Virginia shall govern this License
|
||||
Agreement only as to issues arising under or with respect to
|
||||
Paragraphs 4, 5, and 7 of this License Agreement. Nothing in this
|
||||
License Agreement shall be deemed to create any relationship of
|
||||
agency, partnership, or joint venture between CNRI and Licensee. This
|
||||
License Agreement does not grant permission to use CNRI trademarks or
|
||||
trade name in a trademark sense to endorse or promote products or
|
||||
services of Licensee, or any third party.
|
||||
|
||||
8. By clicking on the "ACCEPT" button where indicated, or by copying,
|
||||
installing or otherwise using Python 1.6.1, Licensee agrees to be
|
||||
bound by the terms and conditions of this License Agreement.
|
||||
|
||||
ACCEPT
|
||||
|
||||
|
||||
CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
|
||||
--------------------------------------------------
|
||||
Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
|
||||
The Netherlands. All rights reserved.
|
||||
|
||||
Permission to use, copy, modify, and distribute this software and its
|
||||
documentation for any purpose and without fee is hereby granted,
|
||||
provided that the above copyright notice appear in all copies and that
|
||||
both that copyright notice and this permission notice appear in
|
||||
supporting documentation, and that the name of Stichting Mathematisch
|
||||
Centrum or CWI not be used in advertising or publicity pertaining to
|
||||
distribution of the software without specific, written prior
|
||||
permission.
|
||||
|
||||
STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH
|
||||
REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH
|
||||
CENTRUM BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES
|
||||
OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
|
||||
PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
|
||||
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
|
||||
PERFORMANCE OF THIS SOFTWARE.
|
||||
|
||||
|
||||
|
||||
|
||||
Open Source Software Licensed Under the BSD 3-Clause License:
|
||||
----------------------------------------------------------------------------------------
|
||||
1. scikit-learn 0.19.1
|
||||
Copyright (c) 2007–2017 The scikit-learn developers.
|
||||
All rights reserved.
|
||||
|
||||
2. Django 1.11.13
|
||||
Copyright (c) Django Software Foundation and individual contributors.
|
||||
All rights reserved.
|
||||
|
||||
|
||||
Terms of the BSD 3-Clause License:
|
||||
--------------------------------------------------------------------
|
||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
||||
|
||||
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
||||
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
||||
Neither the name of [copyright holder] nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
|
@ -0,0 +1 @@
|
|||
[Click me switch to Chinese version](README.md)
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
[Click me switch to English version](README.en.md)
|
||||
|
||||
|
||||
|
||||
[](https://github.com/tencent/Metis/master/LICENSE.TXT)
|
||||
[](https://github.com/tencent/Metis/releases)
|
||||
[](https://github.com/tencent/Metis/pulls)
|
||||
|
||||
**Metis** 这个名字取自希腊神话中的智慧女神墨提斯（Metis），它是一系列AIOps领域的应用实践集合。主要解决在质量、效率、成本方面的智能运维问题。当前版本开源的时间序列异常检测学件，是从机器学习的角度来解决时序数据的异常检测问题。
|
||||
|
||||
时间序列异常检测学件的实现思路是基于统计判决、无监督和有监督学习对时序数据进行联合检测。通过统计判决、无监督算法进行首层判决，输出疑似异常，其次进行有监督模型判决，得到最终检测结果。检测模型是经大量样本训练生成，可根据样本持续训练更新。
|
||||
|
||||
时间序列异常检测学件在织云企业版本中已覆盖 **20w+** 服务器，承载了 **240w+** 业务指标的异常检测。经过了海量监控数据打磨，该学件在异常检测和运维监控领域具有广泛的应用性。
|
||||
|
||||
## 支持平台
|
||||
|
||||
目前运行的操作系统平台如下：
|
||||
|
||||
- 操作系统：Linux
|
||||
|
||||
## 支持语言
|
||||
|
||||
目前前后端支持的开发语言如下：
|
||||
|
||||
- 前端：JavaScript、TypeScript
|
||||
- 后端：Python 2.7
|
||||
|
||||
## 概览
|
||||
|
||||
* [使用场景](docs/usecase.md)
|
||||
* [代码目录](docs/code_framework.md)
|
||||
* [代码架构](docs/architecture.md)
|
||||
|
||||
## 安装指南
|
||||
|
||||
* 初次安装时，请参考安装说明文档 [install.md](docs/install.md)
|
||||
|
||||
## 使用指南
|
||||
|
||||
* [WEB使用说明](docs/web_userguide.md)
|
||||
* [API使用说明](docs/api_userguide.md)
|
||||
|
||||
## License
|
||||
|
||||
Metis的开源协议为BSD 3-Clause License，详情参见 [LICENSE.TXT](LICENSE.TXT)。
|
||||
|
||||
## 贡献代码
|
||||
|
||||
如果您使用过程中发现问题，请通过 [https://github.com/Tencent/Metis/issues](issues) 来提交并描述相关的问题，您也可以在这里查看其它的 issue ，通过解决这些 issue 来贡献代码。
|
||||
|
||||
如果您是第一次贡献代码，请阅读 [CONTRIBUTING](CONTRIBUTING.md) 了解我们的贡献流程，并提交 pull request 给我们。
|
||||
|
||||
## 联系方式
|
||||
|
||||
qq技术交流群1群：288723616。
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["config", "controller", "dao", "model", "service", "utils"]
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["database", "common", "errorcode"]
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
DEFAULT_WINDOW = 180
|
||||
INPUT_LEN_ENG_MAX = 32
|
||||
INPUT_LEN_CH_MAX = 64
|
||||
INPUT_ITEM_PER_PAGE_MAX = 100
|
||||
INPUT_LIST_LEN_MAX = 5
|
||||
VALUE_LEN_MAX = 50000
|
||||
UPLOAD_PATH = '/tmp/tmpfile_%s.csv'
|
||||
MARK_POSITIVE = 1
|
||||
MARK_NEGATIVE = 2
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
db = 'metis'
|
||||
user = 'metis'
|
||||
passwd = 'metis@123'
|
||||
host = '127.0.0.1'
|
||||
port = 3306
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
OP_SUCCESS = 0
|
||||
THROW_EXP = 1000
|
||||
OP_DB_FAILED = 1001
|
||||
CHECK_PARAM_FAILED = 1002
|
||||
FILE_FORMAT_ERR = 1003
|
||||
NOT_POST = 1004
|
||||
NOT_GET = 1005
|
||||
CAL_FEATURE_ERR = 2001
|
||||
READ_FEATURE_FAILED = 2002
|
||||
TRAIN_ERR = 2003
|
||||
LACK_SAMPLE = 2004
|
||||
|
||||
ERR_CODE = {
|
||||
0: "操作成功",
|
||||
1000: "抛出异常",
|
||||
1001: "数据库操作失败",
|
||||
1002: "参数检查失败",
|
||||
1003: "文件格式有误",
|
||||
1004: "非post请求",
|
||||
1005: "非get请求",
|
||||
2001: "特征计算出错",
|
||||
2002: "读取特征数据失败",
|
||||
2003: "训练出错",
|
||||
2004: "缺少正样本或负样本"
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from django.contrib import admin
|
||||
|
||||
# Register your models here.
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from django.apps import AppConfig
|
||||
|
||||
|
||||
class ApiConfig(AppConfig):
|
||||
name = 'api'
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from django.db import models
|
||||
|
||||
# Create your models here.
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from django.test import TestCase
|
||||
|
||||
# Create your tests here.
|
||||
|
|
@ -0,0 +1,199 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
from __future__ import unicode_literals
|
||||
import json
|
||||
from django.shortcuts import render
|
||||
from django.http import FileResponse
|
||||
from common.render import render_json
|
||||
from app.service.time_series_detector.anomaly_service import *
|
||||
from app.service.time_series_detector.sample_service import *
|
||||
from app.service.time_series_detector.task_service import *
|
||||
from app.service.time_series_detector.detect_service import *
|
||||
from app.config.errorcode import *
|
||||
from app.utils.utils import *
|
||||
|
||||
|
||||
def search_anomaly(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
anomaly_service = AnomalyService()
|
||||
return_dict = anomaly_service.query_anomaly(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def import_sample(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
sample_service = SampleService()
|
||||
return_dict = sample_service.import_file(request.FILES)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def update_sample(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
sample_service = SampleService()
|
||||
return_dict = sample_service.update_sample(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def query_sample(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
sample_service = SampleService()
|
||||
return_dict = sample_service.query_sample(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def update_anomaly(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
sample_service = AnomalyService()
|
||||
return_dict = sample_service.update_anomaly(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def train(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
detect_service = DetectService()
|
||||
return_dict = detect_service.process_train(json.loads(request.body))
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def download_sample(request):
|
||||
if request.method == "GET":
|
||||
try:
|
||||
sample_service = SampleService()
|
||||
file_name = sample_service.sample_download(request.GET['id'])
|
||||
files = open(file_name, 'rb')
|
||||
response = FileResponse(files)
|
||||
response['Content-Type'] = 'application/octet-stream'
|
||||
response['Content-Disposition'] = 'attachment;filename = "SampleExport.csv"'
|
||||
return response
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_GET)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def predict_rate(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
detect_service = DetectService()
|
||||
return_dict = detect_service.rate_predict(json.loads(request.body))
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def predict_value(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
detect_service = DetectService()
|
||||
return_dict = detect_service.value_predict(json.loads(request.body))
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def query_train_task(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
train_service = TrainService()
|
||||
return_dict = train_service.query_train(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def query_train_source(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
sample_service = SampleService()
|
||||
return_dict = sample_service.query_sample_source()
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def delete_train_task(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
train_service = TrainService()
|
||||
return_dict = train_service.delete_train(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def delete_sample(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
sample_service = SampleService()
|
||||
return_dict = sample_service.delete_sample(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
||||
|
||||
def count_sample(request):
|
||||
if request.method == "POST":
|
||||
try:
|
||||
sample_service = SampleService()
|
||||
return_dict = sample_service.count_sample(request.body)
|
||||
except Exception, ex:
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return render_json(return_dict)
|
||||
else:
|
||||
return_dict = build_ret_data(NOT_POST)
|
||||
return render_json(return_dict)
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
import json
|
||||
from django.http import HttpResponse
|
||||
|
||||
|
||||
def render_json(dictionary={}):
|
||||
response = HttpResponse(json.dumps(dictionary), content_type="application/json")
|
||||
response['Access-Control-Allow-Origin'] = '*'
|
||||
response["Access-Control-Allow-Headers"] = "Origin, X-Requested-With, Content-Type"
|
||||
response["Access-Control-Allow-Methods"] = "GET, POST, PUT, OPTIONS"
|
||||
return response
|
||||
Binary file not shown.
|
|
@ -0,0 +1,43 @@
|
|||
#!/usr/bin/env python
|
||||
import os
|
||||
import sys
|
||||
import signal
|
||||
import errno
|
||||
|
||||
|
||||
def wait_child(signum, frame):
|
||||
print('receive SIGCHLD')
|
||||
try:
|
||||
while True:
|
||||
cpid, status = os.waitpid(-1, os.WNOHANG)
|
||||
if cpid == 0:
|
||||
print('no child process was immediately available')
|
||||
break
|
||||
exitcode = status >> 8
|
||||
print('child process %s exit with exitcode %s', cpid, exitcode)
|
||||
except OSError as e:
|
||||
if e.errno == errno.ECHILD:
|
||||
print ('current process has no existing unwaited-for child processes.')
|
||||
else:
|
||||
raise
|
||||
print('handle SIGCHLD end')
|
||||
|
||||
if __name__ == "__main__":
|
||||
signal.signal(signal.SIGCHLD, wait_child)
|
||||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings.settings")
|
||||
try:
|
||||
from django.core.management import execute_from_command_line
|
||||
except ImportError:
|
||||
# The above import may fail for some other reason. Ensure that the
|
||||
# issue is really that Django is missing to avoid masking other
|
||||
# exceptions on Python 2.
|
||||
try:
|
||||
import django
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"Couldn't import Django. Are you sure it's installed and "
|
||||
"available on your PYTHONPATH environment variable? Did you "
|
||||
"forget to activate a virtual environment?"
|
||||
)
|
||||
raise
|
||||
execute_from_command_line(sys.argv)
|
||||
|
|
@ -0,0 +1,120 @@
|
|||
"""
|
||||
Django settings for settings project.
|
||||
|
||||
Generated by 'django-admin startproject' using Django 1.11.13.
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/1.11/topics/settings/
|
||||
|
||||
For the full list of settings and their values, see
|
||||
https://docs.djangoproject.com/en/1.11/ref/settings/
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
|
||||
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
|
||||
# Quick-start development settings - unsuitable for production
|
||||
# See https://docs.djangoproject.com/en/1.11/howto/deployment/checklist/
|
||||
|
||||
# SECURITY WARNING: keep the secret key used in production secret!
|
||||
SECRET_KEY = 'd4b+($dxc75y@n=!ssd^%y78g-u6kbk%6_mg0(ft(n(e=#dfbh'
|
||||
|
||||
# SECURITY WARNING: don't run with debug turned on in production!
|
||||
DEBUG = True
|
||||
|
||||
ALLOWED_HOSTS = ['*']
|
||||
|
||||
|
||||
# Application definition
|
||||
|
||||
INSTALLED_APPS = [
|
||||
'django.contrib.admin',
|
||||
'django.contrib.auth',
|
||||
'django.contrib.contenttypes',
|
||||
'django.contrib.sessions',
|
||||
'django.contrib.messages',
|
||||
'django.contrib.staticfiles',
|
||||
]
|
||||
|
||||
MIDDLEWARE = [
|
||||
'django.middleware.security.SecurityMiddleware',
|
||||
'django.contrib.sessions.middleware.SessionMiddleware',
|
||||
'django.middleware.common.CommonMiddleware',
|
||||
# 'django.middleware.csrf.CsrfViewMiddleware',
|
||||
'django.contrib.auth.middleware.AuthenticationMiddleware',
|
||||
'django.contrib.messages.middleware.MessageMiddleware',
|
||||
'django.middleware.clickjacking.XFrameOptionsMiddleware',
|
||||
]
|
||||
|
||||
ROOT_URLCONF = 'settings.urls'
|
||||
|
||||
TEMPLATES = [
|
||||
{
|
||||
'BACKEND': 'django.template.backends.django.DjangoTemplates',
|
||||
'DIRS': [],
|
||||
'APP_DIRS': True,
|
||||
'OPTIONS': {
|
||||
'context_processors': [
|
||||
'django.template.context_processors.debug',
|
||||
'django.template.context_processors.request',
|
||||
'django.contrib.auth.context_processors.auth',
|
||||
'django.contrib.messages.context_processors.messages',
|
||||
],
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
WSGI_APPLICATION = 'settings.wsgi.application'
|
||||
|
||||
|
||||
# Database
|
||||
# https://docs.djangoproject.com/en/1.11/ref/settings/#databases
|
||||
|
||||
DATABASES = {
|
||||
'default': {
|
||||
'ENGINE': 'django.db.backends.sqlite3',
|
||||
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
# Password validation
|
||||
# https://docs.djangoproject.com/en/1.11/ref/settings/#auth-password-validators
|
||||
|
||||
AUTH_PASSWORD_VALIDATORS = [
|
||||
{
|
||||
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
|
||||
},
|
||||
{
|
||||
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
|
||||
},
|
||||
{
|
||||
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
|
||||
},
|
||||
{
|
||||
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
# Internationalization
|
||||
# https://docs.djangoproject.com/en/1.11/topics/i18n/
|
||||
|
||||
LANGUAGE_CODE = 'en-us'
|
||||
|
||||
TIME_ZONE = 'UTC'
|
||||
|
||||
USE_I18N = True
|
||||
|
||||
USE_L10N = True
|
||||
|
||||
USE_TZ = True
|
||||
|
||||
|
||||
# Static files (CSS, JavaScript, Images)
|
||||
# https://docs.djangoproject.com/en/1.11/howto/static-files/
|
||||
|
||||
STATIC_URL = '/static/'
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
"""settings URL Configuration
|
||||
|
||||
The `urlpatterns` list routes URLs to views. For more information please see:
|
||||
https://docs.djangoproject.com/en/1.11/topics/http/urls/
|
||||
Examples:
|
||||
Function views
|
||||
1. Add an import: from my_app import views
|
||||
2. Add a URL to urlpatterns: url(r'^$', views.home, name='home')
|
||||
Class-based views
|
||||
1. Add an import: from other_app.views import Home
|
||||
2. Add a URL to urlpatterns: url(r'^$', Home.as_view(), name='home')
|
||||
Including another URLconf
|
||||
1. Import the include() function: from django.conf.urls import url, include
|
||||
2. Add a URL to urlpatterns: url(r'^blog/', include('blog.urls'))
|
||||
"""
|
||||
from django.conf.urls import url
|
||||
from django.contrib import admin
|
||||
from api import views as api_views
|
||||
|
||||
urlpatterns = [
|
||||
url(r'^admin/', admin.site.urls),
|
||||
url(r'^SearchAnomaly$', api_views.search_anomaly, name='search_anomaly'),
|
||||
url(r'^ImportSample$', api_views.import_sample, name='import_sample'),
|
||||
url(r'^UpdateSample$', api_views.update_sample, name='update_sample'),
|
||||
url(r'^QuerySample$', api_views.query_sample, name='query_sample'),
|
||||
url(r'^DeleteSample$', api_views.delete_sample, name='delete_sample'),
|
||||
url(r'^CountSample$', api_views.count_sample, name='count_sample'),
|
||||
url(r'^UpdateAnomaly$', api_views.update_anomaly, name='update_anomaly'),
|
||||
url(r'^DownloadSample/', api_views.download_sample, name='download_sample'),
|
||||
url(r'^QueryTrain$', api_views.query_train_task, name='query_train_task'),
|
||||
url(r'^QueryTrainSource$', api_views.query_train_source, name='query_train_source'),
|
||||
url(r'^DeleteTrain$', api_views.delete_train_task, name='delete_train_task'),
|
||||
url(r'^Train$', api_views.train, name='train'),
|
||||
url(r'^PredictRate$', api_views.predict_rate, name='predict_rate'),
|
||||
url(r'^PredictValue$', api_views.predict_value, name='predict_value'),
|
||||
]
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
"""
|
||||
WSGI config for settings project.
|
||||
|
||||
It exposes the WSGI callable as a module-level variable named ``application``.
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/1.11/howto/deployment/wsgi/
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from django.core.wsgi import get_wsgi_application
|
||||
|
||||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings.settings")
|
||||
|
||||
application = get_wsgi_application()
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["time_series_detector"]
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["anomaly_op", "sample_op", "train_op"]
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import time
|
||||
import datetime
|
||||
import MySQLdb
|
||||
from app.config import database
|
||||
from app.dao.time_series_detector.sample_op import *
|
||||
from app.config.common import *
|
||||
from app.config.errorcode import *
|
||||
|
||||
|
||||
class AbnormalOperation(object):
|
||||
|
||||
def __init__(self):
|
||||
self.__conn = MySQLdb.connect(host=database.host, port=database.port, user=database.user, passwd=database.passwd, db=database.db)
|
||||
self.__cur = self.__conn.cursor()
|
||||
self.__cur.execute("SET NAMES UTF8")
|
||||
self.__sample = SampleOperation()
|
||||
|
||||
def __del__(self):
|
||||
self.__conn.close()
|
||||
|
||||
def get_anomaly(self, form):
|
||||
request_page = form['requestPage']
|
||||
item_per_page = form['itemPerPage']
|
||||
attr_id = form['attrId']
|
||||
view_id = form['viewId']
|
||||
beg_limit = (form['itemPerPage'] * (form['requestPage'] - 1))
|
||||
limit = form['itemPerPage']
|
||||
params = []
|
||||
query_str = ""
|
||||
params.append(form['beginTime'])
|
||||
params.append(form['endTime'])
|
||||
if attr_id != "":
|
||||
params.append(attr_id.encode('utf8'))
|
||||
params.append(("%" + attr_id + "%").encode('utf8'))
|
||||
query_str += " and (attr_id = %s or attr_name like %s) "
|
||||
if view_id != "":
|
||||
params.append(view_id.encode('utf8'))
|
||||
params.append(("%" + view_id + "%").encode('utf8'))
|
||||
query_str += "and (view_id = %s or view_name like %s) "
|
||||
|
||||
params.append(beg_limit)
|
||||
params.append(limit)
|
||||
command = 'SELECT id, view_id, view_name, attr_id, attr_name, UNIX_TIMESTAMP(time), data_c, data_b, data_a, mark_flag FROM anomaly WHERE time > from_unixtime(%s) and time < from_unixtime(%s) ' + query_str + 'LIMIT %s, %s;'
|
||||
command_count = 'SELECT count(*) FROM anomaly WHERE time > from_unixtime(%s) and time < from_unixtime(%s) ' + query_str
|
||||
length = self.__cur.execute(command, params)
|
||||
abnormal_list = []
|
||||
query_res = self.__cur.fetchmany(length)
|
||||
for row in query_res:
|
||||
abnormal_list.append({
|
||||
"id": row[0],
|
||||
"viewId": row[1],
|
||||
"viewName": row[2],
|
||||
"attrId": row[3],
|
||||
"attrName": row[4],
|
||||
"time": row[5],
|
||||
"dataC": row[6].split(','),
|
||||
"dataB": row[7].split(','),
|
||||
"dataA": row[8].split(','),
|
||||
"markFlag": row[9]
|
||||
})
|
||||
self.__cur.execute(command_count, params[:-2])
|
||||
total_count = int(self.__cur.fetchone()[0])
|
||||
total_page = int(total_count) / item_per_page
|
||||
current_page = min(request_page, total_page)
|
||||
|
||||
return OP_SUCCESS, {
|
||||
"anomalyList": abnormal_list,
|
||||
"currentPage": current_page,
|
||||
"totalCount": total_count
|
||||
}
|
||||
|
||||
def update_anomaly(self, data):
|
||||
update_str = "UPDATE anomaly set mark_flag = %s where id = %s"
|
||||
params = [data['markFlag'], data['id']]
|
||||
record_num = self.__cur.execute(update_str, params)
|
||||
self.__conn.commit()
|
||||
|
||||
if MARK_NEGATIVE == data['markFlag'] or MARK_POSITIVE == data['markFlag']:
|
||||
select_str = 'SELECT view_name, view_id, attr_name, attr_id, UNIX_TIMESTAMP(time), data_c, data_b, data_a, mark_flag, id FROM anomaly where id = %s'
|
||||
self.__cur.execute(select_str, [data['id']])
|
||||
row = self.__cur.fetchone()
|
||||
insert_data = []
|
||||
window = row[7].count(',')
|
||||
one_item = {
|
||||
"viewName": row[0],
|
||||
"viewId": row[1],
|
||||
"attrName": row[2],
|
||||
"attrId": row[3],
|
||||
"source": "unknown",
|
||||
"trainOrTest": "train",
|
||||
"positiveOrNegative": "positive" if MARK_POSITIVE == data['markFlag'] else "negative",
|
||||
"window": window,
|
||||
"dataC": row[5],
|
||||
"dataB": row[6],
|
||||
"dataA": row[7],
|
||||
"dataTime": row[4],
|
||||
"anomalyId": row[9],
|
||||
}
|
||||
insert_data.append(one_item)
|
||||
ret_code, ret_data = self.__sample.import_sample(insert_data)
|
||||
else:
|
||||
ret_code, ret_data = self.__sample.delete_sample_by_anomaly_id(data)
|
||||
record_num = ret_data
|
||||
return ret_code, record_num
|
||||
|
||||
def insert_anomaly(self, data):
|
||||
insert_str = "INSERT INTO anomaly(view_id, view_name, attr_name, attr_id, time, data_c, data_b, data_a) values(%s, %s, %s, %s, %s, %s, %s, %s);"
|
||||
time_str = datetime.datetime.fromtimestamp(int(time.time())).strftime("%Y-%m-%d %H:%M:%S")
|
||||
params = [data['view_id'], data['view_name'].encode('utf8'), data['attr_name'].encode('utf8'), data['attr_id'], time_str, data['data_c'], data['data_b'], data['data_a']]
|
||||
record_num = self.__cur.execute(insert_str, params)
|
||||
self.__conn.commit()
|
||||
return OP_SUCCESS, record_num
|
||||
|
|
@ -0,0 +1,282 @@
|
|||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import uuid
|
||||
import csv
|
||||
import codecs
|
||||
import MySQLdb
|
||||
from app.config import database
|
||||
from app.config.common import *
|
||||
from app.config.errorcode import *
|
||||
|
||||
|
||||
class SampleOperation(object):
|
||||
|
||||
def __init__(self):
|
||||
self.__conn = MySQLdb.connect(host=database.host, port=database.port, user=database.user, passwd=database.passwd, db=database.db)
|
||||
self.__cur = self.__conn.cursor()
|
||||
self.__cur.execute("SET NAMES UTF8")
|
||||
|
||||
def __del__(self):
|
||||
self.__conn.close()
|
||||
|
||||
def import_sample(self, data):
|
||||
params = []
|
||||
insert_str = "INSERT INTO sample_dataset(view_id, view_name, attr_name, attr_id, source, train_or_test, positive_or_negative, window, data_time, data_c, data_b, data_a, anomaly_id) values(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);"
|
||||
for row in data:
|
||||
params.append([row['viewId'], row['viewName'], row['attrName'], row['attrId'], row['source'], row['trainOrTest'], row['positiveOrNegative'], row['window'], row['dataTime'], row['dataC'], row['dataB'], row['dataA'], row['anomalyId']])
|
||||
num = self.__cur.executemany(insert_str, params)
|
||||
self.__conn.commit()
|
||||
return OP_SUCCESS, num
|
||||
|
||||
def update_sample(self, data):
|
||||
params = []
|
||||
update_str = ""
|
||||
update_data = data['data']
|
||||
if update_data['updateTime'] != "":
|
||||
update_time_str = datetime.datetime.fromtimestamp(update_data['updateTime']).strftime("%Y-%m-%d %H:%M:%S")
|
||||
params.append(update_time_str)
|
||||
update_str += "update_time = %s, "
|
||||
if update_data['viewId'] != "":
|
||||
params.append(update_data['viewId'])
|
||||
update_str += "view_id = %s, "
|
||||
if update_data['viewName'] != "":
|
||||
params.append(update_data['viewName'].encode('utf8'))
|
||||
update_str += "view_name = %s, "
|
||||
if update_data['attrName'] != "":
|
||||
params.append(update_data['attrName'].encode('utf8'))
|
||||
update_str += "attr_name = %s, "
|
||||
if update_data['attrId'] != "":
|
||||
params.append(update_data['attrId'])
|
||||
update_str += "attr_id = %s, "
|
||||
if update_data['source'] != "":
|
||||
params.append(update_data['source'])
|
||||
update_str += "source = %s, "
|
||||
if update_data['trainOrTest'] != "":
|
||||
params.append(update_data['trainOrTest'])
|
||||
update_str += "train_or_test = %s, "
|
||||
if update_data['positiveOrNegative'] != "":
|
||||
params.append(update_data['positiveOrNegative'])
|
||||
update_str += "positive_or_negative = %s, "
|
||||
if update_data['window'] != "":
|
||||
params.append(update_data['window'])
|
||||
update_str += "window = %s, "
|
||||
if update_data['dataTime'] != "":
|
||||
params.append(update_data['dataTime'])
|
||||
update_str += "data_time = %s, "
|
||||
|
||||
if update_str == "":
|
||||
return CHECK_PARAM_FAILED, ""
|
||||
|
||||
command = "UPDATE sample_dataset set " + update_str[:-2] + " where id = %s "
|
||||
all_params = []
|
||||
|
||||
for id_num in data['idList']:
|
||||
all_params.append(tuple(params + [id_num]))
|
||||
num = self.__cur.executemany(command, all_params)
|
||||
self.__conn.commit()
|
||||
return OP_SUCCESS, num
|
||||
|
||||
def sample_query_all(self, data):
|
||||
params = []
|
||||
query_str = ""
|
||||
params.append(DEFAULT_WINDOW)
|
||||
params.append(data['beginTime'])
|
||||
params.append(data['endTime'])
|
||||
|
||||
if data['trainOrTest'] != "":
|
||||
train_str = ""
|
||||
for one_source in data['trainOrTest']:
|
||||
params.append(one_source)
|
||||
train_str += 'train_or_test = %s or '
|
||||
query_str += ' and (' + train_str[:-4] + ') '
|
||||
if data['positiveOrNegative'] != "":
|
||||
params.append(data['positiveOrNegative'])
|
||||
query_str += " and positive_or_negative = %s "
|
||||
if data['source'] != "":
|
||||
source_str = ""
|
||||
for one_source in data['source']:
|
||||
params.append(one_source)
|
||||
source_str += 'source = %s or '
|
||||
query_str += ' and (' + source_str[:-4] + ') '
|
||||
|
||||
command = 'SELECT data_c, data_b, data_a, positive_or_negative FROM sample_dataset WHERE window = %s and data_time > %s and data_time < %s ' + query_str
|
||||
length = self.__cur.execute(command, params)
|
||||
sample_list = []
|
||||
query_res = self.__cur.fetchmany(length)
|
||||
for row in query_res:
|
||||
sample_list.append({
|
||||
"data": row[0] + ',' + row[1] + ',' + row[2],
|
||||
"flag": 1 if row[3] == 'positive' else 0
|
||||
})
|
||||
return sample_list
|
||||
|
||||
def sample_count(self, data):
|
||||
params = []
|
||||
query_str = ""
|
||||
params.append(DEFAULT_WINDOW)
|
||||
params.append(data['beginTime'])
|
||||
params.append(data['endTime'])
|
||||
|
||||
if data['trainOrTest'] != "":
|
||||
train_str = ""
|
||||
for one_source in data['trainOrTest']:
|
||||
params.append(one_source)
|
||||
train_str += "train_or_test = %s or "
|
||||
query_str += " and (" + train_str[:-4] + ") "
|
||||
if data['positiveOrNegative'] != "":
|
||||
params.append(data['positiveOrNegative'])
|
||||
query_str += " and positive_or_negative = %s "
|
||||
if data['source'] != "":
|
||||
source_str = ""
|
||||
for one_source in data['source']:
|
||||
params.append(one_source)
|
||||
source_str += 'source = %s or '
|
||||
query_str += " and (" + source_str[:-4] + ") "
|
||||
|
||||
command = 'SELECT count(*), count(if(positive_or_negative = "positive", 1, NULL)), count(if(positive_or_negative = "negative", 1, NULL)) FROM sample_dataset WHERE window = %s and data_time > %s and data_time < %s ' + query_str
|
||||
length = self.__cur.execute(command, params)
|
||||
|
||||
sample_list = []
|
||||
query_res = self.__cur.fetchmany(length)
|
||||
for row in query_res:
|
||||
sample_list.append({
|
||||
"total_count": int(row[0]),
|
||||
"positive_count": int(row[1]),
|
||||
"negative_count": int(row[2])
|
||||
})
|
||||
|
||||
return OP_SUCCESS, sample_list
|
||||
|
||||
def download_sample(self, data):
|
||||
sample_list = []
|
||||
id_list = data.split(',')
|
||||
format_strings = ','.join(['%s'] * len(id_list))
|
||||
command = 'SELECT view_name, view_id, attr_name, attr_id, source, train_or_test, positive_or_negative, window, data_c, data_b, data_a, data_time FROM sample_dataset WHERE id in (%s) ' % format_strings
|
||||
length = self.__cur.execute(command, id_list)
|
||||
query_res = self.__cur.fetchmany(length)
|
||||
for row in query_res:
|
||||
sample_list.append([
|
||||
row[0],
|
||||
row[1],
|
||||
row[2],
|
||||
row[3],
|
||||
row[4],
|
||||
row[5],
|
||||
row[6],
|
||||
row[7],
|
||||
row[8],
|
||||
row[9],
|
||||
row[10],
|
||||
row[11]
|
||||
])
|
||||
head = ['指标集名称', '指标集id', '指标名称', '指标id', '样本来源', '训练集_测试集', '正样本_负样本', '样本窗口', 'dataC', 'dataB', 'dataA', '数据时间戳']
|
||||
uuid_str = uuid.uuid4().hex[:8]
|
||||
download_file_path = UPLOAD_PATH % uuid_str
|
||||
with open(download_file_path, 'w') as pfile:
|
||||
pfile.write(codecs.BOM_UTF8)
|
||||
writer = csv.writer(pfile)
|
||||
writer.writerow(head)
|
||||
writer.writerows(sample_list)
|
||||
return download_file_path
|
||||
|
||||
def query_sample(self, data):
|
||||
item_per_page = data['itemPerPage']
|
||||
request_page = data['requestPage']
|
||||
beg_limit = (item_per_page * (request_page - 1))
|
||||
limit = (item_per_page)
|
||||
params = []
|
||||
query_str = ""
|
||||
|
||||
if data['beginTime'] != "" and data['endTime'] != "":
|
||||
params.append(data['beginTime'])
|
||||
params.append(data['endTime'])
|
||||
query_str += " and data_time > %s and data_time < %s "
|
||||
if data['attrId'] != "":
|
||||
params.append(data['attrId'].encode('utf8'))
|
||||
params.append(("%" + data['attrId'] + "%").encode('utf8'))
|
||||
query_str += " and (attr_id = %s or attr_name like %s) "
|
||||
if data['viewId'] != "":
|
||||
params.append(data['viewId'].encode('utf8'))
|
||||
params.append(("%" + data['viewId'] + "%").encode('utf8'))
|
||||
query_str += " and (view_id = %s or view_name like %s) "
|
||||
if data['positiveOrNegative'] != "":
|
||||
params.append(data['positiveOrNegative'])
|
||||
query_str += " and positive_or_negative = %s "
|
||||
if data['source'] != "":
|
||||
params.append(data['source'])
|
||||
query_str += " and source = %s "
|
||||
if data['trainOrTest'] != "":
|
||||
params.append(data['trainOrTest'])
|
||||
query_str += " and train_or_test = %s "
|
||||
if data['window'] != "":
|
||||
params.append(data['window'])
|
||||
query_str += " and window = %s "
|
||||
if query_str != "":
|
||||
query_str = " WHERE " + query_str[5:]
|
||||
|
||||
params.append(beg_limit)
|
||||
params.append(limit)
|
||||
command = 'SELECT id, view_id, view_name, attr_id, attr_name, data_time, data_c, data_b, data_a, positive_or_negative, source, train_or_test, window FROM sample_dataset ' + query_str + ' LIMIT %s, %s;'
|
||||
command_count = 'SELECT count(*) FROM sample_dataset ' + query_str
|
||||
length = self.__cur.execute(command, params)
|
||||
|
||||
sample_list = []
|
||||
query_res = self.__cur.fetchmany(length)
|
||||
for row in query_res:
|
||||
sample_list.append({
|
||||
"id": row[0],
|
||||
"viewId": row[1],
|
||||
"viewName": row[2],
|
||||
"attrId": row[3],
|
||||
"attrName": row[4],
|
||||
"time": row[5],
|
||||
"dataC": row[6],
|
||||
"dataB": row[7],
|
||||
"dataA": row[8],
|
||||
"positiveOrNegative": row[9],
|
||||
"source": row[10],
|
||||
"trainOrTest": row[11],
|
||||
"window": row[12]
|
||||
})
|
||||
self.__cur.execute(command_count, params[:-2])
|
||||
total_count = int(self.__cur.fetchone()[0])
|
||||
total_page = total_count / item_per_page
|
||||
current_page = min(request_page, total_page)
|
||||
return 0, {
|
||||
"sampleList": sample_list,
|
||||
"currentPage": current_page,
|
||||
"totalTotal": total_count
|
||||
}
|
||||
|
||||
def delete_sample(self, data):
|
||||
id_num = data['id']
|
||||
command = "delete from sample_dataset where id = %s "
|
||||
num = self.__cur.execute(command, id_num)
|
||||
self.__conn.commit()
|
||||
return OP_SUCCESS, num
|
||||
|
||||
def delete_sample_by_anomaly_id(self, data):
|
||||
id_num = data['id']
|
||||
command = "delete from sample_dataset where anomaly_id = %s "
|
||||
num = self.__cur.execute(command, [id_num])
|
||||
self.__conn.commit()
|
||||
return OP_SUCCESS, num
|
||||
|
||||
def query_sample_source(self):
|
||||
command = "select distinct source from sample_dataset"
|
||||
num = self.__cur.execute(command)
|
||||
source_list = []
|
||||
query_res = self.__cur.fetchmany(num)
|
||||
for row in query_res:
|
||||
source_list.append(row[0])
|
||||
return OP_SUCCESS, {
|
||||
"source": source_list
|
||||
}
|
||||
|
|
@ -0,0 +1,117 @@
|
|||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import MySQLdb
|
||||
from app.config import database
|
||||
from app.config.common import *
|
||||
from app.config.errorcode import *
|
||||
|
||||
|
||||
class TrainOperation(object):
|
||||
|
||||
def __init__(self):
|
||||
self.__conn = MySQLdb.connect(host=database.host, port=database.port, user=database.user, passwd=database.passwd, db=database.db)
|
||||
self.__cur = self.__conn.cursor()
|
||||
self.__cur.execute("SET NAMES UTF8")
|
||||
|
||||
def __del__(self):
|
||||
self.__conn.close()
|
||||
|
||||
def query_train(self, data):
|
||||
request_page = data['requestPage']
|
||||
item_per_page = data['itemPerPage']
|
||||
begin_time = data['beginTime']
|
||||
end_time = data['endTime']
|
||||
task_id = data['taskId']
|
||||
task_status = data['taskStatus']
|
||||
beg_limit = (item_per_page * (request_page - 1))
|
||||
limit = item_per_page
|
||||
params = []
|
||||
query_str = ""
|
||||
|
||||
if task_id != "":
|
||||
params.append(("%" + task_id + "%").encode('utf8'))
|
||||
params.append(("%" + task_id + "%").encode('utf8'))
|
||||
query_str += " and (task_id like %s or model_name like %s) "
|
||||
if begin_time != "" and end_time != "":
|
||||
params.append(begin_time)
|
||||
params.append(end_time)
|
||||
query_str += " and start_time > from_unixtime(%s) and end_time < from_unixtime(%s) "
|
||||
if task_status != "":
|
||||
params.append(task_status)
|
||||
query_str += " and status = %s "
|
||||
|
||||
params.append(beg_limit)
|
||||
params.append(limit)
|
||||
command = 'SELECT task_id, sample_num, postive_sample_num, negative_sample_num, window, model_name, source, UNIX_TIMESTAMP(start_time), UNIX_TIMESTAMP(end_time), status FROM train_task where 1 = 1 ' + query_str + ' order by start_time desc LIMIT %s,%s;'
|
||||
command_count = 'SELECT count(*) FROM train_task where 1 = 1' + query_str
|
||||
length = self.__cur.execute(command, params)
|
||||
task_list = []
|
||||
query_res = self.__cur.fetchmany(length)
|
||||
for row in query_res:
|
||||
task_list.append({
|
||||
"id": row[0],
|
||||
"sampleNum": row[1],
|
||||
"positiveSampleNum": row[2],
|
||||
"negativeSampleNum": row[3],
|
||||
"window": row[4],
|
||||
"modelName": row[5],
|
||||
"source": row[6],
|
||||
"startTime": row[7],
|
||||
"endTime": row[8],
|
||||
"status": row[9]
|
||||
})
|
||||
|
||||
self.__cur.execute(command_count, params[:-2])
|
||||
total_count = int(self.__cur.fetchone()[0])
|
||||
total_page = int(total_count) / item_per_page
|
||||
current_page = min(request_page, total_page)
|
||||
|
||||
return OP_SUCCESS, {
|
||||
"taskList": task_list,
|
||||
"currentPage": current_page,
|
||||
"totalCount": total_count
|
||||
}
|
||||
|
||||
def query_train_source(self):
|
||||
command = "select distinct source from train_task"
|
||||
num = self.__cur.execute(command)
|
||||
source_list = []
|
||||
query_res = self.__cur.fetchmany(num)
|
||||
for row in query_res:
|
||||
source_list.append(row[0])
|
||||
return OP_SUCCESS, {
|
||||
"source": source_list
|
||||
}
|
||||
|
||||
def insert_train_info(self, data):
|
||||
command = "insert into train_task(task_id, sample_num, postive_sample_num, negative_sample_num, window, model_name, source, start_time, end_time, status) values(%s, %s, %s, %s, %s, %s, %s, from_unixtime(%s), from_unixtime(%s), %s)"
|
||||
|
||||
num = self.__cur.execute(command, [data['task_id'], data['sample_num'], data['postive_sample_num'], data['negative_sample_num'], DEFAULT_WINDOW, "", ','.join(data['source']), data['begin_time'], data['end_time'], data['status']])
|
||||
self.__conn.commit()
|
||||
return num
|
||||
|
||||
def delete_train(self, data):
|
||||
task_id = data['taskId']
|
||||
command = "delete from train_task where task_id = %s "
|
||||
num = self.__cur.execute(command, [task_id])
|
||||
self.__conn.commit()
|
||||
return OP_SUCCESS, num
|
||||
|
||||
def update_model_info(self, data):
|
||||
command = "UPDATE train_task SET end_time = from_unixtime(%s), status = (%s), model_name = %s where task_id = %s"
|
||||
num = self.__cur.execute(command, [data['end_time'], data['status'], data['model_name'], data['task_id']])
|
||||
self.__conn.commit()
|
||||
return num
|
||||
|
||||
def update_sample_info(self, data):
|
||||
command = "UPDATE train_task SET end_time = from_unixtime(%s), status = %s, sample_num = %s, postive_sample_num = %s, negative_sample_num =%s where task_id = %s"
|
||||
num = self.__cur.execute(command, [data['end_time'], data['status'], data['sample_num'], data['postive_sample_num'], data['negative_sample_num'], data['task_id']])
|
||||
self.__conn.commit()
|
||||
return num
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["time_series_detector"]
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["time_series_detector"]
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["algorithm", "feature", "anomaly_service", "sample_service", "task_service", "detect_service"]
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["ewma", "gbdt", "statistic", "isolation_forest", "xgboosting", "polynomial_interpolation", "ewma_and_polynomial"]
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding=utf-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
class Ewma(object):
|
||||
"""
|
||||
In statistical quality control, the EWMA chart (or exponentially weighted moving average chart)
|
||||
is a type of control chart used to monitor either variables or attributes-type data using the monitored business
|
||||
or industrial process's entire history of output. While other control charts treat rational subgroups of samples
|
||||
individually, the EWMA chart tracks the exponentially-weighted moving average of all prior sample means.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/EWMA_chart
|
||||
"""
|
||||
|
||||
def __init__(self, alpha=0.3, coefficient=3):
|
||||
"""
|
||||
:param alpha: Discount rate of ewma, usually in (0.2, 0.3).
|
||||
:param coefficient: Coefficient is the width of the control limits, usually in (2.7, 3.0).
|
||||
"""
|
||||
self.alpha = alpha
|
||||
self.coefficient = coefficient
|
||||
|
||||
def predict(self, X):
|
||||
"""
|
||||
Predict if a particular sample is an outlier or not.
|
||||
|
||||
:param X: the time series to detect of
|
||||
:param type X: pandas.Series
|
||||
:return: 1 denotes normal, 0 denotes abnormal
|
||||
"""
|
||||
s = [X[0]]
|
||||
for i in range(1, len(X)):
|
||||
temp = self.alpha * X[i] + (1 - self.alpha) * s[-1]
|
||||
s.append(temp)
|
||||
s_avg = np.mean(s)
|
||||
sigma = np.sqrt(np.var(X))
|
||||
ucl = s_avg + self.coefficient * sigma * np.sqrt(self.alpha / (2 - self.alpha))
|
||||
lcl = s_avg - self.coefficient * sigma * np.sqrt(self.alpha / (2 - self.alpha))
|
||||
if s[-1] > ucl or s[-1] < lcl:
|
||||
return 0
|
||||
return 1
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
from app.service.time_series_detector.algorithm import ewma
|
||||
from app.service.time_series_detector.algorithm import polynomial_interpolation
|
||||
|
||||
|
||||
class EwmaAndPolynomialInterpolation(object):
|
||||
|
||||
def __init__(self, alpha=0.3, coefficient=3, threshold=0.15, degree=4):
|
||||
"""
|
||||
:param alpha: Discount rate of ewma, usually in (0.2, 0.3).
|
||||
:param coefficient: Coefficient is the width of the control limits, usually in (2.7, 3.0).
|
||||
:param threshold: The critical point of normal.
|
||||
:param degree: Depth of iteration.
|
||||
"""
|
||||
self.alpha = alpha
|
||||
self.coefficient = coefficient
|
||||
self.degree = degree
|
||||
self.threshold = threshold
|
||||
|
||||
def predict(self, X, window=180):
|
||||
"""
|
||||
Predict if a particular sample is an outlier or not.
|
||||
|
||||
:param X: the time series to detect of
|
||||
:param type X: pandas.Series
|
||||
:param: window: the length of window
|
||||
:param type window: int
|
||||
:return: 1 denotes normal, 0 denotes abnormal
|
||||
"""
|
||||
ewma_obj = ewma.Ewma(self.alpha, self.coefficient)
|
||||
ewma_ret = ewma_obj.predict(X)
|
||||
if ewma_ret == 1:
|
||||
result = 1
|
||||
else:
|
||||
polynomial_obj = polynomial_interpolation.PolynomialInterpolation(self.threshold, self.degree)
|
||||
polynomial_ret = polynomial_obj.predict(X, window)
|
||||
result = polynomial_ret
|
||||
return result
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import os
|
||||
import pickle
|
||||
import numpy as np
|
||||
from sklearn.ensemble import GradientBoostingClassifier
|
||||
from sklearn.externals import joblib
|
||||
from app.service.time_series_detector.feature import feature_service
|
||||
from app.utils.utils import *
|
||||
from app.config.errorcode import *
|
||||
|
||||
MODEL_PATH = os.path.join(os.path.dirname(__file__), '../../../model/time_series_detector/')
|
||||
DEFAULT_MODEL = MODEL_PATH + "gbdt_default_model"
|
||||
|
||||
|
||||
class Gbdt(object):
|
||||
"""
|
||||
Gradient boosting is a machine learning technique for regression and classification problems,
|
||||
which produces a prediction model in the form of an ensemble of weak prediction models,
|
||||
typically decision trees. It builds the model in a stage-wise fashion like other boosting methods do,
|
||||
and it generalizes them by allowing optimization of an arbitrary differentiable loss function.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/Gradient_boosting
|
||||
"""
|
||||
|
||||
def __init__(self, threshold=0.15, n_estimators=300, max_depth=10, learning_rate=0.05):
|
||||
"""
|
||||
:param threshold: The critical point of normal.
|
||||
:param n_estimators: The number of boosting stages to perform. Gradient boosting is fairly robust to over-fitting so a large number usually results in better performance.
|
||||
:param max_depth: Maximum depth of the individual regression estimators. The maximum depth limits the number of nodes in the tree.
|
||||
:param learning_rate: Learning rate shrinks the contribution of each tree by `learning_rate`. There is a trade-off between learning_rate and n_estimators.
|
||||
"""
|
||||
self.threshold = threshold
|
||||
self.n_estimators = n_estimators
|
||||
self.max_depth = max_depth
|
||||
self.learning_rate = learning_rate
|
||||
|
||||
def __calculate_features(self, data, window=180):
|
||||
"""
|
||||
Caculate time features.
|
||||
|
||||
:param data: the time series to detect of
|
||||
:param window: the length of window
|
||||
"""
|
||||
features = []
|
||||
for index in data:
|
||||
if is_standard_time_series(index["data"], window):
|
||||
temp = []
|
||||
temp.append(feature_service.extract_features(index["data"], window))
|
||||
temp.append(index["flag"])
|
||||
features.append(temp)
|
||||
return features
|
||||
|
||||
def gbdt_train(self, data, task_id, window=180):
|
||||
"""
|
||||
Train a gbdt model.
|
||||
|
||||
:param data: Training dataset.
|
||||
:param task_id: The id of the training task.
|
||||
:param window: the length of window
|
||||
"""
|
||||
X_train = []
|
||||
y_train = []
|
||||
features = self.__calculate_features(data, window)
|
||||
if features:
|
||||
return LACK_SAMPLE
|
||||
for index in features:
|
||||
X_train.append(index[0])
|
||||
y_train.append(index[1])
|
||||
X_train = np.array(X_train)
|
||||
y_train = np.array(y_train)
|
||||
try:
|
||||
grd = GradientBoostingClassifier(n_estimators=self.n_estimators, max_depth=self.max_depth, learning_rate=self.learning_rate)
|
||||
grd.fit(X_train, y_train)
|
||||
model_name = MODEL_PATH + task_id + "_model"
|
||||
joblib.dump(grd, model_name)
|
||||
except Exception, ex:
|
||||
return TRAIN_ERR, str(ex)
|
||||
return OP_SUCCESS, ""
|
||||
|
||||
def predict(self, X, window=180, model_name=DEFAULT_MODEL):
|
||||
"""
|
||||
Predict if a particular sample is an outlier or not.
|
||||
|
||||
:param X: the time series to detect of
|
||||
:param type X: pandas.Series
|
||||
:param window: the length of window
|
||||
:param type window: int
|
||||
:param model_name: the model to use
|
||||
:param type model_name: string
|
||||
:return 1 denotes normal, 0 denotes abnormal
|
||||
"""
|
||||
if is_standard_time_series(X):
|
||||
ts_features = feature_service.extract_features(X, window)
|
||||
ts_features = np.array([ts_features])
|
||||
load_model = pickle.load(open(model_name, "rb"))
|
||||
gbdt_ret = load_model.predict_proba(ts_features)[:, 1]
|
||||
if gbdt_ret[0] < self.threshold:
|
||||
value = 0
|
||||
else:
|
||||
value = 1
|
||||
return [value, gbdt_ret[0]]
|
||||
else:
|
||||
return [0, 0]
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
from sklearn.ensemble import IsolationForest
|
||||
|
||||
|
||||
class IForest(object):
|
||||
"""
|
||||
The IsolationForest 'isolates' observations by randomly selecting a feature and then
|
||||
randomly selecting a split value between the maximum and minimum values of the selected feature.
|
||||
|
||||
https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf
|
||||
"""
|
||||
|
||||
def __init__(self,
|
||||
n_estimators=3,
|
||||
max_samples="auto",
|
||||
contamination=0.15,
|
||||
max_feature=1.,
|
||||
bootstrap=False,
|
||||
n_jobs=1,
|
||||
random_state=None,
|
||||
verbose=0):
|
||||
"""
|
||||
:param n_estimators: The number of base estimators in the ensemble.
|
||||
:param max_samples: The number of samples to draw from X to train each base estimator.
|
||||
:param coefficient: The amount of contamination of the data set, i.e. the proportion of outliers in the data set. Used when fitting to define the threshold on the decision function.
|
||||
:param max_features: The number of features to draw from X to train each base estimator.
|
||||
:param bootstrap: If True, individual trees are fit on random subsets of the training data sampled with replacement. If False, sampling without replacement is performed.
|
||||
:param random_state: If int, random_state is the seed used by the random number generator;
|
||||
If RandomState instance, random_state is the random number generator;
|
||||
If None, the random number generator is the RandomState instance used by `np.random`.
|
||||
:param verbose: Controls the verbosity of the tree building process.
|
||||
"""
|
||||
self.n_estimators = n_estimators
|
||||
self.max_samples = max_samples
|
||||
self.contamination = contamination
|
||||
self.max_feature = max_feature
|
||||
self.bootstrap = bootstrap
|
||||
self.n_jobs = n_jobs
|
||||
self.random_state = random_state
|
||||
self.verbose = verbose
|
||||
|
||||
def predict(self, X, window=180):
|
||||
"""
|
||||
Predict if a particular sample is an outlier or not.
|
||||
|
||||
:param X: the time series to detect of
|
||||
:param type X: pandas.Series
|
||||
:param window: the length of window
|
||||
:param type window: int
|
||||
:return: 1 denotes normal, 0 denotes abnormal.
|
||||
"""
|
||||
x_train = list(range(0, 2 * window + 1)) + list(range(0, 2 * window + 1)) + list(range(0, window + 1))
|
||||
sample_features = zip(x_train, X)
|
||||
clf = IsolationForest(self.n_estimators, self.max_samples, self.contamination, self.max_feature, self.bootstrap, self.n_jobs, self.random_state, self.verbose)
|
||||
clf.fit(sample_features)
|
||||
predict_res = clf.predict(sample_features)
|
||||
if predict_res[-1] == -1:
|
||||
return 0
|
||||
return 1
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
from sklearn.linear_model import Ridge
|
||||
from sklearn.preprocessing import PolynomialFeatures
|
||||
from sklearn.pipeline import make_pipeline
|
||||
|
||||
|
||||
class PolynomialInterpolation(object):
|
||||
"""
|
||||
In statistics, polynomial regression is a form of regression analysis in which the relationship
|
||||
between the independent variable x and the dependent variable y is modelled as an nth degree polynomial in x.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/Polynomial_regression
|
||||
"""
|
||||
|
||||
def __init__(self, threshold=0.15, degree=4):
|
||||
"""
|
||||
:param threshold: The critical point of normal.
|
||||
:param degree: Depth of iteration.
|
||||
"""
|
||||
self.degree = degree
|
||||
self.threshold = threshold
|
||||
|
||||
def predict(self, X, window=180):
|
||||
"""
|
||||
Predict if a particular sample is an outlier or not.
|
||||
|
||||
:param X: the time series to detect of
|
||||
:param type X: pandas.Series
|
||||
:param window: the length of window
|
||||
:param type window: int
|
||||
:return: 1 denotes normal, 0 denotes abnormal
|
||||
"""
|
||||
x_train = list(range(0, 2 * window + 1)) + list(range(0, 2 * window + 1)) + list(range(0, window + 1))
|
||||
x_train = np.array(x_train)
|
||||
x_train = x_train[:, np.newaxis]
|
||||
avg_value = np.mean(X[-(window + 1):])
|
||||
if avg_value > 1:
|
||||
y_train = X / avg_value
|
||||
else:
|
||||
y_train = X
|
||||
model = make_pipeline(PolynomialFeatures(self.degree), Ridge())
|
||||
model.fit(x_train, y_train)
|
||||
if abs(y_train[-1] - model.predict(np.array(x_train[-1]).reshape(1, -1))) > self.threshold:
|
||||
return 0
|
||||
return 1
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
class Statistic(object):
|
||||
"""
|
||||
In statistics, the 68-95-99.7 rule is a shorthand used to remember the percentage of values
|
||||
that lie within a band around the mean in a normal distribution with a width of two, four and
|
||||
six standard deviations, respectively; more accurately, 68.27%, 95.45% and 99.73% of the values
|
||||
lie within one, two and three standard deviations of the mean, respectively.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule
|
||||
"""
|
||||
|
||||
def __init__(self, index=3):
|
||||
"""
|
||||
:param index: multiple of standard deviation
|
||||
:param type: int or float
|
||||
"""
|
||||
self.index = index
|
||||
|
||||
def predict(self, X):
|
||||
"""
|
||||
Predict if a particular sample is an outlier or not.
|
||||
|
||||
:param X: the time series to detect of
|
||||
:param type X: pandas.Series
|
||||
:return: 1 denotes normal, 0 denotes abnormal
|
||||
"""
|
||||
if abs(X[-1] - np.mean(X[:-1])) > self.index * np.std(X[:-1]):
|
||||
return 0
|
||||
return 1
|
||||
|
|
@ -0,0 +1,171 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import os
|
||||
import xgboost as xgb
|
||||
from app.service.time_series_detector.feature import feature_service
|
||||
from app.utils.utils import *
|
||||
from app.config.errorcode import *
|
||||
MODEL_PATH = os.path.join(os.path.dirname(__file__), '../../../model/time_series_detector/')
|
||||
DEFAULT_MODEL = MODEL_PATH + "xgb_default_model"
|
||||
|
||||
|
||||
class XGBoosting(object):
|
||||
"""
|
||||
XGBoost is an optimized distributed gradient boosting library designed to be highly efficient,
|
||||
flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework.
|
||||
XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems
|
||||
in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI)
|
||||
and can solve problems beyond billions of examples.
|
||||
|
||||
https://github.com/dmlc/xgboost
|
||||
"""
|
||||
|
||||
def __init__(self,
|
||||
threshold=0.15,
|
||||
max_depth=10,
|
||||
eta=0.05,
|
||||
gamma=0.1,
|
||||
silent=1,
|
||||
min_child_weight=1,
|
||||
subsample=0.8,
|
||||
colsample_bytree=1,
|
||||
booster='gbtree',
|
||||
objective='binary:logistic',
|
||||
eval_metric='auc'):
|
||||
"""
|
||||
:param threshold: The critical point of normal.
|
||||
:param max_depth: Maximum tree depth for base learners.
|
||||
:param eta: Value means model more robust to overfitting but slower to compute.
|
||||
:param gamma: Minimum loss reduction required to make a further partition on a leaf node of the tree.
|
||||
:param silent: If 1, it will print information about performance. If 2, some additional information will be printed out.
|
||||
:param min_child_weight: Minimum sum of instance weight(hessian) needed in a child.
|
||||
:param subsample: Subsample ratio of the training instance.
|
||||
:param colsample_bytree: Subsample ratio of columns when constructing each tree.
|
||||
:param booster: Specify which booster to use: gbtree, gblinear or dart.
|
||||
:param objective: Specify the learning task and the corresponding learning objective or a custom objective function to be used (see note below).
|
||||
:param eval_metric: If a str, should be a built-in evaluation metric to use. See doc/parameter.md. If callable, a custom evaluation metric.
|
||||
"""
|
||||
self.threshold = threshold
|
||||
self.max_depth = max_depth
|
||||
self.eta = eta
|
||||
self.gamma = gamma
|
||||
self.silent = silent
|
||||
self.min_child_weight = min_child_weight
|
||||
self.subsample = subsample
|
||||
self.colsample_bytree = colsample_bytree
|
||||
self.booster = booster
|
||||
self.objective = objective
|
||||
self.eval_metric = eval_metric
|
||||
|
||||
def __save_libsvm_format(self, data, feature_file_name):
|
||||
"""
|
||||
Save the time features to libsvm format.
|
||||
|
||||
:param data: feature values
|
||||
:param file_name: file saves the time features and label
|
||||
"""
|
||||
try:
|
||||
f = open(feature_file_name, "w")
|
||||
except Exception, ex:
|
||||
return CAL_FEATURE_ERR, str(ex)
|
||||
times = 0
|
||||
for temp in data:
|
||||
if times > 0:
|
||||
f.write("\n")
|
||||
result = ['{0}:{1}'.format(int(index) + 1, value) for index, value in enumerate(temp[0])]
|
||||
f.write(str(temp[1]))
|
||||
for x in result:
|
||||
f.write(' ' + x)
|
||||
times = times + 1
|
||||
return OP_SUCCESS, ""
|
||||
|
||||
def __calculate_features(self, data, feature_file_name, window=180):
|
||||
"""
|
||||
Caculate time features and save as libsvm format.
|
||||
|
||||
:param data: the time series to detect of
|
||||
:param feature_file_name: the file to use
|
||||
:param window: the length of window
|
||||
"""
|
||||
features = []
|
||||
for index in data:
|
||||
if is_standard_time_series(index["data"], window):
|
||||
temp = []
|
||||
temp.append(feature_service.extract_features(index["data"], window))
|
||||
temp.append(index["flag"])
|
||||
features.append(temp)
|
||||
try:
|
||||
ret_code, ret_data = self.__save_libsvm_format(features, feature_file_name)
|
||||
except Exception, ex:
|
||||
ret_code = CAL_FEATURE_ERR
|
||||
ret_data = str(ex)
|
||||
return ret_code, ret_data
|
||||
|
||||
def xgb_train(self, data, task_id, num_round=300):
|
||||
"""
|
||||
Train an xgboost model.
|
||||
|
||||
:param data: Training dataset.
|
||||
:param task_id: The id of the training task.
|
||||
:param num_round: Max number of boosting iterations.
|
||||
"""
|
||||
model_name = MODEL_PATH + task_id + "_model"
|
||||
feature_file_name = MODEL_PATH + task_id + "_features"
|
||||
ret_code, ret_data = self.__calculate_features(data, feature_file_name)
|
||||
if ret_code != OP_SUCCESS:
|
||||
return ret_code, ret_data
|
||||
try:
|
||||
dtrain = xgb.DMatrix(feature_file_name)
|
||||
except Exception, ex:
|
||||
return READ_FEATURE_FAILED, str(ex)
|
||||
params = {
|
||||
'max_depth': self.max_depth,
|
||||
'eta': self.eta,
|
||||
'gamma': self.gamma,
|
||||
'silent': self.silent,
|
||||
'min_child_weight': self.min_child_weight,
|
||||
'subsample': self.subsample,
|
||||
'colsample_bytree': self.colsample_bytree,
|
||||
'booster': self.booster,
|
||||
'objective': self.objective,
|
||||
'eval_metric': self.eval_metric,
|
||||
}
|
||||
try:
|
||||
bst = xgb.train(params, dtrain, num_round)
|
||||
bst.save_model(model_name)
|
||||
except Exception, ex:
|
||||
return TRAIN_ERR, str(ex)
|
||||
return OP_SUCCESS, ""
|
||||
|
||||
def predict(self, X, window=180, model_name=DEFAULT_MODEL):
|
||||
"""
|
||||
:param X: the time series to detect of
|
||||
:type X: pandas.Series
|
||||
:param window: the length of window
|
||||
:param model_name: Use a xgboost model to predict a particular sample is an outlier or not.
|
||||
:return 1 denotes normal, 0 denotes abnormal.
|
||||
"""
|
||||
if is_standard_time_series(X, window):
|
||||
ts_features = []
|
||||
features = [10]
|
||||
features.extend(feature_service.extract_features(X, window))
|
||||
ts_features.append(features)
|
||||
res_pred = xgb.DMatrix(np.array(ts_features))
|
||||
bst = xgb.Booster({'nthread': 4})
|
||||
bst.load_model(model_name)
|
||||
xgb_ret = bst.predict(res_pred)
|
||||
if xgb_ret[0] < self.threshold:
|
||||
value = 0
|
||||
else:
|
||||
value = 1
|
||||
return [value, xgb_ret[0]]
|
||||
else:
|
||||
return [0, 0]
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import json
|
||||
import traceback
|
||||
from app.dao.time_series_detector.anomaly_op import *
|
||||
from app.utils.utils import *
|
||||
|
||||
|
||||
class AnomalyService(object):
|
||||
|
||||
def __init__(self):
|
||||
self.__anomaly = AbnormalOperation()
|
||||
|
||||
def query_anomaly(self, body):
|
||||
try:
|
||||
form = json.loads(body)
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
ret_code, ret_data = self.__anomaly.get_anomaly(form)
|
||||
return_dict = build_ret_data(ret_code, ret_data)
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def update_anomaly(self, body):
|
||||
try:
|
||||
form = json.loads(body)
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
print form
|
||||
ret_code, ret_data = self.__anomaly.update_anomaly(form)
|
||||
return_dict = build_ret_data(ret_code, ret_data)
|
||||
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
|
@ -0,0 +1,219 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import time
|
||||
import os
|
||||
from multiprocessing import Process
|
||||
from app.dao.time_series_detector import anomaly_op
|
||||
from app.dao.time_series_detector import sample_op
|
||||
from app.dao.time_series_detector import train_op
|
||||
from app.utils.utils import *
|
||||
from app.service.time_series_detector.algorithm import isolation_forest, ewma, polynomial_interpolation, statistic, xgboosting
|
||||
from app.config.errorcode import *
|
||||
MODEL_PATH = os.path.join(os.path.dirname(__file__), '../../model/time_series_detector/')
|
||||
|
||||
|
||||
class DetectService(object):
|
||||
|
||||
def __init__(self):
|
||||
self.sample_op_obj = sample_op.SampleOperation()
|
||||
self.anomaly_op_obj = anomaly_op.AbnormalOperation()
|
||||
self.iforest_obj = isolation_forest.IForest()
|
||||
self.ewma_obj = ewma.Ewma()
|
||||
self.polynomial_obj = polynomial_interpolation.PolynomialInterpolation()
|
||||
self.statistic_obj = statistic.Statistic()
|
||||
self.supervised_obj = xgboosting.XGBoosting()
|
||||
|
||||
def __generate_model(self, data, task_id):
|
||||
"""
|
||||
Start train a model
|
||||
|
||||
:param data: Training dataset.
|
||||
:param task_id: The id of the training task.
|
||||
"""
|
||||
xgb_obj = xgboosting.XGBoosting()
|
||||
# pylint: disable=unused-variable
|
||||
ret_code, ret_data = xgb_obj.xgb_train(data, task_id)
|
||||
current_timestamp = int(time.time())
|
||||
train_op_obj = train_op.TrainOperation()
|
||||
if ret_code == 0:
|
||||
train_status = "complete"
|
||||
params = {
|
||||
"task_id": task_id,
|
||||
"end_time": current_timestamp,
|
||||
"status": train_status,
|
||||
"model_name": task_id + "_model"
|
||||
}
|
||||
else:
|
||||
train_status = "failed"
|
||||
params = {
|
||||
"task_id": task_id,
|
||||
"end_time": current_timestamp,
|
||||
"status": train_status,
|
||||
"model_name": ""
|
||||
}
|
||||
train_op_obj.update_model_info(params)
|
||||
|
||||
def process_train(self, data):
|
||||
"""
|
||||
Start a process to train model
|
||||
:param data: Training dataset.
|
||||
"""
|
||||
sample_params = {
|
||||
"trainOrTest": data["trainOrTest"],
|
||||
"positiveOrNegative": data["positiveOrNegative"],
|
||||
"source": data["source"],
|
||||
"beginTime": data["beginTime"],
|
||||
"endTime": data["endTime"]
|
||||
}
|
||||
samples = self.sample_op_obj.sample_query_all(sample_params)
|
||||
train_op_obj = train_op.TrainOperation()
|
||||
samples_list = []
|
||||
positive_count = 0
|
||||
negative_count = 0
|
||||
for index in samples:
|
||||
samples_list.append({"flag": index["flag"], "data": map(int, index["data"].split(','))})
|
||||
if index["flag"] == 1:
|
||||
positive_count = positive_count + 1
|
||||
else:
|
||||
negative_count = negative_count + 1
|
||||
task_id = str(int(round(time.time() * 1000)))
|
||||
train_params = {
|
||||
"begin_time": int(time.time()),
|
||||
"end_time": int(time.time()),
|
||||
"task_id": task_id,
|
||||
"status": "running",
|
||||
"source": data["source"],
|
||||
"sample_num": len(samples_list),
|
||||
"postive_sample_num": positive_count,
|
||||
"negative_sample_num": negative_count
|
||||
}
|
||||
if positive_count == 0 or negative_count == 0:
|
||||
return build_ret_data(LACK_SAMPLE, "")
|
||||
train_op_obj.insert_train_info(train_params)
|
||||
try:
|
||||
process = Process(target=self.__generate_model, args=(samples_list, task_id, ))
|
||||
process.start()
|
||||
except Exception:
|
||||
train_status = "failed"
|
||||
params = {
|
||||
"task_id": task_id,
|
||||
"end_time": int(time.time()),
|
||||
"status": train_status,
|
||||
"model_name": ""
|
||||
}
|
||||
train_op_obj.update_model_info(params)
|
||||
return build_ret_data(OP_SUCCESS, "")
|
||||
|
||||
def __list_is_digit(self, data):
|
||||
for index in data:
|
||||
try:
|
||||
float(index)
|
||||
except ValueError:
|
||||
return False
|
||||
return True
|
||||
|
||||
def __check_param(self, data):
|
||||
if ("viewName" not in data.keys()) or ("attrId" not in data.keys()) or ("attrName" not in data.keys()) or ("time" not in data.keys()) or ("dataC" not in data.keys()) or ("dataB" not in data.keys()) or ("dataA" not in data.keys()):
|
||||
return CHECK_PARAM_FAILED, "missing parameter"
|
||||
if not data['dataA']:
|
||||
return CHECK_PARAM_FAILED, "dataA can not be empty"
|
||||
if not data['dataB']:
|
||||
return CHECK_PARAM_FAILED, "dataB can not be empty"
|
||||
if not data['dataC']:
|
||||
return CHECK_PARAM_FAILED, "dataC can not be empty"
|
||||
if not self.__list_is_digit(data['dataA'].split(',')):
|
||||
return CHECK_PARAM_FAILED, "dataA contains illegal numbers"
|
||||
if not self.__list_is_digit(data['dataB'].split(',')):
|
||||
return CHECK_PARAM_FAILED, "dataB contains illegal numbers"
|
||||
if not self.__list_is_digit(data['dataC'].split(',')):
|
||||
return CHECK_PARAM_FAILED, "dataC contains illegal numbers"
|
||||
if "window" in data:
|
||||
window = data["window"]
|
||||
else:
|
||||
window = 180
|
||||
if len(data['dataC'].split(',')) != (2 * window + 1):
|
||||
return CHECK_PARAM_FAILED, "dataC is not long enough"
|
||||
if len(data['dataB'].split(',')) != (2 * window + 1):
|
||||
return CHECK_PARAM_FAILED, "dataB is not long enough"
|
||||
if len(data['dataA'].split(',')) != (window + 1):
|
||||
return CHECK_PARAM_FAILED, "dataA is not long enough"
|
||||
return OP_SUCCESS, ""
|
||||
|
||||
def value_predict(self, data):
|
||||
"""
|
||||
Predict the data
|
||||
|
||||
:param data: the time series to detect of
|
||||
"""
|
||||
ret_code, ret_data = self.__check_param(data)
|
||||
if ret_code != OP_SUCCESS:
|
||||
return build_ret_data(ret_code, ret_data)
|
||||
if "taskId" in data and data["taskId"]:
|
||||
model_name = MODEL_PATH + data["taskId"] + "_model"
|
||||
else:
|
||||
model_name = MODEL_PATH + "xgb_default_model"
|
||||
combined_data = data["dataC"] + "," + data["dataB"] + "," + data["dataA"]
|
||||
time_series = map(int, combined_data.split(','))
|
||||
if "window" in data:
|
||||
window = data["window"]
|
||||
else:
|
||||
window = 180
|
||||
statistic_result = self.statistic_obj.predict(time_series)
|
||||
ewma_result = self.ewma_obj.predict(time_series)
|
||||
polynomial_result = self.polynomial_obj.predict(time_series, window)
|
||||
iforest_result = self.iforest_obj.predict(time_series, window)
|
||||
if statistic_result == 0 or ewma_result == 0 or polynomial_result == 0 or iforest_result == 0:
|
||||
xgb_result = self.supervised_obj.predict(time_series, window, model_name)
|
||||
res_value = xgb_result[0]
|
||||
prob = xgb_result[1]
|
||||
else:
|
||||
res_value = 1
|
||||
prob = 1
|
||||
ret_data = {"ret": res_value, "p": str(prob)}
|
||||
if ret_data["ret"] == 0:
|
||||
anomaly_params = {
|
||||
"view_id": data["viewId"],
|
||||
"view_name": data["viewName"],
|
||||
"attr_id": data["attrId"],
|
||||
"attr_name": data["attrName"],
|
||||
"time": data["time"],
|
||||
"data_c": data["dataC"],
|
||||
"data_b": data["dataB"],
|
||||
"data_a": data["dataA"]
|
||||
}
|
||||
self.anomaly_op_obj.insert_anomaly(anomaly_params)
|
||||
return build_ret_data(OP_SUCCESS, ret_data)
|
||||
|
||||
def rate_predict(self, data):
|
||||
ret_code, ret_data = check_value(data)
|
||||
if ret_code != OP_SUCCESS:
|
||||
return build_ret_data(ret_code, ret_data)
|
||||
combined_data = data["dataC"] + "," + data["dataB"] + "," + data["dataA"]
|
||||
time_series = map(float, combined_data.split(','))
|
||||
statistic_result = self.statistic_obj.predict(time_series)
|
||||
if statistic_result == 0:
|
||||
prob = 0
|
||||
else:
|
||||
prob = 1
|
||||
ret_data = {"ret": statistic_result, "p": str(prob)}
|
||||
if ret_data["ret"] == 0:
|
||||
anomaly_params = {
|
||||
"view_id": data["viewId"],
|
||||
"view_name": data["viewName"],
|
||||
"attr_id": data["attrId"],
|
||||
"attr_name": data["attrName"],
|
||||
"time": data["time"],
|
||||
"data_c": data["dataC"],
|
||||
"data_b": data["dataB"],
|
||||
"data_a": data["dataA"]
|
||||
}
|
||||
self.anomaly_op_obj.insert_anomaly(anomaly_params)
|
||||
return build_ret_data(OP_SUCCESS, ret_data)
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["classification_features", "feature_service", "fitting_features", "statistical_features"]
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
import tsfresh.feature_extraction.feature_calculators as ts_feature_calculators
|
||||
|
||||
|
||||
def time_series_autocorrelation(x):
|
||||
"""
|
||||
Calculates the autocorrelation of the specified lag, according to the formula [1]
|
||||
|
||||
.. math::
|
||||
|
||||
\\frac{1}{(n-l)\sigma^{2}} \\sum_{t=1}^{n-l}(X_{t}-\\mu )(X_{t+l}-\\mu)
|
||||
|
||||
where :math:`n` is the length of the time series :math:`X_i`, :math:`\sigma^2` its variance and :math:`\mu` its
|
||||
mean. `l` denotes the lag.
|
||||
|
||||
.. rubric:: References
|
||||
|
||||
[1] https://en.wikipedia.org/wiki/Autocorrelation#Estimation
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:param lag: the lag
|
||||
:type lag: int
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
lag = int((len(x) - 3) / 5)
|
||||
return ts_feature_calculators.autocorrelation(x, lag)
|
||||
|
||||
|
||||
def time_series_coefficient_of_variation(x):
|
||||
"""
|
||||
Calculates the coefficient of variation, mean value / square root of variation
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return np.mean(x) / np.sqrt(np.var(x))
|
||||
|
||||
|
||||
def time_series_binned_entropy(x):
|
||||
"""
|
||||
First bins the values of x into max_bins equidistant bins.
|
||||
Then calculates the value of
|
||||
|
||||
.. math::
|
||||
|
||||
- \\sum_{k=0}^{min(max\\_bins, len(x))} p_k log(p_k) \\cdot \\mathbf{1}_{(p_k > 0)}
|
||||
|
||||
where :math:`p_k` is the percentage of samples in bin :math:`k`.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:param max_bins: the maximal number of bins
|
||||
:type max_bins: int
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
max_bins = [2, 4, 6, 8, 10, 20]
|
||||
result = []
|
||||
for value in max_bins:
|
||||
result.append(ts_feature_calculators.binned_entropy(x, value))
|
||||
return result
|
||||
|
||||
# add yourself classification features here...
|
||||
|
||||
|
||||
def get_classification_features(x):
|
||||
classification_features = []
|
||||
classification_features.append(time_series_autocorrelation(x))
|
||||
classification_features.append(time_series_coefficient_of_variation(x))
|
||||
classification_features.extend(time_series_binned_entropy(x))
|
||||
# append yourself classification features here...
|
||||
|
||||
return classification_features
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import statistical_features
|
||||
import classification_features
|
||||
import fitting_features
|
||||
from app.utils import utils
|
||||
|
||||
|
||||
def extract_features(time_series, window):
|
||||
"""
|
||||
Extracts three types of features from the time series.
|
||||
|
||||
:param time_series: the time series to extract the feature of
|
||||
:type time_series: pandas.Series
|
||||
:param window: the length of window
|
||||
:type window: int
|
||||
:return: the value of features
|
||||
:return type: list with float
|
||||
"""
|
||||
if not utils.is_standard_time_series(time_series, window):
|
||||
# add your report of this error here...
|
||||
|
||||
return []
|
||||
|
||||
# spilt time_series
|
||||
split_time_series = utils.split_time_series(time_series, window)
|
||||
# nomalize time_series
|
||||
normalized_split_time_series = utils.normalize_time_series(split_time_series)
|
||||
s_features = statistical_features.get_statistical_features(normalized_split_time_series[4])
|
||||
f_features = fitting_features.get_fitting_features(normalized_split_time_series)
|
||||
c_features = classification_features.get_classification_features(normalized_split_time_series[0] + normalized_split_time_series[1][1:] + normalized_split_time_series[2] + normalized_split_time_series[3][1:] + normalized_split_time_series[4])
|
||||
# combine features with types
|
||||
features = s_features + f_features + c_features
|
||||
return features
|
||||
|
|
@ -0,0 +1,226 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
def time_series_moving_average(x):
|
||||
"""
|
||||
Returns the difference between the last element of x and the smoothed value after Moving Average Algorithm
|
||||
The Moving Average Algorithm is M_{n} = (x_{n-w+1}+...+x_{n})/w, where w is a parameter
|
||||
The parameter w is chosen in {1, 6, 11, 16, 21, 26, 31, 36, 41, 46} and the set of parameters can be changed.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/Moving_average
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: list with float
|
||||
"""
|
||||
temp_list = []
|
||||
for w in range(1, 50, 5):
|
||||
temp = np.mean(x[-w:])
|
||||
temp_list.append(temp)
|
||||
return list(np.array(temp_list) - x[-1])
|
||||
|
||||
|
||||
def time_series_weighted_moving_average(x):
|
||||
"""
|
||||
Returns the difference between the last element of x and the smoothed value after Weighted Moving Average Algorithm
|
||||
The Moving Average Algorithm is M_{n} = (1*x_{n-w+1}+...+(w-1)*x_{n-1}+w*x_{n})/w, where w is a parameter
|
||||
The parameter w is chosen in {1, 6, 11, 16, 21, 26, 31, 36, 41, 46} and the set of parameters can be changed.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/Moving_average
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: list with float
|
||||
"""
|
||||
temp_list = []
|
||||
for w in range(1, 50, 5):
|
||||
w = min(len(x), w) # avoid the case len(value_list) < w
|
||||
coefficient = np.array(range(1, w + 1))
|
||||
temp_list.append((np.dot(coefficient, x[-w:])) / (w * (w + 1) / 2))
|
||||
return list(np.array(temp_list) - x[-1])
|
||||
|
||||
|
||||
def time_series_exponential_weighted_moving_average(x):
|
||||
"""
|
||||
Returns the difference between the last element of x and the smoothed value after Exponential Moving Average Algorithm
|
||||
The Moving Average Algorithm is s[i] = alpha * x[i] + (1 - alpha) * s[i-1], where alpha is a parameter
|
||||
The parameter w is chosen in {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9} and the set of parameters can be changed.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/Exponential_smoothing
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: list with float
|
||||
"""
|
||||
temp_list = []
|
||||
for j in range(1, 10):
|
||||
alpha = j / 10.0
|
||||
s = [x[0]]
|
||||
for i in range(1, len(x)):
|
||||
temp = alpha * x[i] + (1 - alpha) * s[-1]
|
||||
s.append(temp)
|
||||
temp_list.append(s[-1] - x[-1])
|
||||
return temp_list
|
||||
|
||||
|
||||
def time_series_double_exponential_weighted_moving_average(x):
|
||||
"""
|
||||
Returns the difference between the last element of x and the smoothed value after Double Exponential Moving Average Algorithm
|
||||
The Moving Average Algorithm is s[i] = alpha * x[i] + (1 - alpha) * (s[i-1] + b[i-1]), b[i] = gamma * (s[i] - s[i-1]) + (1 - gamma) * b[i-1]
|
||||
where alpha and gamma are parameters.
|
||||
The parameter alpha is chosen in {0.1, 0.3, 0.5, 0.7, 0.9} and the set of parameters can be changed.
|
||||
The parameter gamma is chosen in {0.1, 0.3, 0.5, 0.7, 0.9} and the set of parameters can be changed.
|
||||
|
||||
WIKIPEDIA: https://en.wikipedia.org/wiki/Exponential_smoothing
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: list with float
|
||||
"""
|
||||
temp_list = []
|
||||
for j1 in range(1, 10, 2):
|
||||
for j2 in range(1, 10, 2):
|
||||
alpha = j1 / 10.0
|
||||
gamma = j2 / 10.0
|
||||
s = [x[0]]
|
||||
b = [(x[3] - x[0]) / 3] # s is the smoothing part, b is the trend part
|
||||
for i in range(1, len(x)):
|
||||
temp1 = alpha * x[i] + (1 - alpha) * (s[-1] + b[-1])
|
||||
s.append(temp1)
|
||||
temp2 = gamma * (s[-1] - s[-2]) + (1 - gamma) * b[-1]
|
||||
b.append(temp2)
|
||||
temp_list.append(s[-1] - x[-1])
|
||||
return temp_list
|
||||
|
||||
|
||||
def time_series_periodic_features(data_c_left, data_c_right, data_b_left, data_b_right, data_a):
|
||||
"""
|
||||
Returns the difference between the last element of data_a and the last element of data_b_left,
|
||||
the difference between the last element of data_a and the last element of data_c_left.
|
||||
|
||||
:param data_c_left: the time series of historical reference data
|
||||
:type data_c_left: pandas.Series
|
||||
:param data_c_right: the time series of historical reference data
|
||||
:type data_c_right: pandas.Series
|
||||
:param data_b_left: the time series of historical reference data
|
||||
:type data_b_left: pandas.Series
|
||||
:param data_b_right: the time series of historical reference data
|
||||
:type data_b_right: pandas.Series
|
||||
:param data_a: the time series to calculate the feature of
|
||||
:type data_a: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: list with float
|
||||
"""
|
||||
periodic_features = []
|
||||
|
||||
'''
|
||||
Add the absolute value of difference between today and a week ago and its sgn as two features
|
||||
Add the absolute value of difference between today and yesterday and its sgn as two features
|
||||
'''
|
||||
|
||||
temp_value = data_c_left[-1] - data_a[-1]
|
||||
periodic_features.append(abs(temp_value))
|
||||
if temp_value < 0:
|
||||
periodic_features.append(-1)
|
||||
else:
|
||||
periodic_features.append(1)
|
||||
|
||||
temp_value = data_b_left[-1] - data_a[-1]
|
||||
periodic_features.append(abs(temp_value))
|
||||
if temp_value < 0:
|
||||
periodic_features.append(-1)
|
||||
else:
|
||||
periodic_features.append(1)
|
||||
|
||||
'''
|
||||
If the last value of today is larger than the whole subsequence of a week ago,
|
||||
then return the difference between the maximum of the whole subsequence of a week ago and the last value of today.
|
||||
Others are similar.
|
||||
'''
|
||||
|
||||
periodic_features.append(min(max(data_c_left) - data_a[-1], 0))
|
||||
periodic_features.append(min(max(data_c_right) - data_a[-1], 0))
|
||||
periodic_features.append(min(max(data_b_left) - data_a[-1], 0))
|
||||
periodic_features.append(min(max(data_b_right) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_c_left) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_c_right) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_b_left) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_b_right) - data_a[-1], 0))
|
||||
|
||||
'''
|
||||
If the last value of today is larger than the subsequence of a week ago,
|
||||
then return the difference between the maximum of the whole subsequence of a week ago and the last value of today.
|
||||
Others are similar.
|
||||
'''
|
||||
|
||||
for w in range(1, 180, 30):
|
||||
periodic_features.append(min(max(data_c_left[-w:]) - data_a[-1], 0))
|
||||
periodic_features.append(min(max(data_c_right[:w]) - data_a[-1], 0))
|
||||
periodic_features.append(min(max(data_b_left[-w:]) - data_a[-1], 0))
|
||||
periodic_features.append(min(max(data_b_right[:w]) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_c_left[-w:]) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_c_right[:w]) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_b_left[-w:]) - data_a[-1], 0))
|
||||
periodic_features.append(max(min(data_b_right[:w]) - data_a[-1], 0))
|
||||
|
||||
'''
|
||||
Add the difference of mean values between two subsequences
|
||||
'''
|
||||
|
||||
for w in range(1, 180, 20):
|
||||
temp_value = np.mean(data_c_left[-w:]) - np.mean(data_a[-w:])
|
||||
periodic_features.append(abs(temp_value))
|
||||
if temp_value < 0:
|
||||
periodic_features.append(-1)
|
||||
else:
|
||||
periodic_features.append(1)
|
||||
|
||||
temp_value = np.mean(data_c_right[:w]) - np.mean(data_a[-w:])
|
||||
periodic_features.append(abs(temp_value))
|
||||
if temp_value < 0:
|
||||
periodic_features.append(-1)
|
||||
else:
|
||||
periodic_features.append(1)
|
||||
|
||||
temp_value = np.mean(data_b_left[-w:]) - np.mean(data_a[-w:])
|
||||
periodic_features.append(abs(temp_value))
|
||||
if temp_value < 0:
|
||||
periodic_features.append(-1)
|
||||
else:
|
||||
periodic_features.append(1)
|
||||
|
||||
temp_value = np.mean(data_b_right[:w]) - np.mean(data_a[-w:])
|
||||
periodic_features.append(abs(temp_value))
|
||||
if temp_value < 0:
|
||||
periodic_features.append(-1)
|
||||
else:
|
||||
periodic_features.append(1)
|
||||
return periodic_features
|
||||
|
||||
# add yourself fitting features here...
|
||||
|
||||
|
||||
def get_fitting_features(x_list):
|
||||
fitting_features = []
|
||||
fitting_features.extend(time_series_moving_average(x_list[4]))
|
||||
fitting_features.extend(time_series_weighted_moving_average(x_list[4]))
|
||||
fitting_features.extend(time_series_exponential_weighted_moving_average(x_list[4]))
|
||||
fitting_features.extend(time_series_double_exponential_weighted_moving_average(x_list[4]))
|
||||
fitting_features.extend(time_series_periodic_features(x_list[0], x_list[1], x_list[2], x_list[3], x_list[4]))
|
||||
# append yourself fitting features here...
|
||||
|
||||
return fitting_features
|
||||
|
|
@ -0,0 +1,446 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import tsfresh.feature_extraction.feature_calculators as ts_feature_calculators
|
||||
|
||||
|
||||
def time_series_maximum(x):
|
||||
"""
|
||||
Calculates the highest value of the time series x.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.maximum(x)
|
||||
|
||||
|
||||
def time_series_minimum(x):
|
||||
"""
|
||||
Calculates the lowest value of the time series x.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.minimum(x)
|
||||
|
||||
|
||||
def time_series_mean(x):
|
||||
"""
|
||||
Returns the mean of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.mean(x)
|
||||
|
||||
|
||||
def time_series_variance(x):
|
||||
"""
|
||||
Returns the variance of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.variance(x)
|
||||
|
||||
|
||||
def time_series_standard_deviation(x):
|
||||
"""
|
||||
Returns the standard deviation of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.standard_deviation(x)
|
||||
|
||||
|
||||
def time_series_skewness(x):
|
||||
"""
|
||||
Returns the sample skewness of x (calculated with the adjusted Fisher-Pearson standardized
|
||||
moment coefficient G1).
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.skewness(x)
|
||||
|
||||
|
||||
def time_series_kurtosis(x):
|
||||
"""
|
||||
Returns the kurtosis of x (calculated with the adjusted Fisher-Pearson standardized
|
||||
moment coefficient G2).
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.kurtosis(x)
|
||||
|
||||
|
||||
def time_series_median(x):
|
||||
"""
|
||||
Returns the median of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.median(x)
|
||||
|
||||
|
||||
def time_series_abs_energy(x):
|
||||
"""
|
||||
Returns the absolute energy of the time series which is the sum over the squared values
|
||||
|
||||
.. math::
|
||||
|
||||
E = \\sum_{i=1,\ldots, n} x_i^2
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.abs_energy(x)
|
||||
|
||||
|
||||
def time_series_absolute_sum_of_changes(x):
|
||||
"""
|
||||
Returns the sum over the absolute value of consecutive changes in the series x
|
||||
|
||||
.. math::
|
||||
|
||||
\\sum_{i=1, \ldots, n-1} \\mid x_{i+1}- x_i \\mid
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.absolute_sum_of_changes(x)
|
||||
|
||||
|
||||
def time_series_variance_larger_than_std(x):
|
||||
"""
|
||||
Boolean variable denoting if the variance of x is greater than its standard deviation. Is equal to variance of x
|
||||
being larger than 1
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: int
|
||||
"""
|
||||
return int(ts_feature_calculators.variance_larger_than_standard_deviation(x))
|
||||
|
||||
|
||||
def time_series_count_above_mean(x):
|
||||
"""
|
||||
Returns the number of values in x that are higher than the mean of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.count_above_mean(x)
|
||||
|
||||
|
||||
def time_series_count_below_mean(x):
|
||||
"""
|
||||
Returns the number of values in x that are lower than the mean of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.count_below_mean(x)
|
||||
|
||||
|
||||
def time_series_first_location_of_maximum(x):
|
||||
"""
|
||||
Returns the first location of the maximum value of x.
|
||||
The position is calculated relatively to the length of x.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.first_location_of_maximum(x)
|
||||
|
||||
|
||||
def time_series_first_location_of_minimum(x):
|
||||
"""
|
||||
Returns the first location of the minimal value of x.
|
||||
The position is calculated relatively to the length of x.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.first_location_of_minimum(x)
|
||||
|
||||
|
||||
def time_series_last_location_of_maximum(x):
|
||||
"""
|
||||
Returns the relative last location of the maximum value of x.
|
||||
The position is calculated relatively to the length of x.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.last_location_of_maximum(x)
|
||||
|
||||
|
||||
def time_series_last_location_of_minimum(x):
|
||||
"""
|
||||
Returns the last location of the minimal value of x.
|
||||
The position is calculated relatively to the length of x.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.last_location_of_minimum(x)
|
||||
|
||||
|
||||
def time_series_has_duplicate(x):
|
||||
"""
|
||||
Checks if any value in x occurs more than once
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: bool
|
||||
"""
|
||||
return ts_feature_calculators.has_duplicate(x)
|
||||
|
||||
|
||||
def time_series_has_duplicate_max(x):
|
||||
"""
|
||||
Checks if the maximum value of x is observed more than once
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: bool
|
||||
"""
|
||||
return ts_feature_calculators.has_duplicate_max(x)
|
||||
|
||||
|
||||
def time_series_has_duplicate_min(x):
|
||||
"""
|
||||
Checks if the minimal value of x is observed more than once
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: bool
|
||||
"""
|
||||
return ts_feature_calculators.has_duplicate_min(x)
|
||||
|
||||
|
||||
def time_series_longest_strike_above_mean(x):
|
||||
"""
|
||||
Returns the length of the longest consecutive subsequence in x that is bigger than the mean of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.longest_strike_above_mean(x)
|
||||
|
||||
|
||||
def time_series_longest_strike_below_mean(x):
|
||||
"""
|
||||
Returns the length of the longest consecutive subsequence in x that is smaller than the mean of x
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.longest_strike_below_mean(x)
|
||||
|
||||
|
||||
def time_series_mean_abs_change(x):
|
||||
"""
|
||||
Returns the mean over the absolute differences between subsequent time series values which is
|
||||
|
||||
.. math::
|
||||
|
||||
\\frac{1}{n} \\sum_{i=1,\ldots, n-1} | x_{i+1} - x_{i}|
|
||||
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.mean_abs_change(x)
|
||||
|
||||
|
||||
def time_series_mean_change(x):
|
||||
"""
|
||||
Returns the mean over the absolute differences between subsequent time series values which is
|
||||
|
||||
.. math::
|
||||
|
||||
\\frac{1}{n} \\sum_{i=1,\ldots, n-1} x_{i+1} - x_{i}
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.mean_change(x)
|
||||
|
||||
|
||||
def time_series_percentage_of_reoccurring_datapoints_to_all_datapoints(x):
|
||||
"""
|
||||
Returns the percentage of unique values, that are present in the time series
|
||||
more than once.
|
||||
|
||||
len(different values occurring more than once) / len(different values)
|
||||
|
||||
This means the percentage is normalized to the number of unique values,
|
||||
in contrast to the percentage_of_reoccurring_values_to_all_values.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.percentage_of_reoccurring_datapoints_to_all_datapoints(x)
|
||||
|
||||
|
||||
def time_series_ratio_value_number_to_time_series_length(x):
|
||||
"""
|
||||
Returns a factor which is 1 if all values in the time series occur only once,
|
||||
and below one if this is not the case.
|
||||
In principle, it just returns
|
||||
|
||||
# unique values / # values
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.ratio_value_number_to_time_series_length(x)
|
||||
|
||||
|
||||
def time_series_sum_of_reoccurring_data_points(x):
|
||||
"""
|
||||
Returns the sum of all data points, that are present in the time series
|
||||
more than once.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.sum_of_reoccurring_data_points(x)
|
||||
|
||||
|
||||
def time_series_sum_of_reoccurring_values(x):
|
||||
"""
|
||||
Returns the sum of all values, that are present in the time series
|
||||
more than once.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return ts_feature_calculators.sum_of_reoccurring_values(x)
|
||||
|
||||
|
||||
def time_series_sum_values(x):
|
||||
"""
|
||||
Calculates the sum over the time series values
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: bool
|
||||
"""
|
||||
return ts_feature_calculators.sum_values(x)
|
||||
|
||||
|
||||
def time_series_range(x):
|
||||
"""
|
||||
Calculates the range value of the time series x.
|
||||
|
||||
:param x: the time series to calculate the feature of
|
||||
:type x: pandas.Series
|
||||
:return: the value of this feature
|
||||
:return type: float
|
||||
"""
|
||||
return time_series_maximum(x) - time_series_minimum(x)
|
||||
|
||||
# add yourself statistical features here...
|
||||
|
||||
|
||||
def get_statistical_features(x):
|
||||
statistical_features = []
|
||||
statistical_features.append(time_series_maximum(x))
|
||||
statistical_features.append(time_series_minimum(x))
|
||||
statistical_features.append(time_series_mean(x))
|
||||
statistical_features.append(time_series_variance(x))
|
||||
statistical_features.append(time_series_standard_deviation(x))
|
||||
statistical_features.append(time_series_skewness(x))
|
||||
statistical_features.append(time_series_kurtosis(x))
|
||||
statistical_features.append(time_series_median(x))
|
||||
statistical_features.append(time_series_abs_energy(x))
|
||||
statistical_features.append(time_series_absolute_sum_of_changes(x))
|
||||
statistical_features.append(time_series_variance_larger_than_std(x))
|
||||
statistical_features.append(time_series_count_above_mean(x))
|
||||
statistical_features.append(time_series_count_below_mean(x))
|
||||
statistical_features.append(time_series_first_location_of_maximum(x))
|
||||
statistical_features.append(time_series_first_location_of_minimum(x))
|
||||
statistical_features.append(time_series_last_location_of_maximum(x))
|
||||
statistical_features.append(time_series_last_location_of_minimum(x))
|
||||
statistical_features.append(int(time_series_has_duplicate(x)))
|
||||
statistical_features.append(int(time_series_has_duplicate_max(x)))
|
||||
statistical_features.append(int(time_series_has_duplicate_min(x)))
|
||||
statistical_features.append(time_series_longest_strike_above_mean(x))
|
||||
statistical_features.append(time_series_longest_strike_below_mean(x))
|
||||
statistical_features.append(time_series_mean_abs_change(x))
|
||||
statistical_features.append(time_series_mean_change(x))
|
||||
statistical_features.append(time_series_percentage_of_reoccurring_datapoints_to_all_datapoints(x))
|
||||
statistical_features.append(time_series_ratio_value_number_to_time_series_length(x))
|
||||
statistical_features.append(time_series_sum_of_reoccurring_data_points(x))
|
||||
statistical_features.append(time_series_sum_of_reoccurring_values(x))
|
||||
statistical_features.append(time_series_sum_values(x))
|
||||
statistical_features.append(time_series_range(x))
|
||||
# append yourself statistical features here...
|
||||
|
||||
return statistical_features
|
||||
|
|
@ -0,0 +1,150 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import json
|
||||
import traceback
|
||||
import csv
|
||||
from app.dao.time_series_detector.sample_op import *
|
||||
from app.config.errorcode import *
|
||||
from app.utils.utils import *
|
||||
from app.config.common import *
|
||||
|
||||
|
||||
class SampleService(object):
|
||||
|
||||
def __init__(self):
|
||||
self.__sample = SampleOperation()
|
||||
uuid_str = uuid.uuid4().hex[:8]
|
||||
self.__upload_file_path = UPLOAD_PATH % uuid_str
|
||||
|
||||
def import_sample(self, data):
|
||||
try:
|
||||
ret_code, ret_data = self.__sample.import_sample(data)
|
||||
return_dict = build_ret_data(ret_code, {"count": ret_data})
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def import_file(self, file_data):
|
||||
try:
|
||||
pfile = file_data['sample_file']
|
||||
with open(self.__upload_file_path, 'wb+') as destination:
|
||||
for chunk in pfile.chunks():
|
||||
destination.write(chunk.replace('\x00', ''))
|
||||
data = []
|
||||
csv_reader = csv.reader(open(self.__upload_file_path))
|
||||
next(csv_reader)
|
||||
count = 0
|
||||
positive_count = 0
|
||||
negative_count = 0
|
||||
for row in csv_reader:
|
||||
one_item = {"viewName": row[0],
|
||||
"viewId": row[1],
|
||||
"attrName": row[2],
|
||||
"attrId": row[3],
|
||||
"source": row[4],
|
||||
"trainOrTest": row[5],
|
||||
"positiveOrNegative": row[6],
|
||||
"window": row[7],
|
||||
"dataC": row[8],
|
||||
"dataB": row[9],
|
||||
"dataA": row[10],
|
||||
"dataTime": int(row[11]),
|
||||
"updateTime": int(row[11]),
|
||||
"time": int(row[11]),
|
||||
"anomalyId": "0"}
|
||||
check_code, check_msg = check_value(one_item)
|
||||
if OP_SUCCESS != check_code:
|
||||
return build_ret_data(check_code, check_msg)
|
||||
data.append(one_item)
|
||||
if row[6] == "positive":
|
||||
positive_count = positive_count + 1
|
||||
elif row[6] == "negative":
|
||||
negative_count = negative_count + 1
|
||||
count = count + 1
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(FILE_FORMAT_ERR, str(ex))
|
||||
return return_dict
|
||||
|
||||
import_ret = self.import_sample(data)
|
||||
if OP_SUCCESS == import_ret['code']:
|
||||
ret_data = {"positiveCount": positive_count, "negativeCount": negative_count, "totalCount": count}
|
||||
import_ret["data"] = ret_data
|
||||
return import_ret
|
||||
|
||||
def update_sample(self, body):
|
||||
try:
|
||||
form = json.loads(body)
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
ret_code, ret_data = self.__sample.update_sample(form)
|
||||
return_dict = build_ret_data(ret_code, {"count": ret_data})
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def query_sample(self, body):
|
||||
try:
|
||||
form = json.loads(body)
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
ret_code, ret_data = self.__sample.query_sample(form)
|
||||
return_dict = build_ret_data(ret_code, ret_data)
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def sample_download(self, body):
|
||||
ret_data = ""
|
||||
try:
|
||||
if len(body) > VALUE_LEN_MAX:
|
||||
return ""
|
||||
ret_data = self.__sample.download_sample(body)
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
ret_data = build_ret_data(THROW_EXP, str(ex))
|
||||
return ret_data
|
||||
|
||||
def delete_sample(self, body):
|
||||
try:
|
||||
form = json.loads(body)
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
ret_code, ret_data = self.__sample.delete_sample(form)
|
||||
return_dict = build_ret_data(ret_code, {"count": ret_data})
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def count_sample(self, body):
|
||||
form = json.loads(body)
|
||||
try:
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
ret_code, ret_data = self.__sample.sample_count(form)
|
||||
return_dict = build_ret_data(ret_code, {"count": ret_data})
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def query_sample_source(self):
|
||||
try:
|
||||
ret_code, ret_data = self.__sample.query_sample_source()
|
||||
return_dict = build_ret_data(ret_code, ret_data)
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import json
|
||||
import traceback
|
||||
from app.dao.time_series_detector.train_op import *
|
||||
from app.config.errorcode import *
|
||||
from app.utils.utils import *
|
||||
|
||||
|
||||
class TrainService(object):
|
||||
|
||||
def __init__(self):
|
||||
self.__train_op = TrainOperation()
|
||||
|
||||
def query_train(self, body):
|
||||
try:
|
||||
form = json.loads(body)
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
ret_code, ret_data = self.__train_op.query_train(form)
|
||||
return_dict = build_ret_data(ret_code, ret_data)
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def query_train_source(self):
|
||||
try:
|
||||
ret_code, ret_data = self.__train_op.query_train_source()
|
||||
return_dict = build_ret_data(ret_code, ret_data)
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
|
||||
def delete_train(self, body):
|
||||
try:
|
||||
form = json.loads(body)
|
||||
ret_code, ret_data = check_value(form)
|
||||
if OP_SUCCESS == ret_code:
|
||||
ret_code, ret_data = self.__train_op.delete_train(form)
|
||||
return_dict = build_ret_data(ret_code, ret_data)
|
||||
except Exception, ex:
|
||||
traceback.print_exc()
|
||||
return_dict = build_ret_data(THROW_EXP, str(ex))
|
||||
return return_dict
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,25 @@
|
|||
SET FOREIGN_KEY_CHECKS=0;
|
||||
-- ----------------------------
|
||||
-- Table structure for `train_task`
|
||||
-- ----------------------------
|
||||
DROP TABLE IF EXISTS `train_task`;
|
||||
CREATE TABLE `train_task` (
|
||||
`id` int(11) NOT NULL AUTO_INCREMENT,
|
||||
`task_id` char(255) DEFAULT NULL,
|
||||
`sample_num` int(11) DEFAULT NULL,
|
||||
`postive_sample_num` int(11) DEFAULT NULL,
|
||||
`negative_sample_num` int(11) DEFAULT NULL,
|
||||
`window` int(2) DEFAULT NULL,
|
||||
`model_name` varchar(20) DEFAULT NULL,
|
||||
`source` varchar(255) DEFAULT NULL,
|
||||
`start_time` timestamp NULL DEFAULT NULL,
|
||||
`end_time` timestamp NULL DEFAULT NULL,
|
||||
`status` varchar(11) DEFAULT NULL,
|
||||
PRIMARY KEY (`id`),
|
||||
KEY `id` (`id`)
|
||||
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
|
||||
|
||||
-- ----------------------------
|
||||
-- Records of train_task
|
||||
-- ----------------------------
|
||||
INSERT INTO `train_task` VALUES ('1', '1535790960079', '90675', '45228', '45447', '180', 'xgb_default_model', 'Metis', '2018-09-01 16:36:00', '2018-09-01 16:45:40', 'complete');
|
||||
|
|
@ -0,0 +1 @@
|
|||
__all__ = ["utils"]
|
||||
|
|
@ -0,0 +1,161 @@
|
|||
#!/usr/bin/python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
from app.config.errorcode import *
|
||||
from app.config.common import *
|
||||
|
||||
|
||||
def is_standard_time_series(time_series, window=180):
|
||||
"""
|
||||
Check the length of time_series. If window = 180, then the length of time_series should be 903.
|
||||
The mean value of last window should be larger than 0.
|
||||
|
||||
:param time_series: the time series to check, like [data_c, data_b, data_a]
|
||||
:type time_series: pandas.Series
|
||||
:param window: the length of window
|
||||
:return: True or False
|
||||
:return type: boolean
|
||||
"""
|
||||
if len(time_series) == 5 * window + 3 and np.mean(time_series[(4 * window + 2):]) > 0:
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
|
||||
def split_time_series(time_series, window=180):
|
||||
"""
|
||||
Spilt the time_series into five parts. Each has a length of window + 1
|
||||
|
||||
:param time_series: [data_c, data_b, data_a]
|
||||
:param window: the length of window
|
||||
:return: spilt list [[data_c_left], [data_c_right], [data_b_left], [data_b_right], [data_a]]
|
||||
"""
|
||||
data_c_left = time_series[0:(window + 1)]
|
||||
data_c_right = time_series[window:(2 * window + 1)]
|
||||
data_b_left = time_series[(2 * window + 1):(3 * window + 2)]
|
||||
data_b_right = time_series[(3 * window + 1):(4 * window + 2)]
|
||||
data_a = time_series[(4 * window + 2):]
|
||||
split_time_series = []
|
||||
split_time_series.append(data_c_left)
|
||||
split_time_series.append(data_c_right)
|
||||
split_time_series.append(data_b_left)
|
||||
split_time_series.append(data_b_right)
|
||||
split_time_series.append(data_a)
|
||||
return split_time_series
|
||||
|
||||
|
||||
def normalize_time_series(split_time_series):
|
||||
"""
|
||||
Normalize the split_time_series.
|
||||
|
||||
:param split_time_series: [[data_c_left], [data_c_right], [data_b_left], [data_b_right], [data_a]]
|
||||
:return: all list / mean(split_time_series)
|
||||
"""
|
||||
value = np.mean(split_time_series[4])
|
||||
if value > 1:
|
||||
normalized_data_c_left = list(split_time_series[0] / value)
|
||||
normalized_data_c_right = list(split_time_series[1] / value)
|
||||
normalized_data_b_left = list(split_time_series[2] / value)
|
||||
normalized_data_b_right = list(split_time_series[3] / value)
|
||||
normalized_data_a = list(split_time_series[4] / value)
|
||||
else:
|
||||
normalized_data_c_left = split_time_series[0]
|
||||
normalized_data_c_right = split_time_series[1]
|
||||
normalized_data_b_left = split_time_series[2]
|
||||
normalized_data_b_right = split_time_series[3]
|
||||
normalized_data_a = split_time_series[4]
|
||||
normalized_split_time_series = []
|
||||
normalized_split_time_series.append(normalized_data_c_left)
|
||||
normalized_split_time_series.append(normalized_data_c_right)
|
||||
normalized_split_time_series.append(normalized_data_b_left)
|
||||
normalized_split_time_series.append(normalized_data_b_right)
|
||||
normalized_split_time_series.append(normalized_data_a)
|
||||
return normalized_split_time_series
|
||||
|
||||
|
||||
def build_ret_data(ret_code, data=""):
|
||||
return {"code": ret_code, "msg": ERR_CODE[ret_code], "data": data}
|
||||
|
||||
|
||||
def validate_value(data):
|
||||
if isinstance(data, unicode):
|
||||
if len(data) > INPUT_LEN_ENG_MAX:
|
||||
return CHECK_PARAM_FAILED
|
||||
elif isinstance(data, str):
|
||||
if len(data) > INPUT_LEN_ENG_MAX:
|
||||
return CHECK_PARAM_FAILED
|
||||
elif isinstance(data, list):
|
||||
if len(data) > INPUT_LIST_LEN_MAX:
|
||||
return CHECK_PARAM_FAILED
|
||||
for item in data:
|
||||
ret_code = validate_value(item)
|
||||
if ret_code != 0:
|
||||
return ret_code
|
||||
return 0
|
||||
|
||||
|
||||
def check_value(data):
|
||||
if 'attrId' in data:
|
||||
ret_code = validate_value(data['attrId'])
|
||||
if ret_code != 0:
|
||||
return CHECK_PARAM_FAILED, "attrId too long"
|
||||
if 'attrName' in data:
|
||||
ret_code = validate_value(data['attrName'])
|
||||
if ret_code != 0:
|
||||
return CHECK_PARAM_FAILED, "attrName too long"
|
||||
if 'viewId' in data:
|
||||
ret_code = validate_value(data['viewId'])
|
||||
if ret_code != 0:
|
||||
return CHECK_PARAM_FAILED, "viewId too long"
|
||||
if 'viewName' in data:
|
||||
ret_code = validate_value(data['viewName'])
|
||||
if ret_code != 0:
|
||||
return CHECK_PARAM_FAILED, "viewName too long"
|
||||
if 'itemPerPage' in data:
|
||||
if data['itemPerPage'] > INPUT_ITEM_PER_PAGE_MAX:
|
||||
return CHECK_PARAM_FAILED, "itemPerPage too big"
|
||||
if 'beginTime' in data:
|
||||
if len(str(data['beginTime'])) > INPUT_LEN_ENG_MAX:
|
||||
return CHECK_PARAM_FAILED, "beginTime too long"
|
||||
if 'endTime' in data:
|
||||
if len(str(data['endTime'])) > INPUT_LEN_ENG_MAX:
|
||||
return CHECK_PARAM_FAILED, "endTime too long"
|
||||
if 'updateTime' in data:
|
||||
if len(str(data['updateTime'])) > INPUT_LEN_ENG_MAX:
|
||||
return CHECK_PARAM_FAILED, "updateTime too long"
|
||||
if 'source' in data:
|
||||
ret_code = validate_value(data['source'])
|
||||
if ret_code != 0:
|
||||
return CHECK_PARAM_FAILED, "source too long"
|
||||
if 'trainOrTest' in data:
|
||||
ret_code = validate_value(data['source'])
|
||||
if ret_code != 0:
|
||||
return CHECK_PARAM_FAILED, "trainOrTest too long"
|
||||
if 'positiveOrNegative' in data:
|
||||
ret_code = validate_value(data['positiveOrNegative'])
|
||||
if ret_code != 0:
|
||||
return CHECK_PARAM_FAILED, "positiveOrNegative too long"
|
||||
if 'window' in data:
|
||||
if len(str(data['window'])) > INPUT_LEN_ENG_MAX:
|
||||
return CHECK_PARAM_FAILED, "window"
|
||||
if 'dataTime' in data:
|
||||
if len(str(data['dataTime'])) > INPUT_LEN_ENG_MAX:
|
||||
return CHECK_PARAM_FAILED, "dataTime too long"
|
||||
if 'dataC' in data:
|
||||
if len(str(data['dataC'])) > VALUE_LEN_MAX:
|
||||
return CHECK_PARAM_FAILED, "dataC too long"
|
||||
if 'dataB' in data:
|
||||
if len(str(data['dataB'])) > VALUE_LEN_MAX:
|
||||
return CHECK_PARAM_FAILED, "dataB too long"
|
||||
if 'dataA' in data:
|
||||
if len(str(data['dataA'])) > VALUE_LEN_MAX:
|
||||
return CHECK_PARAM_FAILED, "dataA too long"
|
||||
return 0, ""
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#!/usr/bin/env bash
|
||||
# Run tests
|
||||
|
||||
# echo path
|
||||
echo $DIR, `pwd`
|
||||
echo "hello world"
|
||||
|
||||
py.test -x -vv -s `pwd`/tests/
|
||||
sh `pwd`/ci/init_mysql_data.sh
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
FROM sherlockren/metis-full
|
||||
WORKDIR /metis
|
||||
ADD . /metis
|
||||
EXPOSE 80
|
||||
EXPOSE 8080
|
||||
|
||||
RUN chmod +x /metis/init.sh
|
||||
|
||||
CMD ["/bin/sh","/metis/init.sh"]
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
/usr/sbin/nginx
|
||||
nohup /usr/bin/mysqld &
|
||||
|
||||
sleep 3
|
||||
|
||||
export PYTHONPATH=/metis
|
||||
|
||||
sed -i 's/127.0.0.1/9.9.9.9/g' /metis/uweb/dist/app.js
|
||||
sed -i 's/127.0.0.1/9.9.9.9/g' /metis/uweb/dist/_sample_sampleinfo.js
|
||||
|
||||
sleep 3
|
||||
|
||||
nohup python /metis/app/controller/manage.py runserver 0.0.0.0:38324 &
|
||||
/bin/sh
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
if [ $# == 0 ]
|
||||
then
|
||||
echo "Need at least 1 argument!"
|
||||
exit
|
||||
fi
|
||||
ip=$1
|
||||
sed -i "s/9.9.9.9/${ip}/g" init.sh
|
||||
docker build -t local/metis-demo:1.0 .
|
||||
docker run -i -t -p80:80 -p8080:8080 local/metis-demo:1.0
|
||||
|
|
@ -0,0 +1,127 @@
|
|||
## API接口文档
|
||||
|
||||
用户可使用API接口对时间序列进行异常检测，检测后的结果通过WEB管理端查看和管理。 服务端提供两个异常检测的API接口供不同场景调用：
|
||||
|
||||
1、量值检测：适用于大多数类型数据的检测，使用无监督和有监督联合检测，会加载检测模型
|
||||
|
||||
2、率值检测：适用于严格正态分布类型数据的检测，使用无监督算法进行检测，如成功率等生死指标数据的检测
|
||||
|
||||
- API请求调用请使用搭建的后端服务地址
|
||||
- 当前检测时间窗口选取为3小时，每分钟1个数据点，即窗口值为180
|
||||
- 同比数据日期和时间段的选择可根据实际情况调整，文档中两个同比数据分别去昨日和一周前的同比
|
||||
|
||||
针对当前一个值的检测，需要依赖过去三段数据，数据选取规则参考示例图：
|
||||
|
||||
|
||||
### 量值检测
|
||||
|
||||
* API： POST /{ip}:{port}/PredictValue
|
||||
* 功能说明：根据参考数据检测最近一个数据点是否异常
|
||||
* 请求参数request：
|
||||
|
||||
```
|
||||
{
|
||||
"viewId":"2012",
|
||||
"viewName":"登陆功能",
|
||||
"attrId":"19201",
|
||||
"attrName":"ptlogin登陆请求总量",
|
||||
"taskId":"1530608070706",
|
||||
"window":180,
|
||||
"dataC":"9,10,152,...,255,...,16",
|
||||
"dataB":"9,10,152,...,255,...,18",
|
||||
"dataA":"9,10,152,...,458"
|
||||
}
|
||||
```
|
||||
|
||||
* request字段说明：
|
||||
|
||||
| 名称 | 类型 |必填| 默认值 | 说明 |
|
||||
| --- | --- | --- |--- | --- | ---|
|
||||
| viewId| string| 是|无|指标集ID |
|
||||
| viewName| string| 是| 无|指标集名称|
|
||||
| attrId| string| 是| 无|指标ID|
|
||||
| attrName| string| 是| 无|指标名称|
|
||||
| taskId| string| 否| 无|使用的检测模型，如不传，则采用系统默认模型|
|
||||
| window| int| 是| 无|窗口值，目前支持180|
|
||||
| dataC| string| 是| 无|待检测的1个点对应一周前同时刻的点 + 前后小时的数据，361个数据点按时间顺序拼接，英文逗号分隔|
|
||||
| dataB| string| 是| 无|待检测的1个点对应昨日同时刻的点 + 前后三小时的数据，361个数据点按时间顺序拼接，英文逗号分隔|
|
||||
| dataA| string| 是| 无|待检测的1个点+前三小时的数据，共181个数据点，181个数据点按时间顺序拼接，英文逗号分隔|
|
||||
|
||||
|
||||
* 详情参数response：
|
||||
```
|
||||
{
|
||||
"code":0,
|
||||
"msg":"操作成功",
|
||||
"data":
|
||||
{
|
||||
"ret":0,
|
||||
"p":"0.05",
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
* response 字段说明：
|
||||
|
||||
| 名称 | 类型 | 说明 |
|
||||
|---|---|---|---|
|
||||
| code | int | 返回码。0:成功；非0:失败 |
|
||||
| msg | string | 返回消息 |
|
||||
| ret | int | 检测结果是否异常。0:异常；1:正常 |
|
||||
| p | string | 概率值，值越小，置信度越高，目前p<0.15，判决为异常 |
|
||||
|
||||
### 率值检测
|
||||
|
||||
* API： POST /{ip}:{port}/PredictRate
|
||||
* 功能说明：根据参考数据检测最近一个数据点是否异常
|
||||
* 请求参数request：
|
||||
|
||||
```
|
||||
{
|
||||
"viewId":"2012",
|
||||
"viewName":"登陆功能",
|
||||
"attrId":"19201",
|
||||
"attrName":"ptlogin登陆成功率",
|
||||
"window":180,
|
||||
"dataC":"100,99.8,100,...,100,...,100",
|
||||
"dataB":"99.5,100,100,...,99.6,...,100",
|
||||
"dataA":"100,98.5,100,...,85.9"
|
||||
}
|
||||
```
|
||||
|
||||
* request字段说明：
|
||||
|
||||
| 名称 | 类型 |必填| 默认值 | 说明 |
|
||||
| --- | --- | --- |--- | --- | ---|
|
||||
| viewId| string| 是|无|指标集ID |
|
||||
| viewName| string| 是| 无|指标集名称|
|
||||
| attrId| string| 是| 无|指标ID|
|
||||
| attrName| string| 是| 无|指标名称|
|
||||
| window| int| 是| 无|窗口值，目前支持180|
|
||||
| dataC| string| 是| 无|待检测的1个点对应一周前同时刻的点 + 前后小时的数据，361个数据点按时间顺序拼接，英文逗号分隔|
|
||||
| dataB| string| 是| 无|待检测的1个点对应昨日同时刻的点 + 前后三小时的数据，361个数据点按时间顺序拼接，英文逗号分隔|
|
||||
| dataA| string| 是| 无|待检测的1个点+前三小时的数据，共181个数据点，181个数据点按时间顺序拼接，英文逗号分隔|
|
||||
|
||||
|
||||
* 详情参数response：
|
||||
|
||||
```
|
||||
{
|
||||
"code":0,
|
||||
"msg":"操作成功",
|
||||
"data":
|
||||
{
|
||||
"ret":0,
|
||||
"p":"0",
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
* response 字段说明：
|
||||
|
||||
| 名称 | 类型 | 说明 |
|
||||
|---|---|---|---|
|
||||
| code | int | 返回码。0:成功；非0:失败 |
|
||||
| msg | string | 返回消息 |
|
||||
| ret | int | 检测结果是否异常。0:异常；1:正常 |
|
||||
| p | string | 概率值，值越小，置信度越高 |
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
## 时间序列异常检测学件的架构
|
||||
|
||||
|
||||
|
||||
时间序列异常检测学件的整体分层涉及，可以分为以下四层：
|
||||
|
||||
1. **数据层（DB）**：存储检测异常信息、样本信息、任务信息等
|
||||
|
||||
2. **服务层(server)**： 服务层划分为四大模块
|
||||
|
||||
1. **数据驱动模块DAO**： 封装了和DB层常见的数据操作接口。
|
||||
|
||||
2. **特征计算模块feature**： 提供三类时间序列的特征（统计特征、拟合特征、分类特征）用于对时序数据进行特征提取，在监督学习和训练中使用。
|
||||
|
||||
3. **算法模块feature**： 提供常见的几种机器学习算法封装（统计判别算法、指数移动平均算法、多项式算法、GBDT和xgboost等）用于对序数据进行联合仲裁检测。
|
||||
|
||||
4. **业务模块business**： 业务模块是基于原子接口封装，完成API层的具体业务逻辑。
|
||||
|
||||
3. **接口层(api)**： 提供API能力，时间序列异常检测接口和WEB管理的操作接口。
|
||||
|
||||
4. **WEB层(web)**： 系统提供的WEB服务，通过服务界面，用户可以进行异常查询、打标标注、样本库管理、模型训练等操作。
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
## 项目目录结构
|
||||
|
||||
项目开发的目录结构保持一致，容易理解并方便管理。
|
||||
|
||||
## 目录结构
|
||||
|
||||
- `/app/` 服务端总工作目录
|
||||
|
||||
`/app/controller/` 路由入口Action层
|
||||
|
||||
`/app/config/` 业务配置层
|
||||
|
||||
`/app/dao/` 数据库表实例层
|
||||
|
||||
`/app/model/` 模型文件存放目录
|
||||
|
||||
`/app/service/` 业务逻辑层
|
||||
|
||||
`/app/service/algorithm/` 算法层
|
||||
|
||||
`/app/service/feature/` 特征层
|
||||
|
||||
`/app/utils/` 存放公共函数
|
||||
|
||||
- `/uweb/` 管理端总工作目录
|
||||
|
||||
`/uweb/custom/` WEB端所需静态文件目录
|
||||
|
||||
`/uweb/lib/` WEB端框架目录
|
||||
|
||||
`/uweb/src/` WEB端开发目录
|
||||
|
||||
`/uweb/src/pages/` WEB端所有页面的目录
|
||||
|
||||
`/uweb/src/plugins/` WEB端自定义插件目录
|
||||
|
||||
`/uweb/src/app.json` WEB端配置文件
|
||||
|
||||
`/uweb/src/app.less` WEB端全局样式文件
|
||||
|
||||
`/uweb/dist/` WEB端打包后的静态文件目录
|
||||
|
||||
项目中支持以下类型的文件：
|
||||
1. `.json`: 配置文件
|
||||
2. `.uwx`: UWEB 视图文件
|
||||
3. `.uw`: UWEB 逻辑脚本
|
||||
4. `.js`: 普通 JavaScript 逻辑脚本
|
||||
5. `.ts`: 普通 TypeScript 逻辑脚本
|
||||
6. `.less`: Less 样式文件
|
||||
7. `.css`: CSS 样式文件
|
||||
8. `.jsx`: 开发自定义插件时可使用的 JavaScript React 脚本文件
|
||||
9. `.tsx`: 开发自定义插件时可使用的 TypeScript React 脚本文件
|
||||
10. `.png`、`.jpg`、`.gif`、`.svg`: 图片文件
|
||||
|
||||
- `/docs/` 项目文档存放目录
|
||||
|
||||
|
||||
## 调用关系
|
||||
|
||||
`/app/controller/` 为服务端路由入口，可调用service业务层
|
||||
|
||||
`/app/service/` 为service业务层，可调用私有对象dao数据库层
|
||||
|
||||
`/app/model/` 模型文件存放目录，供service业务层加载
|
||||
|
||||
`/app/utils/` 公共函数层全局可调用
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 5.9 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 12 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 138 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 6.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 58 KiB |
|
|
@ -0,0 +1,192 @@
|
|||
## 安装文档
|
||||
|
||||
# 目录
|
||||
> * [手工安装](#chapter-1)
|
||||
>> * [依赖环境](#chapter-1-1)
|
||||
>> * [数据库环境安装](#chapter-1-2)
|
||||
>> * [服务端环境安装](#chapter-1-3)
|
||||
>> * [WEB管理端环境安装](#chapter-1-4)
|
||||
>
|
||||
> * [docker方式部署](#chapter-2)
|
||||
>
|
||||
本安装文档仅描述了在一台服务器上安装搭建整个Metis的过程，目的是为了让用户对Metis的部署搭建、运行等整体认识。
|
||||
|
||||
如要用于生产环境，需要更多考虑分布式系统下容错、容灾能力。若有需要，可以加入Metis的qq技术交流群：288723616。
|
||||
|
||||
# 1. <a id="chapter-1"></a>手工安装
|
||||
## 1.1. <a id="chapter-1-1"></a>依赖环境
|
||||
|
||||
| 软件 | 软件要求 |
|
||||
| --- | --- |
|
||||
| linux内核版本:| CentOS 7.4 |
|
||||
| python版本:| 2.7版本|
|
||||
| mysql版本:| 5.6.26及以上版本|
|
||||
| Node.js版本:| 8.11.1及以上版本|s
|
||||
| Django版本:| 1.11.13及以上版本|
|
||||
|
||||
运行服务器要求：1台普通安装linux系统的机器即可（推荐CentOS系统）。2.服务器需要开放80和8080端口
|
||||
|
||||
以下步骤假定安装机器的代码目录是 `/data/Metis/`，可根据实际情况更改。
|
||||
|
||||
## 1.2. <a id="chapter-1-2"></a>数据库环境安装
|
||||
|
||||
### 1.2.1. mysql 安装介绍
|
||||
|
||||
采用yum源安装或者在mysql官网下载源码安装，安装好后检测mysql服务是否正常工作。
|
||||
|
||||
```
|
||||
yum install mariadb-server
|
||||
systemctl start mariadb
|
||||
```
|
||||
|
||||
### 1.2.2. 初始化数据库
|
||||
|
||||
为了方便用户快速使用，提供了50+异常检测结果数据和300+样本数据供大家使用。
|
||||
|
||||
1、创建需要的数据库用户名并授权，连接mysql客户端并执行
|
||||
|
||||
```
|
||||
grant all privileges on *.* to metis@127.0.0.1 identified by 'metis@123';
|
||||
flush privileges;
|
||||
```
|
||||
|
||||
2、创建数据库 `metis`，在命令行下执行
|
||||
|
||||
```
|
||||
mysqladmin -umetis -pmetis@123 -h127.0.0.1 create metis
|
||||
```
|
||||
|
||||
3、将`/Metis/app/sql/`目录下的sql初始化文件，导入数据`metis`数据库
|
||||
|
||||
```
|
||||
mysql -umetis -pmetis@123 -h127.0.0.1 metis < /data/Metis/app/sql/time_series_detector/anomaly.sql
|
||||
mysql -umetis -pmetis@123 -h127.0.0.1 metis < /data/Metis/app/sql/time_series_detector/sample_dataset.sql
|
||||
mysql -umetis -pmetis@123 -h127.0.0.1 metis < /data/Metis/app/sql/time_series_detector/train_task.sql
|
||||
```
|
||||
|
||||
4、将数据库配置信息更新到服务端配置文件`database.py`
|
||||
```
|
||||
vim /data/Metis/app/config/database.py
|
||||
```
|
||||
改写配置
|
||||
```
|
||||
db = 'metis'
|
||||
user = 'metis'
|
||||
passwd = 'metis@123'
|
||||
host = '127.0.0.1'
|
||||
port = 3306
|
||||
```
|
||||
|
||||
## 1.3. <a id="chapter-1-3"></a>服务端环境安装
|
||||
|
||||
服务端python程序需要依赖django、numpy、tsfresh、MySQL-python、scikit-learn、scikit-learn等包
|
||||
|
||||
### 1.3.1. yum 安装依赖包
|
||||
|
||||
```
|
||||
yum install python-pip
|
||||
pip install --upgrade pip
|
||||
yum install gcc libffi-devel python-devel openssl-devel
|
||||
yum install mysql-devel
|
||||
```
|
||||
|
||||
### 1.3.2. pip 安装python依赖包
|
||||
|
||||
通过工程目录下docs/requirements.txt安装
|
||||
|
||||
```
|
||||
pip install -I -r requirements.txt
|
||||
```
|
||||
|
||||
### 1.3.3. 工作目录加入环境变量
|
||||
|
||||
```
|
||||
export PYTHONPATH=/data/Metis:$PYTHONPATH
|
||||
```
|
||||
|
||||
为了保证下次登陆可以导入环境变量,请将环境变量配置写入/etc/profile文件
|
||||
|
||||
### 1.3.4. 部署Django服务端
|
||||
|
||||
部署生产环境时可通过nginx和uwsgi部署，具体请参考对应官网说明
|
||||
|
||||
### 1.3.5. 启动服务端
|
||||
|
||||
启动服务端程序
|
||||
|
||||
```
|
||||
python /data/Metis/app/controller/manage.py runserver {ip}:{port}
|
||||
```
|
||||
|
||||
## 1.4. <a id="chapter-1-4"></a>WEB管理端环境安装
|
||||
|
||||
### 1.4.1. Node.js安装
|
||||
|
||||
需先安装[Node.js](https://nodejs.org/en/download/)，并且Node.js的版本需不低于 8.11.1
|
||||
|
||||
### 1.4.2. npm install安装前端依赖
|
||||
|
||||
安装 uweb/pacakge.json 配置文件中依赖的第三方安装包
|
||||
|
||||
进入要uweb目录，执行npm install
|
||||
|
||||
### 1.4.3. 编译代码
|
||||
|
||||
修改uweb/src/app.json 文件的后端地址配置: "origin": "http://${ip}:${port}" , ip和port对应服务端地址
|
||||
|
||||
运行npm run build
|
||||
|
||||
将uweb目录下的custom文件夹下复制到uweb目录下生成的dist文件夹中
|
||||
|
||||
将nginx配置文件中的root定位到uweb目录下的dist文件夹
|
||||
|
||||
nginx配置如下：
|
||||
|
||||
```
|
||||
server {
|
||||
listen 80;
|
||||
root /*/uweb/dist;
|
||||
location / {
|
||||
add_header Cache-Control max-age=0;
|
||||
gzip on;
|
||||
gzip_min_length 1k;
|
||||
gzip_buffers 16 64k;
|
||||
gzip_http_version 1.1;
|
||||
gzip_comp_level 6;
|
||||
gzip_types text/plain application/x-javascript text/css application/xml;
|
||||
gzip_vary on;
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
||||
location /index.html {
|
||||
add_header Cache-Control 'no-store';
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 1.4.4. 启动WEB服务
|
||||
|
||||
nginx正常启动后，打开浏览器并访问 http://${ip}:80/
|
||||
|
||||
### 1.4.5. 本地修改调试
|
||||
|
||||
如本地修改代码，发布更新方式如下：
|
||||
|
||||
npm run build 项目代码开发完成后，执行该命令打包项目代码。在项目根目录会生成一个 dist 目录，然后复制custom目录，放至dist目录下。发布时，将 dist 目录中的全部文件作为静态文件，放至服务器指定的静态文件目录即可
|
||||
|
||||
# 2. <a id="chapter-5"></a>docker方式部署
|
||||
|
||||
## 2.1. 安装docker
|
||||
|
||||
```
|
||||
yum install docker
|
||||
service docker start
|
||||
```
|
||||
|
||||
## 2.2. <a id="chapter-2"></a> 部署docker环境
|
||||
执行Meits/docker/start.sh ${本机ip},等待部署完成
|
||||
|
||||
部署完成后,可以通过浏览器直接访问:http://${IP}
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
Django==1.10.3
|
||||
MySQL-python==1.2.5
|
||||
numpy==1.15.2
|
||||
scikit-learn==0.20.0
|
||||
tsfresh==0.11.1
|
||||
xgboost==0.80
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
|
||||
## 使用场景
|
||||
|
||||
时间序列异常检测学件经过海量监控数据打磨，在异常检测和运维监控领域具有广泛的应用性。
|
||||
|
||||
### 功能特性
|
||||
|
||||
时间序列异常检测学件：
|
||||
|
||||
* 异常检测：时间序列的异常检测
|
||||
* 特征提取：提供时间序列统计、拟合、分类特征的提取功能
|
||||
* 异常查询：可对检测到的异常视图进行管理
|
||||
* 标注打标：可进行异常的标注和取消标注，标注结果为正负样本
|
||||
* 样本管理：提供样本库的管理能力，查询、编辑、上传、导出、删除
|
||||
* 模型管理：提供训练模型功能和训练任务管理
|
||||
|
||||
### 应用数据场景
|
||||
|
||||
* 操作系统数据：适用于检测操作系统层面的基础监控数据，例如CPU、内存、磁盘、流量、包量等。
|
||||
* 应用程序数据：适用于检测应用程序运行中记录的时序数据，例如读写量、调用量、自定义监控指标等。
|
||||
* KPI指标数据：适用于检测业务KPI数据，例如交易量、收入值、在线数、成功率、失败量等业务关键指标。
|
||||
|
||||
### 应用案例场景
|
||||
|
||||
* 监控告警：可取代传统阈值监控方式，智能检测时序数据的异常。
|
||||
* 关联分析：可通过分析检测结果的关联性和根源性，实现异常精准定位。
|
||||
* 影响评估：可检测出大范围故障或网络波动，在故障发生时评估出故障影响。
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
## WEB管理端使用指南
|
||||
|
||||
介绍WEB管理端的使用说明，主要包含异常查询、标注打标、样本库管理、训练模型等功能的使用说明
|
||||
|
||||
## 异常查询
|
||||
|
||||
入口：`异常视图-异常查询` 可查询检测结果为异常的结果信息
|
||||
|
||||
1、支持时间、指标集、指标维度联合查询
|
||||
|
||||
|
||||
|
||||
2、支持曲线交互：查看大图、放缩、曲线选择等
|
||||
|
||||
|
||||
|
||||
|
||||
## 标注打标
|
||||
|
||||
入口：`异常视图-异常查询` 可对检测结果进行标注，标注后的数据以样本形式转存入样本库
|
||||
|
||||
1、支持样本标注，标记为正样本或负样本
|
||||
|
||||
|
||||
|
||||
2、支持取消标注
|
||||
|
||||
|
||||
|
||||
## 样本库
|
||||
|
||||
入口：`样本库-样本管理` 可对样本数据进行增删改查等基本操作
|
||||
|
||||
1、支持时间、样本来源、窗口、分类集等维度联合查询
|
||||
|
||||
|
||||
|
||||
2、支持单样本查看、编辑、删除操作
|
||||
|
||||
3、支持批量编辑、导出操作
|
||||
|
||||
|
||||
|
||||
4、支持导入样本操作
|
||||
|
||||
|
||||
|
||||
## 训练模型
|
||||
|
||||
入口：`样本库-训练模型` 可根据样本数据进行检测模型的训练，训练的模型可用于异常检测
|
||||
|
||||
1、支持时间、样本来源、任务状态等维度联合查询
|
||||
|
||||
|
||||
|
||||
2、支持删除任务记录
|
||||
|
||||
3、支持新建训练任务
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
test:
|
||||
pwd
|
||||
ls
|
||||
export PYTHONPATH=$PYTHONPATH:`pwd`/app
|
||||
sh `pwd`/ci/run_tests.sh
|
||||
#pylint --rcfile=`pwd`/pylint.conf ./app/dao
|
||||
#pylint --rcfile=`pwd`/pylint.conf ./app/service
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
{
|
||||
"name": "metis",
|
||||
"version": "1.0.0",
|
||||
"description": "",
|
||||
"main": "index.js",
|
||||
"directories": {
|
||||
"test": "tests"
|
||||
},
|
||||
"scripts": {
|
||||
"test": "echo \"Error: no test specified\" && exit 1",
|
||||
"commitmsg": "commitlint -e $GIT_PARAMS"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/lxd1190/metis_test.git"
|
||||
},
|
||||
"author": "",
|
||||
"license": "ISC",
|
||||
"bugs": {
|
||||
"url": "https://github.com/lxd1190/metis_test/issues"
|
||||
},
|
||||
"homepage": "https://github.com/lxd1190/metis_test#readme",
|
||||
"devDependencies": {
|
||||
"husky": "^0.14.3",
|
||||
"@commitlint/travis-cli": "^6.2.0",
|
||||
"@commitlint/config-conventional": "^6.1.3",
|
||||
"@commitlint/cli": "^6.2.0"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
from unittest import TestCase
|
||||
import random
|
||||
|
||||
class DataTestCase(TestCase):
|
||||
def create_test_data_a(self):
|
||||
return [850600,889768,883237,896313,870407,868385,865300,889802,894983,836835,937571,904475,892846,878769,886624,892638,894804,889133,908860,
|
||||
904439,896944,910079,932156,927790,936513,944358,922693,905639,929855,824757,1020900,918838,966000,936090,921495,988048,963848,959618,
|
||||
948817,963953,955761,964989,980420,927674,962113,956436,967907,975038,946675,875024]
|
||||
|
||||
def generate_random_str(self, randomlength=16):
|
||||
random_str = ''
|
||||
base_str = 'ABCDEFGHIGKLMNOPQRSTUVWXYZabcdefghigklmnopqrstuvwxyz0123456789'
|
||||
length = len(base_str) - 1
|
||||
for i in range(randomlength):
|
||||
random_str += base_str[random.randint(0, length)]
|
||||
return random_str
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
class TestClass:
|
||||
|
||||
def test_one(self):
|
||||
x = "this"
|
||||
assert 'h' in x
|
||||
|
||||
def test_two(self):
|
||||
x = "hello"
|
||||
assert 'hello' in x
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
# content of test_sample.py
|
||||
def func(x):
|
||||
return x + 1
|
||||
|
||||
|
||||
def test_answer():
|
||||
assert func(4) == 5
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: UTF-8 -*-
|
||||
"""
|
||||
Tencent is pleased to support the open source community by making Metis available.
|
||||
Copyright (C) 2018 THL A29 Limited, a Tencent company. All rights reserved.
|
||||
Licensed under the BSD 3-Clause License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
||||
https://opensource.org/licenses/BSD-3-Clause
|
||||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
"""
|
||||
|
||||
from app.tests.fixtures import DataTestCase
|
||||
from app.service.feature.statistical_features import *
|
||||
|
||||
|
||||
class FeatureTestCase(DataTestCase):
|
||||
|
||||
def test_features(self):
|
||||
testdata_a = self.create_test_data_a()
|
||||
self.assertTrue(time_series_minimum(testdata_a) == 1020900)
|
||||
self.assertTrue(time_series_minimum(testdata_a) == 824757)
|
||||
self.assertTrue((time_series_mean(testdata_a) - 919324.34) < 1e-2)
|
||||
def test_two(self):
|
||||
x = "hello"
|
||||
assert 'hello' in x
|
||||
|
||||
if __name__ == '__main__':
|
||||
a = FeatureTestCase()
|
||||
a.test_features()
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
import pytest
|
||||
|
||||
|
||||
def f():
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
def test_mytest():
|
||||
with pytest.raises(SystemExit):
|
||||
f()
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
.vscode/
|
||||
dist/
|
||||
|
|
@ -0,0 +1 @@
|
|||
Metis前端代码
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
指标集名称,指标集id,指标名称,指标id,数据来源,训练集/测试集,正样本/负样本,样本窗口,dataC,dataB,dataA,数据时间戳
|
||||
登陆验证,1857889,登陆成功量,20158,monitor,test,positive,180,"2895,2661,2694,2728,3045,3016,3021,3005,2912,2873,3042,3157,3202,3296,3301,3319,3349,3225,3383,3597,3449,3532,3609,3556,3748,3601,3564,3681,3738,3933,3687,3663,3729,3626,3609,3833,3879,3999,3893,4174,3924,4111,3918,4083,3780,4169,4053,4192,3690,4005,4083,4494,4021,4035,4069,3804,4352,4000,3883,3927,3663,4130,3867,3860,3681,4051,4101,3892,3855,3589,3850,3773,3776,3525,3687,3444,3510,3564,3463,3696,3680,3576,3565,3643,3651,3660,3524,3497,3308,3279,3002,3345,3273,3284,3171,3219,3124,3152,2836,3012,3054,2982,2833,2954,2859,2850,2816,2892,2687,2724,2719,2663,2726,2710,2571,2318,2874,2604,2439,2372,2565,2435,2561,2590,2454,2490,2386,2449,2406,2482,2280,2573,2472,2335,2316,2405,2572,2479,2251,2420,2532,2486,2350,2560,2541,2459,2357,2599,2516,2346,2571,2519,2444,2348,2685,2451,2413,2569,2589,2537,2507,2425,2623,2576,2669,2663,2646,2485,2525,2519,2523,2575,2546,2599,2760,2489,2779,2585,2454,2622,2632,2479,2592,2486,2685,2654,2498,2637,2642,2648,2528,2663,2607,2397,2613,2603,2694,2540,2590,2647,2485,2459,2590,2607,2461,2473,2522,2642,2445,2579,2597,2356,2566,2677,2419,2409,2532,2582,2529,2595,2585,2599,2410,2594,2564,2383,2528,2614,2523,2502,2616,2567,2343,2462,2650,2333,2767,2522,2313,2260,2534,2485,2386,2585,2627,2502,2442,2539,2575,2425,2596,2466,2499,2386,2564,2590,2357,2557,2534,2472,2367,2548,2625,2736,2536,2518,2617,2518,2616,2588,2670,2551,2665,2570,2492,2715,2618,2653,2658,2681,2872,2757,2617,2758,2602,2606,2735,2723,2669,2699,2896,2638,2791,2717,2725,2773,2957,2768,2653,2775,2786,2672,2720,2848,3040,2798,2952,2823,2728,2948,2911,2978,2885,2893,2897,2828,2970,3074,2999,2979,2914,2911,3088,3018,3058,3044,2923,3174,3098,3118,2878,2769,3153,2998,3133,3164,2894,3045,3207,3225,3089,3179,3228,3036,3125,3182,3123,3320,3026,3218,3135,3199,3077,3266,3396,2974,3344,3266,3281,3184,3056","1706,1667,1698,1794,1758,1756,1649,1809,1867,1908,1835,1865,1986,1990,2090,1961,1884,1977,2102,2147,2174,2204,2350,2176,2353,2479,2448,2492,2379,2478,2566,2425,2507,2486,2550,2559,3024,2683,2790,2724,2873,2666,2745,2829,2886,2889,2842,2876,3045,3072,3048,2854,2979,2945,3146,2903,3133,2893,2933,2936,2777,2910,3253,2986,3000,2899,2786,2815,2869,2834,2850,2957,2883,2829,2914,2746,2907,2714,2885,2926,2808,2799,2790,2631,2795,2733,2779,2624,2515,2625,2734,2462,2563,2553,2445,2501,2492,2426,2475,2490,2371,2421,2194,2269,2156,2376,2225,2133,2262,2249,2188,2144,2081,2152,1937,2067,2225,2146,1926,2029,2034,1969,1871,2104,1880,2036,1902,1944,1869,1933,1885,1966,2036,1926,1864,2028,2070,1989,1907,1956,2012,1895,1994,2077,1882,1911,2080,1996,1986,1950,1974,2026,2087,1902,2105,2023,1994,2041,2171,2064,1991,2081,2084,2156,2080,2238,2111,2254,2155,2230,2160,2078,2167,2154,2251,2018,2454,2215,2075,2223,2226,2272,2113,2270,2145,2195,2146,2237,2166,2045,2268,2222,2231,2239,2294,2224,2201,2055,2302,2329,2142,2033,2290,2148,2157,2356,2163,2272,2098,2173,2097,2146,2246,2282,2208,2214,2279,2075,2300,2040,2179,2061,2203,2103,2225,2080,2186,2129,2182,2087,2097,2199,2188,2213,2059,1874,2408,2213,1985,2087,2172,2150,2015,2148,2055,2240,2102,2082,2184,2049,2205,2148,2118,2301,2190,2193,2202,2041,2148,2294,2127,2005,2208,2383,2129,2241,2252,2361,2269,2209,2336,2223,2351,2320,2283,2237,2255,2440,2340,2174,2473,2328,2505,2344,2453,2327,2321,2526,2277,2299,2273,2297,2275,2268,2143,1898,2269,1977,1805,1921,1861,1872,1757,1864,1953,1831,1906,1923,1872,1859,1906,1956,1861,1980,1955,1974,2006,1952,1862,1903,1964,2009,1864,2146,2104,1971,1989,1969,1896,2021,2232,2057,1900,2164,1920,2002,2154,2102,2065,2184,2150,2051,2041,2043,2130,1990,2185,2260,2122,2072,2218,1958,2160,2089,2109,2025,2303,2217,2088,2171,2106","1270,1276,1285,1266,1345,1383,1240,1370,1345,1276,1351,1485,1363,1496,1465,1525,1495,1462,1585,1704,1592,1640,1722,1647,1765,1701,1778,1709,1841,1863,1816,1912,1879,1883,1897,2016,2025,2160,2109,2049,1972,2096,2048,2182,2108,2146,2127,2242,2168,2285,2230,2140,2257,2372,2183,2123,2464,2299,2249,2177,2127,2219,2485,2278,2173,2208,2272,2288,2175,2224,2203,2248,2126,2100,2092,2233,2216,2120,2182,2023,2154,2016,2051,2008,2054,2074,2154,1983,1851,1900,1805,2014,1905,1791,1845,1788,1779,1814,1916,1749,1704,1710,1697,1707,1730,1720,1604,1685,1656,1572,1655,1621,1579,1489,1553,1376,1641,1436,1392,1485,1473,1393,1412,1441,1436,1432,1410,1393,1398,1424,1398,1339,1414,1420,1419,1390,1465,1420,1465,1430,1391,1405,1409,1483,1457,1409,1451,1479,1463,1477,1506,1507,1452,1501,1604,1474,1379,1525,1441,1440,1633,1538,1498,1544,1597,1612,1497,1565,1609,1570,1631,1720,1563,1517,1634,1445,1811,1622,1465,1641,1604",1530525867
|
||||
登陆验证,1857889,登陆成功量,20159,monitor,test,positive,180,"2895,2661,2694,2728,3045,3016,3021,3005,2912,2873,3042,3157,3202,3296,3301,3319,3349,3225,3383,3597,3449,3532,3609,3556,3748,3601,3564,3681,3738,3933,3687,3663,3729,3626,3609,3833,3879,3999,3893,4174,3924,4111,3918,4083,3780,4169,4053,4192,3690,4005,4083,4494,4021,4035,4069,3804,4352,4000,3883,3927,3663,4130,3867,3860,3681,4051,4101,3892,3855,3589,3850,3773,3776,3525,3687,3444,3510,3564,3463,3696,3680,3576,3565,3643,3651,3660,3524,3497,3308,3279,3002,3345,3273,3284,3171,3219,3124,3152,2836,3012,3054,2982,2833,2954,2859,2850,2816,2892,2687,2724,2719,2663,2726,2710,2571,2318,2874,2604,2439,2372,2565,2435,2561,2590,2454,2490,2386,2449,2406,2482,2280,2573,2472,2335,2316,2405,2572,2479,2251,2420,2532,2486,2350,2560,2541,2459,2357,2599,2516,2346,2571,2519,2444,2348,2685,2451,2413,2569,2589,2537,2507,2425,2623,2576,2669,2663,2646,2485,2525,2519,2523,2575,2546,2599,2760,2489,2779,2585,2454,2622,2632,2479,2592,2486,2685,2654,2498,2637,2642,2648,2528,2663,2607,2397,2613,2603,2694,2540,2590,2647,2485,2459,2590,2607,2461,2473,2522,2642,2445,2579,2597,2356,2566,2677,2419,2409,2532,2582,2529,2595,2585,2599,2410,2594,2564,2383,2528,2614,2523,2502,2616,2567,2343,2462,2650,2333,2767,2522,2313,2260,2534,2485,2386,2585,2627,2502,2442,2539,2575,2425,2596,2466,2499,2386,2564,2590,2357,2557,2534,2472,2367,2548,2625,2736,2536,2518,2617,2518,2616,2588,2670,2551,2665,2570,2492,2715,2618,2653,2658,2681,2872,2757,2617,2758,2602,2606,2735,2723,2669,2699,2896,2638,2791,2717,2725,2773,2957,2768,2653,2775,2786,2672,2720,2848,3040,2798,2952,2823,2728,2948,2911,2978,2885,2893,2897,2828,2970,3074,2999,2979,2914,2911,3088,3018,3058,3044,2923,3174,3098,3118,2878,2769,3153,2998,3133,3164,2894,3045,3207,3225,3089,3179,3228,3036,3125,3182,3123,3320,3026,3218,3135,3199,3077,3266,3396,2974,3344,3266,3281,3184,3056","1706,1667,1698,1794,1758,1756,1649,1809,1867,1908,1835,1865,1986,1990,2090,1961,1884,1977,2102,2147,2174,2204,2350,2176,2353,2479,2448,2492,2379,2478,2566,2425,2507,2486,2550,2559,3024,2683,2790,2724,2873,2666,2745,2829,2886,2889,2842,2876,3045,3072,3048,2854,2979,2945,3146,2903,3133,2893,2933,2936,2777,2910,3253,2986,3000,2899,2786,2815,2869,2834,2850,2957,2883,2829,2914,2746,2907,2714,2885,2926,2808,2799,2790,2631,2795,2733,2779,2624,2515,2625,2734,2462,2563,2553,2445,2501,2492,2426,2475,2490,2371,2421,2194,2269,2156,2376,2225,2133,2262,2249,2188,2144,2081,2152,1937,2067,2225,2146,1926,2029,2034,1969,1871,2104,1880,2036,1902,1944,1869,1933,1885,1966,2036,1926,1864,2028,2070,1989,1907,1956,2012,1895,1994,2077,1882,1911,2080,1996,1986,1950,1974,2026,2087,1902,2105,2023,1994,2041,2171,2064,1991,2081,2084,2156,2080,2238,2111,2254,2155,2230,2160,2078,2167,2154,2251,2018,2454,2215,2075,2223,2226,2272,2113,2270,2145,2195,2146,2237,2166,2045,2268,2222,2231,2239,2294,2224,2201,2055,2302,2329,2142,2033,2290,2148,2157,2356,2163,2272,2098,2173,2097,2146,2246,2282,2208,2214,2279,2075,2300,2040,2179,2061,2203,2103,2225,2080,2186,2129,2182,2087,2097,2199,2188,2213,2059,1874,2408,2213,1985,2087,2172,2150,2015,2148,2055,2240,2102,2082,2184,2049,2205,2148,2118,2301,2190,2193,2202,2041,2148,2294,2127,2005,2208,2383,2129,2241,2252,2361,2269,2209,2336,2223,2351,2320,2283,2237,2255,2440,2340,2174,2473,2328,2505,2344,2453,2327,2321,2526,2277,2299,2273,2297,2275,2268,2143,1898,2269,1977,1805,1921,1861,1872,1757,1864,1953,1831,1906,1923,1872,1859,1906,1956,1861,1980,1955,1974,2006,1952,1862,1903,1964,2009,1864,2146,2104,1971,1989,1969,1896,2021,2232,2057,1900,2164,1920,2002,2154,2102,2065,2184,2150,2051,2041,2043,2130,1990,2185,2260,2122,2072,2218,1958,2160,2089,2109,2025,2303,2217,2088,2171,2106","1270,1276,1285,1266,1345,1383,1240,1370,1345,1276,1351,1485,1363,1496,1465,1525,1495,1462,1585,1704,1592,1640,1722,1647,1765,1701,1778,1709,1841,1863,1816,1912,1879,1883,1897,2016,2025,2160,2109,2049,1972,2096,2048,2182,2108,2146,2127,2242,2168,2285,2230,2140,2257,2372,2183,2123,2464,2299,2249,2177,2127,2219,2485,2278,2173,2208,2272,2288,2175,2224,2203,2248,2126,2100,2092,2233,2216,2120,2182,2023,2154,2016,2051,2008,2054,2074,2154,1983,1851,1900,1805,2014,1905,1791,1845,1788,1779,1814,1916,1749,1704,1710,1697,1707,1730,1720,1604,1685,1656,1572,1655,1621,1579,1489,1553,1376,1641,1436,1392,1485,1473,1393,1412,1441,1436,1432,1410,1393,1398,1424,1398,1339,1414,1420,1419,1390,1465,1420,1465,1430,1391,1405,1409,1483,1457,1409,1451,1479,1463,1477,1506,1507,1452,1501,1604,1474,1379,1525,1441,1440,1633,1538,1498,1544,1597,1612,1497,1565,1609,1570,1631,1720,1563,1517,1634,1445,1811,1622,1465,1641,1604",1530525867
|
||||
|
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 2.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.0 KiB |
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,4 @@
|
|||
{
|
||||
"presets": ["env", "react", "stage-0"],
|
||||
"plugins": ["transform-runtime", "add-module-exports", "syntax-dynamic-import"]
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue