modify robots.txt and add notification not to use thunder

index.html

@@ -61,6 +61,9 @@
           清华大学TUNA开源镜像源正在重建，如有任何问题，请发送邮件到 thu-opensource-mirror-admin<span style="hidden"></span>&#64;googlegroups<span style="hide"></span>&#46;com 反馈。<br />
           Tsinghua TUNA OpenSouce Mirror is being reconstructed，if you have any issues, please email thu-opensource-mirror-admin<span style="hide"></span>&#64;googlegroups<span style="hide"></span>&#46;com.
         </div>
+        <div class="alert alert-warning">
+          本站带宽资源有限，为了您和他人可以长久使用本站服务，请勿使用迅雷下载本站内容。
+        </div>
         <h3># 镜像源列表 </h3>
       </div>
     </div>

robots.txt

@@ -3,8 +3,10 @@
 User-agent: *
 
 Disallow: /apache
+Disallow: /android
 Disallow: /archlinux
 Disallow: /archlinuxarm
+Disallow: /archlinuxcn
 Disallow: /centos
 Disallow: /chakra
 Disallow: /CPAN
@@ -19,6 +21,7 @@ Disallow: /debian-security
 Disallow: /debian-weekly-builds
 Disallow: /deepin
 Disallow: /deepin-releases
+Disallow: /docker
 Disallow: /epel
 Disallow: /fedora
 Disallow: /freebsd
@@ -40,6 +43,7 @@ Disallow: /slackware
 Disallow: /tex-archive
 Disallow: /ubuntu
 Disallow: /ubuntu-releases
+Disallow: /videolan-ftp
 
 Disallow: /stg