使apt支持bt下载。采用python开发,基本完成。

Python/PHP/Perl 开发与设计
头像
oneleaf
论坛管理员
帖子: 10221
注册时间: 2005-03-27 0:06
系统: Ubuntu 12.04
送出感谢: 7 次
接收感谢: 103 次

使apt支持bt下载。采用python开发,基本完成。

#1

帖子 oneleaf » 2006-04-12 13:50

程序采用GPLV2授权
需要 bittorrent 包支持
sudo apt-get install bittorrent
上次由 oneleaf 在 2006-04-21 20:03,总共编辑 4 次。
头像
firehare
论坛版主
帖子: 2625
注册时间: 2005-04-10 16:54
来自: 温州大学
送出感谢: 0
接收感谢: 2 次
联系:

#2

帖子 firehare » 2006-04-12 14:10

伟大的想法!
支持!
以后大家就有福了!
我心无畏,源自于我心无知。
图片
头像
karron
帖子: 6226
注册时间: 2005-06-11 14:03
来自: 不明真相的群众
送出感谢: 0
接收感谢: 0
联系:

#3

帖子 karron » 2006-04-12 17:53

想法不错.BT这种思想真是泽被昌生啊
我的blog,关于技术,软件,linux,vim <---- 所有博客均被河蟹.
头像
oneleaf
论坛管理员
帖子: 10221
注册时间: 2005-03-27 0:06
系统: Ubuntu 12.04
送出感谢: 7 次
接收感谢: 103 次

#4

帖子 oneleaf » 2006-04-12 18:15

临时贴一下服务器生成bt文件的代码,并不需要这个脚本,主要是修改为每请求一次,自动生成种子文件。

代码: 全选

#!/bin/bash
ToDir=/share/bt/btfiles/
SourceDir=/share
Url=http://bt.ubuntu.org.cn:6969/announce
cd $SourceDir
for file in `find ubuntu-cn/*`; do
  if [ -f $file ]
  then
     ToFile=$ToDir$file.torrent
     mkdir -p `dirname $ToFile`
     echo "makemetafile "$file" to "$ToFile
     btmakemetafile $file $Url --target $ToFile
  fi
done
http://bt.ubuntu.org.cn/

代码: 全选

bttrack --port 6969 \
    --dfile /var/log/bttrack/dlinfo \
    --allowed_dir /share/bt/btfiles/ \
    --logfile /var/log/bttrack/bttrack.log 
下一步的任务是完成种子服务的脚本。
上次由 oneleaf 在 2006-04-21 20:23,总共编辑 7 次。
terry6394
帖子: 48
注册时间: 2006-01-02 12:40
送出感谢: 0
接收感谢: 0

#5

帖子 terry6394 » 2006-04-12 19:03

不大明白
taoyh
帖子: 108
注册时间: 2005-09-26 19:26
送出感谢: 0
接收感谢: 0

#6

帖子 taoyh » 2006-04-17 15:18

支持!这也算一个项目吧?
头像
firehare
论坛版主
帖子: 2625
注册时间: 2005-04-10 16:54
来自: 温州大学
送出感谢: 0
接收感谢: 2 次
联系:

#7

帖子 firehare » 2006-04-17 17:31

当然应该算是一个项目了,项目没有大小,自成体系,能用就成!
我心无畏,源自于我心无知。
图片
头像
millenniumdark
论坛版主
帖子: 4159
注册时间: 2005-07-02 14:41
系统: Ubuntu 14.04 (Kylin)
送出感谢: 42 次
接收感谢: 9 次
联系:

#8

帖子 millenniumdark » 2006-04-17 19:37

期待啊。
头像
oneleaf
论坛管理员
帖子: 10221
注册时间: 2005-03-27 0:06
系统: Ubuntu 12.04
送出感谢: 7 次
接收感谢: 103 次

#9

帖子 oneleaf » 2006-04-20 14:15

服务端程序BtLoaderServer

代码: 全选

#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright (c) 2006 UbuntuChina <http://www.ubuntu.org.cn>
# License: GPLv2
# Author: oneleaf <oneleaf AT gmail.com>
# Version: 0.1

import SimpleHTTPServer
import BaseHTTPServer,SocketServer
import httplib
from time import sleep
from threading import Event,Thread
import BitTorrent.download, BitTorrent.bencode, BitTorrent.btmakemetafile
import os
from distutils import dir_util   

class RequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
    def do_GET(self):
        getfile(self,self.path)

btpath="/share/bt/btfiles"
archivepath="/share"
btfiles={}
stoptime=3600
timeout=60
btfilesize=200000

def getfile(self,url):
    file=archivepath+url
    if url=="" or url=="/" or url=="\\":
        self.send_response(200)
        self.send_header("content-type","text/html")
        self.end_headers()
        self.wfile.write("<HTML>\n")
        self.wfile.write("<HEAD>\n")
        self.wfile.write("<TITLE>Index of "+url+"</TITLE>\n")
        self.wfile.write("</HEAD>\n")
        self.wfile.write("<BODY>\n")
        self.wfile.write("<h1>Index of /</h1>")
        self.wfile.write("<ul>")
        self.wfile.write("<li><a href=\"ubuntu\">ubuntu/</a>\n</li>")
        self.wfile.write("<li><a href=\"ubuntu-cn\">ubuntu-cn/</a></li>\n")
        self.wfile.write("</ul>")
        self.wfile.write("<address>Bt Server</address>\n")
        self.wfile.write("</BODY>\n")
        self.wfile.write("</HTML>\n")
        return
    if not os.path.exists(file):
        self.send_response(403,"No File Find")
        self.send_header("content-type","text/html")
        self.end_headers()
        self.wfile.write("No File Find!")
        return
    if os.path.isdir(file):
        self.send_response(200)
        self.send_header("content-type","text/html")
        self.end_headers()
        self.wfile.write("<HTML>\n")
        self.wfile.write("<HEAD>\n")
        self.wfile.write("<TITLE>Index of "+url+"</TITLE>\n")
        self.wfile.write("</HEAD>\n")
        self.wfile.write("<BODY>\n")
        self.wfile.write("<h1>Index of "+url+"</h1>")
        if url[len(url)-1]<>"/":url=url+"/"
        self.wfile.write("<ul>")
        self.wfile.write("<li><a href=\""+url[:url.rfind('/',0,len(url)-1)+1]+"\">Parent Directory</a></li>\n")
        for mfile in os.listdir(file):
            if os.path.isdir(file+"/"+mfile):
                if mfile[len(mfile)-1]<>"/":mfile=mfile+"/"
                self.wfile.write("<li><a href=\""+url+mfile+"\">"+mfile+"</a></li>\n")
            else:
                size=os.path.getsize(file+"/"+mfile)
                if size>btfilesize:
                    self.wfile.write("<li><a href=\""+url+mfile+"\">"+mfile+"[BT]</a>  "+str(size)+"</li>\n")
                else:
                    self.wfile.write("<li><a href=\""+url+mfile+"\">"+mfile+"</a>  "+str(size)+"</li>\n")                    
        self.wfile.write("</ul>")
        self.wfile.write("<address>Bt Server</address>\n")
        self.wfile.write("</BODY>\n")
        self.wfile.write("</HTML>\n")
    else:
        filesize=os.path.getsize(file)
        if filesize<btfilesize:
            self.send_response(200)
            self.send_header("content-type","file")
            self.send_header("Content-Disposition","attachment;filename="+file[file.rfind('/')+1:])
            self.send_header("Content-length",filesize)
            self.end_headers()
            self.wfile.write(open(file).read())                       
        else:
            btfile=btpath+url+'.torrent'
            makebtfile(btfile,file)
            if os.path.exists(btfile) and loadbtfile(btfile,file)==1:
                self.send_response(200)
                self.send_header("content-type","file")
                self.send_header("Content-Disposition","attachment;filename="+btfile[btfile.rfind('/')+1:])
                self.send_header("Source-length",filesize)
                self.end_headers()
                self.wfile.write(open(btfile).read())           
            else:
                self.send_response(403,"No File Find")
                self.send_header("content-type","text/html")
                self.end_headers()
                self.wfile.write("No File Find!")           
           

def loadbtfile(btfile,file):
    btdownload=Btdownload(btfile, file)
    btdownload.btdownload()
    while btdownload.hasdownload==0:
        if btdownload.timeout<1 and btdownload.fractionDone==0.0:
            btdownload.join()
            return 0
    return 1

def makebtfile(btfile,file):
    dir_util.mkpath(os.path.split(btfile)[0])
    url="http://bt.ubuntu.org.cn:6969/announce"
    BitTorrent.btmakemetafile.make_meta_file(file, url, piece_len_exp=16 ,target=btfile)
    
class Btdownload:
    def __init__(self, btfile, file):
        self.btfile = btfile
        self.file = file
        self.event = Event()
        self.hasdownload=0
        self.timeout=timeout
        self.fractionDone=0.0
        self.thread=None
        
    def choose(self,default, size, saveas, dir): return saveas
    def param(self, params):pass
    def display(self,dict):     
        if btfiles.has_key(self.btfile):
            if btfiles[self.btfile]['stoptime']>0:
                if btfiles[self.btfile]['stoptime']%100==0:
                    print self.file[self.file.rfind('/')+1:],
                    if dict.has_key('activity'): print dict['activity'],
                    if dict.has_key('fractionDone'):
                        self.fractionDone=float(int(dict['fractionDone']*1000))/10 
                        if dict.get('fractionDone') <>1.0: print "fractionDone",str(self.fractionDone)+"%",
                    if dict.has_key('upRate'):print 'upRate',dict['upRate'],
                    if dict.has_key('downRate'):print 'downrate', dict['downRate'],
                    if dict.has_key('upTotal'):print 'uptotal', dict['upTotal'],
                    if dict.has_key('downTotal'):print 'downtotal', dict['downTotal'],
                    print 'stopTime',btfiles[self.btfile]['stoptime']
                btfiles[self.btfile]['stoptime']=btfiles[self.btfile]['stoptime']-1
            else:
                self.join()
                del btfiles[self.btfile]       
        self.timeout=self.timeout-1;
        
    def error(self,message): print message
    def finished(self):
        self.hasdownload=1
        print self.file[self.file.rfind('/')+1:],'finished'       
    def newpath(self, path): self.file = path       
    def btdownload(self):
        if self.btfile in btfiles.keys():
            btfiles[self.btfile]['stoptime']=stoptime
            self.hasdownload=1           
        else:
            btfiles[self.btfile]={'stoptime':stoptime,'object':self}
        args=(['--url','file://'+self.btfile,'--saveas', self.file], self.choose, self.display, self.finished, self.error, self.event, 100, self.newpath, self.param)
        self.thread=Thread(target=BitTorrent.download.download,args=args)
        self.thread.start()
    def join(self):
        if self.event:
            self.event.set()
            self.event = None
        self.timeout=-1


class myWebServer(SocketServer.ThreadingMixIn,BaseHTTPServer.HTTPServer): pass
       
if __name__ == "__main__":
    try:
        server = myWebServer(('', 8080), RequestHandler)
        server.serve_forever()
    finally:
        for btfile in btfiles.keys():
            btfiles[btfile]['object'].join()
上次由 oneleaf 在 2006-04-25 9:53,总共编辑 1 次。
头像
oneleaf
论坛管理员
帖子: 10221
注册时间: 2005-03-27 0:06
系统: Ubuntu 12.04
送出感谢: 7 次
接收感谢: 103 次

#10

帖子 oneleaf » 2006-04-25 9:52

客户端: aptbt.py

代码: 全选

#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright (c) 2006 UbuntuChina <http://www.ubuntu.org.cn>
# License: GPLv2
# Author: oneleaf <oneleaf AT gmail.com>
# Version: 0.3

import SimpleHTTPServer
import BaseHTTPServer,SocketServer
import httplib
import BitTorrent.download, BitTorrent.bencode
from threading import Event,Thread
import os
import tempfile
from distutils import dir_util       
from time import sleep
from StringIO import StringIO

class RequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
    def do_GET(self):
        f = self.send_head()
        if f:
            self.copyfile(f, self.wfile)
            f.close()
    def do_HEAD(self):
        f = self.send_head()
        if f:
            f.close()
    def send_head(self):
        url=self.path
        print "get",url
        conn = httplib.HTTPConnection(btsite+':8080')
        conn.request("GET",url)
        response = conn.getresponse()
        print "get file end."
        if (response.msg.gettype()=="text/html") or (response.msg.getmaintype()=="image"):   
            self.send_response(response.status,response.reason)
            self.send_header("content-type",response.msg.getmaintype())       
            self.end_headers()   
            f = StringIO()
            f.write(response.read())
            return f
        if response.status<>200 :
            self.send_error(response.status,response.reason)
            return None
        debfile='/tmp/archive'+url
        dir_util.mkpath(os.path.split(debfile)[0])
        if not response.getheader('Content-Disposition').endswith('.torrent'):
            self.send_response(200)
            setcontent_type(self,url)
            self.send_header("Content-length",response.getheader("Content-length"))
            self.end_headers()
            dir_util.mkpath(os.path.split(debfile)[0])
            f=open(debfile,'w+b')
            f.wirte(response.read())
            f.close()
            return open(f,'rb')
        else:
            setcontent_type(self,url)
            self.send_header("Content-length",response.getheader("Source-length"))               
            btfile=debfile+".torrent"
            tfile=open(debfile+".torrent",'w+b')
            tfile.write(response.read())
            tfile.flush()
            tfile.close()
            if downloadfilefrombt(self,btfile,debfile)==1:
                self.send_response(200)
                self.end_headers()
                tfile=open(debfile,'rb')
                return tfile
            else:
                self.send_response(200)
                self.end_headers()
                return open(debfile,'rb');
                downfile(self,url)
                btdownload=Btdownload(btfile, debfile)
                btdownload.btdownload()  
    
          
btsite="bt.ubuntu.org.cn"
site="ubuntu.cn99.com"
stoptime=36000
timeout=10

def setcontent_type(self,url):
     if url.endswith('deb'):
         self.send_header("content-type","application/dpkg")
     elif url.endswith('tar.gz'):
         self.send_header("content-type","application/x-gtar")
     elif url.endswith('diff.gz'):
         self.send_header("content-type","application/x-gzip")
     elif url.endswith('.tgz'):
         self.send_header("content-type","application/x-gtar")
     else:
         self.send_header("content-type","text/plain")


def downfile(self,url):
    debfile='/tmp/archive'+url
    dir_util.mkpath(os.path.split(debfile)[0])
    conn = httplib.HTTPConnection(site)
    conn.request("GET",url)
    response = conn.getresponse()
    self.send_response(response.status,response.reason)
    self.send_header("content-type",response.msg.getmaintype())       
    self.end_headers()   
    debfile=open(debfile,'rw+b')
    debfile.write(response.read())
    debfile.close()

                         
def downloadfilefrombt(self, btfile, debfile):
    print "download",debfile[debfile.rfind('/')+1:],"begin..."
    for i in range(2):
        btdownload=Btdownload(btfile, debfile)
        btdownload.btdownload()
        while btdownload.hasdownload==0:
            if btdownload.timeout<1:
                print "download",debfile[debfile.rfind('/')+1:],"timeout,restarting..."
                break
            else:
                self.send_header("fractionDone",btdownload.fractionDone)
            sleep(1)
        if btdownload.hasdownload==1:
            print "download",debfile[debfile.rfind('/')+1:],"end."           
            return 1     
    print "download",debfile[debfile.rfind('/')+1:],"error!" 
    return 0
       
btfiles={}
class Btdownload:
    def __init__(self, btfile, debfile):
        self.btfile = btfile
        self.debfile = debfile
        self.event = Event()
        self.hasdownload=0
        self.fractionDone=0.0
        self.thread=None
        self.timeout=timeout
    def setTimeout(self):
        while self.fractionDone==0.0:
             self.timeout=self.timeout-1
             sleep(1)
    def choose(self,default, size, saveas, dir): return saveas
    def param(self, params):pass
    def display(self,dict):
        if self.timeout < 1:
           self.join()
        if dict.has_key('fractionDone'):
            self.fractionDone=float(int(dict['fractionDone']*1000))/10         
            if dict.get('fractionDone') <> 1.0:
                print self.debfile[self.debfile.rfind('/')+1:],
                print "fractionDone",str(self.fractionDone)+"%",
                if dict.has_key('activity'): print dict['activity'],
                if dict.has_key('downRate'):print 'downrate', int(dict['downRate'])/1000,"K",
                print
        if btfiles.has_key(self.btfile):
            if btfiles[self.btfile]['stoptime']>0:
                btfiles[self.btfile]['stoptime']=btfiles[self.btfile]['stoptime']-1
            else:
                self.join()
                del btfiles[self.btfile]   
                os.remove(self.btfile)
                os.remove(self.debfile)               
         
    def error(self,message): print message
    def finished(self): 
        self.hasdownload=1

    def newpath(self, path): self.debfile = path       
    def btdownload(self):
        if self.btfile in btfiles.keys():
            btfiles[self.btfile]['stoptime']=stoptime
        else:
            btfiles[self.btfile]={'stoptime':stoptime,'object':self}
        args=(['--url','file://'+self.btfile,'--saveas', self.debfile], self.choose, self.display, self.finished, self.error, self.event, 100, self.newpath, self.param)
        self.thread=Thread(None,BitTorrent.download.download,"BitTorrent.download.download", args)
        self.thread.start()
        Thread(target=self.setTimeout).start()
    def join(self):
        if self.event:
            self.event.set()
            self.event = None
        self.timeout=-1
         
class myWebServer(SocketServer.ThreadingMixIn,BaseHTTPServer.HTTPServer): pass
         
if __name__ == "__main__":
    try:
        server = myWebServer(('', 8080), RequestHandler)
        server.serve_forever()
    finally:
        for btfile in btfiles.keys():
            btfiles[btfile]['object'].join()
            os.remove(btfiles[btfile]['object'].btfile)
            os.remove(btfiles[btfile]['object'].debfile) 
上次由 oneleaf 在 2006-04-25 23:14,总共编辑 1 次。
jjheng
帖子: 36
注册时间: 2005-09-29 17:07
送出感谢: 0
接收感谢: 0
联系:

#11

帖子 jjheng » 2006-04-25 11:16

初步试用了一下,能够使用,非常兴奋。提出以下修改意见:
1、尽快打包成deb,放在ubuntu-cn的源中,方便用户安装和升级。
2、将aptbt.py制作成服务器脚本在后台运行,并将脚本添加的/etc/init.d目录中,设置系统启动时自动运行。
头像
oneleaf
论坛管理员
帖子: 10221
注册时间: 2005-03-27 0:06
系统: Ubuntu 12.04
送出感谢: 7 次
接收感谢: 103 次

#12

帖子 oneleaf » 2006-04-27 18:06

最新修改的客户端,更新日期:2006-5-8

需要最新的bt 4.4库支持。
附件
aptbt.py.tar.gz
(1.82 KiB) 下载 134 次
BitTorrent-4.4.0.tar.gz
(1.2 MiB) 下载 155 次
头像
oneleaf
论坛管理员
帖子: 10221
注册时间: 2005-03-27 0:06
系统: Ubuntu 12.04
送出感谢: 7 次
接收感谢: 103 次

#13

帖子 oneleaf » 2006-05-29 13:17

:D
附件
NewbtServer.py.tar.gz
(2.65 KiB) 下载 136 次
aptbt.tar.gz
(325.27 KiB) 下载 150 次
klz1007
帖子: 125
注册时间: 2006-03-11 3:35
送出感谢: 0
接收感谢: 0
联系:

等半天也不能update,不知是怎么回事。

#14

帖子 klz1007 » 2006-06-18 20:07

get http://127.0.0.1:8080/ubuntu/dists/dapp ... elease.gpg
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 40924)
Traceback (most recent call last):
File "/usr/lib/python2.4/SocketServer.py", line 463, in process_request_thread self.finish_request(request, client_address)
File "/usr/lib/python2.4/SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.4/SocketServer.py", line 521, in __init__
self.handle()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 316, in handle
self.handle_one_request()
File "/usr/lib/python2.4/BaseHTTPServer.py", line 310, in handle_one_request
method()
File "./aptbt.py", line 145, in do_GET
f = self.send_head()
File "./aptbt.py", line 162, in send_head
conn.request("GET",url)
File "/usr/lib/python2.4/httplib.py", line 804, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.4/httplib.py", line 827, in _send_request
self.endheaders()
File "/usr/lib/python2.4/httplib.py", line 798, in endheaders
self._send_output()
File "/usr/lib/python2.4/httplib.py", line 679, in _send_output
self.send(msg)
File "/usr/lib/python2.4/httplib.py", line 646, in send
self.connect()
File "/usr/lib/python2.4/httplib.py", line 630, in connect
raise socket.error, msg
error: (110, 'Connection timed out')
---
头像
oneleaf
论坛管理员
帖子: 10221
注册时间: 2005-03-27 0:06
系统: Ubuntu 12.04
送出感谢: 7 次
接收感谢: 103 次

#15

帖子 oneleaf » 2006-06-18 20:42

这个需要服务器上开bt和运行上面的服务端才可以,由于长期没有人使用,我关闭了,仅仅留下代码给大家参考.
回复

回到 “Python/Php/Perl”