how can i write the queue.put in this case

This is the place for queries that don't fit in any of the other categories.

how can i write the queue.put in this case

Postby Paranoid Android » Fri Oct 25, 2013 7:40 am

I'm writing a program to get the domain in same server and it also can scan these domains directory.

Code: Select all
#!/usr/bin/env python
#encoding = utf-8
import threading
import urllib,urllib2,httplib
from urllib2 import Request, urlopen, URLError

import Queue,sys
import re
concurrent = 5
url = sys.argv[1]

class Scanner(threading.Thread):

    def __init__(self, work_q):
        self.work_q = work_q

    def getdomains(self):
        doreq = Request(''+ url)
        response = urlopen(doreq)
        html =
        domains = re.findall('<br><a href=\"(.*?)\" target=\"_blank\"',html)
        return domains

    def run(self):
        alldomains = self.getdomains()
        pathline = [line.rstrip() for line in open("path.txt")]

        while True:
            for aim in alldomains:
                for path in pathline:
                    path = self.work_q.get()

                    req = Request(aim+path)
                        response = urlopen(req)
                    except URLError, e:
                        if hasattr(e, 'reason'):
                            print aim+path,'Not Found'
                        elif hasattr(e,'code'):
                            print aim+path,'Not Found'
                            logs = open('log.txt',"a+")
                            print "[x] Failed to create log file"
                        print aim+path,"Found"

def main():

    work_q = Queue.Queue()
    paths = [line.rstrip() for line in open("path.txt")]
    for i in range(concurrent):
        t = Scanner(work_q)

    for path in paths:



The problem is this program only do the loop of the path, so i only can get the scan result of the first website. I've found the problem,

Code: Select all
for path in paths:
        work_q.put(path) # The program finishes when it puts all the path

If you want to help me to test this program, you may need some directory of website(save it as path.txt)

Code: Select all
Paranoid Android
Posts: 1
Joined: Fri Oct 25, 2013 7:33 am

Return to General Coding Help

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Yahoo [Bot] and 6 guests