thread confusion

This is the place for queries that don't fit in any of the other categories.

thread confusion

Postby metulburr » Fri Jun 21, 2013 4:45 pm

I for the life of me can not figure out threads. The frustration piling up because of them make we want to not use them at all, for the exception that i beleive i need them. Pretty much creating 2 while loops where one is handling one certian events, and the other handling another certian events. Where the secondary while loop must be shut on and off based on events going on in the first while loop.

So the example would be:
Code: Select all
import time
import threading

class Threader(threading.Thread):
   
    def __init__(self):
        self._should_stop = threading.Event()
        threading.Thread.__init__(self)
        self.x = 0

    def run(self):
        while not self._should_stop.is_set():
            print(self.x)
            self.x += 1
            time.sleep(1)
            #...
        self._should_stop.clear()

    def stop(self):
        self._should_stop.set()
        while self._should_stop.is_set():
            print('not running')
            time.sleep(1)
            #...

       
t = Threader()
t.start()
while True:
    print('running main loop')
    if t.x >= 10:
        t.stop()
        #break
    time.sleep(1)
\
where the global while loop is considered loop 1, and the run and stop loop, being loop 2.

output is:
Code: Select all
...
running main loop
7
running main loop
8
running main loop
9
running main loop
not running
running main loop
not running
not running
not running
not running
not running
...

my expected output is to be printing "running main loop" instead when stop() is called. As that loop needs to always be running. But control stops that and goes to the stop loop instead?

I dont really understand the process at all. Most of the examples i find confuse me even more. MY ultimate goal is to have a main loop contorlling certain events at all times, and that spawn another thread within it another while loop. Where the main loop has the ability to shut on and shut off the second while loop as needed.
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1476
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: thread confusion

Postby metulburr » Fri Jun 21, 2013 4:59 pm

oh i forgot the flag:
Code: Select all
import time
import threading

class Threader(threading.Thread):
   
    def __init__(self):
        self._should_stop = threading.Event()
        threading.Thread.__init__(self)
        self.x = 0
        self.torun = True

    def run(self):
        while not self._should_stop.is_set():
            print(self.x)
            self.x += 1
            time.sleep(1)
            #...
        self._should_stop.clear()

    def stop(self):
        self._should_stop.set()
        while self._should_stop.is_set():
            print('not running')
            print(self.x)
            time.sleep(1)
            #...

       
t = Threader()
t.start()
while True:
    print('running main loop')
    if t.x == 5:
        if t.torun:
            t.torun = False
            t.stop()
        #break
    elif t.x < 5:
        if not t.torun:
            t.torun = True
            t.run()
    time.sleep(1)
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1476
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: thread confusion

Postby Yoriz » Fri Jun 21, 2013 6:31 pm

I think this line
Code: Select all
if t.x == 5:

does not always evaluate to true because the thread could have increased x to greater then 5 when it comes to check it, to resolve this check for equals or greater then 5.

when the mainloop calls t.stop() the mainloop has to wait until t.stop() has finished before it can carry on to resolve this make another thread call t.stop()

Does this do what you are expecting ?
Code: Select all
import time
import threading
import functools


def runAsync(func):
    '''Decorates a method to run in a separate thread'''
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        func_hl = threading.Thread(target=func, args=args, kwargs=kwargs)
        func_hl.start()
        return func_hl
    return wrapper


class Threader(threading.Thread):

    def __init__(self):
        self._should_stop = threading.Event()
        threading.Thread.__init__(self)
        self.x = 0
        self.torun = True

    def run(self):
        while not self._should_stop.is_set():
            print(self.x)
            self.x += 1
            time.sleep(1)
            # ...
        self._should_stop.clear()

    @runAsync
    def stop(self):
        self._should_stop.set()
        while self._should_stop.is_set():
            print('not running')
            print(self.x)
            time.sleep(1)
            # ...


t = Threader()
t.start()
while True:
    print('running main loop')
    if t.x >= 5:
        if t.torun:
            t.torun = False
            t.stop()
        # break
    elif t.x < 5:
        if not t.torun:
            t.torun = True
            t.run()
    time.sleep(1)
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 871
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: thread confusion

Postby metulburr » Fri Jun 21, 2013 8:59 pm

yeah it does. Although with decorators and threads of both of which i am unfamiliar with, I dont quite understand it
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1476
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: thread confusion

Postby Yoriz » Fri Jun 21, 2013 10:32 pm

Mainloop and threaded are made to be seemingly doing stuff at the same time but there are actually switched between one and the other and the order cant be guaranteed for order, this means x could be changed from 5 to 6 before the mainloop gets a chance to ask if it equals 5.
So the mainloop can continue doing its stuff the method stop in threaded has been decorated with runAsync, which means when you call the method stop you are actually calling runAsync which calls the stop method in a new thread.

Does this diagram help at all ?
Thread Diagram.jpg
Thread Diagram.jpg (42.23 KiB) Viewed 1004 times
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 871
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: thread confusion

Postby metulburr » Sat Jun 22, 2013 8:04 am

yeah i think i get it. It just seems rediculous that it gets that complicated when just trying to get an extra thread. And complicated meaning an extra 50 lines of code of which isnt really understood by me. It seems like it would be more easier to create the second loop in another module, and use subprocess to turn it on and off
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1476
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: thread confusion

Postby Yoriz » Sat Jun 22, 2013 10:36 am

I made threader using my Y_Mvc code which can be found here (shameless plug :lol: ), usually the mediator updates a gui, but i made it just print instead.
Because i have an observer of value it gets to know about every change so seeing when value == 5 will always work, the various threads needed are created and there is a queue to ensure data is read/written by only one thing at a time.
Code: Select all
import functools
import threading
import time
import ymvc


def runAsync(func):
    '''Decorates a method to run in a separate thread'''
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        func_hl = threading.Thread(target=func, args=args, kwargs=kwargs)
        func_hl.start()
        return func_hl
    return wrapper


class Threader(ymvc.Model):
    def __init__(self):
        super(Threader, self).__init__()
        self.addObsAttrs('value', 'counting')
        self.bind(self.start)
        self.bind(self.stop)
        self.value = 0
        self.counting = False

    @runAsync
    @ymvc.onMsgSignal('start')
    def start(self):
        self.counting = True
        self.waitInQueue()
        while self.counting:
            print 'Counting'
            time.sleep(1)
            self.value += 1
            self.waitInQueue()

    @runAsync
    @ymvc.onMsgSignal('stop')
    def stop(self):
        self.counting = False


class CountMediator(ymvc.Mediator):
    def __init__(self):
        super(CountMediator, self).__init__()
        self.threader = Threader()
        self.threader.bind(self.onThreaderValue)
        self.threader.bind(self.onThreaderCounting)
        self.threader.notifyMsg('start')
        while True:
            print 'running main loop'
            time.sleep(1)

    @ymvc.onAttrSignal
    def onThreaderValue(self, value):
        print 'Threader count: {}'.format(value)
        if value == 5:
            self.threader.notifyMsg('stop')

    @ymvc.onAttrSignal
    def onThreaderCounting(self, counting):
        if counting:
            print 'Threader started counting'
        else:
            print 'Threader not counting'

if __name__ == '__main__':
    mediator = CountMediator()

Output
Code: Select all
running main loop
Threader count: 0
Threader not counting
Threader started counting
Counting
running main loop
Threader count: 1
Counting
running main loop
Threader count: 2
Counting
running main loop
Threader count: 3
Counting
running main loop
Threader count: 4
Counting
running main loop
Threader count: 5
Threader not counting
running main loop
running main loop
running main loop ... ect
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 871
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: thread confusion

Postby metulburr » Sun Jun 23, 2013 10:12 pm

ok now i am thinking starting from scratch. Starting from the beginning. I think that is getting out of my league at this point.
I am attempting to just return a value from the function being threaded, which it appears to be working. Now i am confuised on what a.join() does? As it appears to nullify the fact of even doing threads in the first place as it does the function first and then the main loop?


Code: Select all
import time
import threading
import queue

queue = queue.Queue()

def print_time(name, delay, q):
   count = 0
   #q.get()
   while True:
      q.put(count)
      if count == 5:
         break
      time.sleep(delay)
      print(name)
      count += 1
      

a = threading.Thread(target=lambda:print_time('first thread', 1, queue))
a.start()

#b = threading.Thread(target=lambda:print_time('second thread', 2, queue))
#b.start()

#a.join()
for i in range(5):
   time.sleep(1)
   q = queue.get()
   print(q)
print('Main script at end')


Mainloop and threaded are made to be seemingly doing stuff at the same time but there are actually switched between one and the other and the order cant be guaranteed for order, this means x could be changed from 5 to 6 before the mainloop gets a chance to ask if it equals 5.

I dont understand. So you are saying threads do not actually execute at the same time? Kind of just a rough estimate

EDIT:
ok so i read that in a book. I guess so, though it doesnt make sense.

EDIT2:
i dont really understand the queue, or the method of before. It seems more reasonable to use classes to retain information instead of returning them:
Code: Select all
import threading

class Power:
    def __init__(self, i):
        self.i = i
    def action(self, p):
        self.i = self.i ** p
       
obj = Power(2)
print(obj.i)
a = threading.Thread(target=lambda:obj.action(5))
a.start()

print(obj.i)
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1476
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: thread confusion

Postby metulburr » Sun Jun 23, 2013 10:55 pm

another thing i do not understand is linking the two threads. Assuming i sleep the method for a couple seconds to imitate a long running function, but yet i do not want to make the last print until that is complete. How would you put a wait in there?

Code: Select all
import time
import threading

class Power:
    def __init__(self, i):
        self.i = i
    def action(self, p):
        time.sleep(2)
        self.i = self.i ** p
       
obj = Power(10)
print(obj.i)
a = threading.Thread(target=lambda:obj.action(5))
a.start()


print(obj.i)
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1476
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: thread confusion

Postby Yoriz » Sun Jun 23, 2013 11:06 pm

Have a watch of this video about the python gil
The join makes the thread your in wait until the thread you joined on to finish what its doing before continuing
i'm a bit lost on what your asking now as the post i was looking at now has edit and edit 2 then theres another post lol.

There is a module called Futures that was added in python 3 and also back ported to python 2 thats written on top of threading you might find that easier to use depending on your use case.
python futures for python 2 (download)
python 3 futures (part of standard library)
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 871
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: thread confusion

Postby Yoriz » Sun Jun 23, 2013 11:52 pm

metulburr wrote:another thing i do not understand is linking the two threads. Assuming i sleep the method for a couple seconds to imitate a long running function, but yet i do not want to make the last print until that is complete. How would you put a wait in there?

Code: Select all
import time
import threading

class Power:
    def __init__(self, i):
        self.i = i
    def action(self, p):
        time.sleep(2)
        self.i = self.i ** p
       
obj = Power(10)
print(obj.i)
a = threading.Thread(target=lambda:obj.action(5))
a.start()


print(obj.i)

Use a join on thread a
Code: Select all
import time
import threading

class Power:
    def __init__(self, i):
        self.i = i
    def action(self, p):
        time.sleep(2)
        self.i = self.i ** p
       
obj = Power(10)
print(obj.i)
a = threading.Thread(target=lambda:obj.action(5))
a.start()

a.join()
print(obj.i)

Outputs
Code: Select all
10
100000
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 871
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: thread confusion

Postby metulburr » Mon Jun 24, 2013 12:03 am

ok that vid made sense.

I think my confusion came from the fact that i thought threads basically spawned another process essentially. In which that wasnt making sense. But after watching that video i think i get it now.

So without joining the two threads each do a portion, and with join() the main thread waits for thread 2 to finish. My question is: What if you wanted half way down thread 2 to start the main thread again.....lets say you use join() on the main thread, then create thread 2 which just printed ints of a for loop of 0-4 , and when thread 2 gets to lets say, 2, you wanted the join() to not exist and the main thread to start working again in conjuntion with thread 2.
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1476
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: thread confusion

Postby Yoriz » Mon Jun 24, 2013 12:35 am

metulburr wrote:So without joining the two threads each do a portion, and with join() the main thread waits for thread 2 to finish. My question is: What if you wanted half way down thread 2 to start the main thread again.....lets say you use join() on the main thread, then create thread 2 which just printed ints of a for loop of 0-4 , and when thread 2 gets to lets say, 2, you wanted the join() to not exist and the main thread to start working again in conjuntion with thread 2.


Maybe this does the above if i undertood it correctly
Code: Select all
import time
import threading

class Threader(threading.Thread):
   
    def __init__(self):
        self._should_stop = threading.Event()
        threading.Thread.__init__(self)
        self.event = threading.Event()
        self.x = 0

    def run(self):
        while True:
            print('Threader x value: {}'.format(self.x))
            if self.x == 2:
                self.event.set()
            time.sleep(1)
            self.x += 1
       
mainloopCount = 0
while True:
    print('mainloopCounter: {}'.format(mainloopCount))
    if mainloopCount == 5:
        print('mainloop reached 5')
       
        threader = Threader()
        print('starting threader')
        threader.start()
        print('pausing mainloop until threader == 2')
        threader.event.wait()
        print('Threader reached 2, restarting mainloop')
    time.sleep(1)
    mainloopCount += 1

Output
Code: Select all
mainloopCounter: 0
mainloopCounter: 1
mainloopCounter: 2
mainloopCounter: 3
mainloopCounter: 4
mainloopCounter: 5
mainloop reached 5
starting threader
Threader x value: 0
pausing mainloop until threader == 2
Threader x value: 1
Threader x value: 2
Threader reached 2, restarting mainloop
mainloopCounter: 6
Threader x value: 3
Threader x value: 4
mainloopCounter: 7 .. ect
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 871
Joined: Fri Feb 08, 2013 1:35 am
Location: UK


Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 1 guest