Serialized execution of C++ routines through Python Ext.

A forum for general discussion of the Python programming language.

Serialized execution of C++ routines through Python Ext.

Postby DEV_MEDDIFF » Thu Apr 04, 2013 12:33 pm

Hello Group,

We are working on a multithreaded application. We have written an extension that enables us to execute C++ routines.

There's a C++ routine that takes between 5-10 seconds to complete. So when we call this routine from one of the python thread. Other threads, wait for its execution to complete. We have re-confirmed, that we are not locking in C++ routine.

Is this the expected behavior for python extensions to C++?

Please suggest what we can do to overcome this.

Thanks
dev@meddiff
DEV_MEDDIFF
 
Posts: 1
Joined: Thu Apr 04, 2013 12:10 pm

Re: Serialized execution of C++ routines through Python Ext.

Postby setrofim » Thu Apr 04, 2013 3:08 pm

Yes, this is expected Python behavior -- Python (specifically, CPython) has something called a Global Interpreter Lock (GIL). This basically means that the Python interpreter can only execute Python code in one thread at a time. To get around it, you need to release the GIL inside your extension while executing the long-running C++ routine and then re-acquire it before returning into Python code. See here for how this can be done.
setrofim
 
Posts: 288
Joined: Mon Mar 04, 2013 7:52 pm


Return to General Discussions

Who is online

Users browsing this forum: metulburr and 2 guests