importer: avoid duplicate module load(!); closes #113.

Amazed this one managed to scrape through for so long. Calling
__import__ from within find_module() was causing the target module, in
this case cookielib, to be loaded *then overwritten* by a subsequent
duplicate load higher in the stack.

The result is that cookielib was loaded twice, and, per usual Python
import semantics, a reference to the partially initialized first
cookielib was installed in sys.modules while its code executed.

At the end of cookielib on 2.x, it imports _LWPCookieJar, which in turn
imports the partially built cookielib from sys.modules, then subclasses
the CookieJar from /that/ module.

Everything is wonderful. Then the call returns back up into the import
mechanism which restarts the entire process -- only this time,
_LWPCookieJar is /not/ reinitialized, so the copy in sys.modules is
still left with types pointing at the old module!

So the duplicate import creates a new CookieJar which is not the base
class of LWPCookieJar. Tada! 3 hours debugging.

This is probably a performance fix in disguise, didn't realize things
were so broken. It may also be a regression elsewhere. Urgently need to
finish the tests.
This commit is contained in:
David Wilson 2018-03-08 06:18:01 +05:45
parent 316eebbe29
commit cf01c6b710
3 changed files with 19 additions and 8 deletions

View File

@ -1,2 +1 @@
#localhost localhost
z

View File

@ -1,7 +1,9 @@
---
- hosts: all - hosts: all
gather_facts: false
tasks: tasks:
- name: Get auth token - name: Get auth token
uri: uri:
url: "https://httpbin.org/post" url: "https://httpbin.org/post"
@ -13,7 +15,7 @@
register: r_token register: r_token
no_log: false no_log: false
run_once: true run_once: true
delegate_to: localhost register: foo
- assert:
that: foo.status == 200

View File

@ -452,6 +452,16 @@ class Importer(object):
def __repr__(self): def __repr__(self):
return 'Importer()' return 'Importer()'
def builtin_find_module(self, fullname):
parent, _, modname = fullname.rpartition('.')
if parent:
path = sys.modules[parent].__path__
else:
path = None
fp, pathname, description = imp.find_module(modname, path)
if fp:
fp.close()
def find_module(self, fullname, path=None): def find_module(self, fullname, path=None):
if hasattr(_tls, 'running'): if hasattr(_tls, 'running'):
return None return None
@ -473,7 +483,7 @@ class Importer(object):
return None return None
try: try:
__import__(fullname, {}, {}, ['']) self.builtin_find_module(fullname)
_v and LOG.debug('%r: %r is available locally', self, fullname) _v and LOG.debug('%r: %r is available locally', self, fullname)
except ImportError: except ImportError:
_v and LOG.debug('find_module(%r) returning self', fullname) _v and LOG.debug('find_module(%r) returning self', fullname)
@ -483,7 +493,7 @@ class Importer(object):
def _refuse_imports(self, fullname): def _refuse_imports(self, fullname):
if is_blacklisted_import(self, fullname): if is_blacklisted_import(self, fullname):
raise ImportError('Refused') raise ImportError('Refused: ' + fullname)
f = sys._getframe(2) f = sys._getframe(2)
requestee = f.f_globals['__name__'] requestee = f.f_globals['__name__']