­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ 3 nNjGC@s ddlmZddlZddlZddlZddlmZmZddlm Z m Z ddlm Z ddl m Z mZmZddlmZdd lmZdd lmZdd lmZdd lmZd ddgZejeZd6Zd7Zej d/eZ!d0d1Z"ej#e"e!ej#e"e!d2Z$e e d2Z%Gd3d d eZ&Gd4dde&Z'd5dZ(dS)8)absolute_importN)HTTPHeaderDictRecentlyUsedContainer)HTTPConnectionPoolHTTPSConnectionPool)port_by_scheme)LocationValueError MaxRetryErrorProxySchemeUnknown)six)urljoin)RequestMethods) parse_url)Retry PoolManager ProxyManagerproxy_from_urlkey_file cert_file cert_reqsca_certs ssl_version ca_cert_dir ssl_context key_schemekey_hostkey_port key_timeout key_retries key_strict key_blockkey_source_address key_key_file key_cert_file key_cert_reqs key_ca_certskey_ssl_versionkey_ca_cert_dirkey_ssl_context key_maxsize key_headers key__proxykey__proxy_headerskey_socket_optionskey__socks_optionskey_assert_hostnamekey_assert_fingerprintkey_server_hostnamePoolKeycCs|j}|dj|d<|dj|d<x4d D],}||kr.||dk r.t||j||<q.W|jd}|dk r|t||d<x&t|jD]}|j||d|<qWx|j D]}||krd||<qW|f|S) a Create a pool key out of a request context dictionary. According to RFC 3986, both the scheme and host are case-insensitive. Therefore, this function normalizes both before constructing the pool key for an HTTPS request. If you wish to change this behaviour, provide alternate callables to ``key_fn_by_scheme``. :param key_class: The class to use when constructing the key. This should be a namedtuple with the ``scheme`` and ``host`` keys at a minimum. :type key_class: namedtuple :param request_context: A dictionary-like object that contain the context for a request. :type request_context: dict :return: A namedtuple that can be used as a connection pool key. :rtype: PoolKey schemehostheaders_proxy_headers_socks_optionsNZsocket_optionsZkey_)r6r7r8) copylower frozensetitemsgettuplelistkeyspop_fields)Z key_classrequest_contextcontextkeyZ socket_optsZfieldrF!/usr/lib/python3.6/poolmanager.py_default_key_normalizer;s     rH)httphttpsc@sxeZdZdZdZdddZddZdd Zdd d Zd d Z dddZ ddZ dddZ d ddZ ddZd!ddZdS)"ra$ Allows for arbitrary requests while transparently keeping track of necessary connection pools for you. :param num_pools: Number of connection pools to cache before discarding the least recently used pool. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param \**connection_pool_kw: Additional parameters are used to create fresh :class:`urllib3.connectionpool.ConnectionPool` instances. Example:: >>> manager = PoolManager(num_pools=2) >>> r = manager.request('GET', 'http://google.com/') >>> r = manager.request('GET', 'http://google.com/mail') >>> r = manager.request('GET', 'http://yahoo.com/') >>> len(manager.pools) 2 N cKs8tj||||_t|ddd|_t|_tj|_dS)NcSs|jS)N)close)prFrFrGsz&PoolManager.__init__..)Z dispose_func)r__init__connection_pool_kwrpoolspool_classes_by_schemekey_fn_by_schemer9)self num_poolsr6rPrFrFrGrOs  zPoolManager.__init__cCs|S)NrF)rTrFrFrG __enter__szPoolManager.__enter__cCs |jdS)NF)clear)rTexc_typeZexc_valZexc_tbrFrFrG__exit__szPoolManager.__exit__cCsf|j|}|dkr|jj}xdD]}|j|dq"W|dkrXxtD]}|j|dqDW|||f|S)a Create a new :class:`ConnectionPool` based on host, port, scheme, and any additional pool keyword arguments. If ``request_context`` is provided, it is provided as keyword arguments to the pool class used. This method is used to actually create the connection pools handed out by :meth:`connection_from_url` and companion methods. It is intended to be overridden for customization. Nr4r5portrI)r4r5rZ)rRrPr9rA SSL_KEYWORDS)rTr4r5rZrCZpool_clsrEkwrFrFrG _new_pools    zPoolManager._new_poolcCs|jjdS)z Empty our store of pools and direct them all to close. This will not affect in-flight connections, but they will not be re-used after completion. N)rQrW)rTrFrFrGrWszPoolManager.clearrIcCsT|s td|j|}|pd|d<|s:tj|djd}||d<||d<|j|S)a Get a :class:`ConnectionPool` based on the host, port, and scheme. If ``port`` isn't given, it will be derived from the ``scheme`` using ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is provided, it is merged with the instance's ``connection_pool_kw`` variable and used to create the new connection pool, if one is needed. zNo host specified.rIr4PrZr5)r _merge_pool_kwargsrr=r:connection_from_context)rTr5rZr4 pool_kwargsrCrFrFrGconnection_from_hosts   z PoolManager.connection_from_hostcCs,|dj}|j|}||}|j||dS)z Get a :class:`ConnectionPool` based on the request context. ``request_context`` must at least contain the ``scheme`` key and its value must be a key in ``key_fn_by_scheme`` instance variable. r4)rC)r:rSconnection_from_pool_key)rTrCr4Zpool_key_constructorpool_keyrFrFrGr`s  z#PoolManager.connection_from_contextc Cs`|jjN|jj|}|r|S|d}|d}|d}|j||||d}||j|<WdQRX|S)z Get a :class:`ConnectionPool` based on the provided pool key. ``pool_key`` should be a namedtuple that only contains immutable objects. At a minimum it must have the ``scheme``, ``host``, and ``port`` fields. r4r5rZ)rCN)rQlockr=r])rTrdrCZpoolr4r5rZrFrFrGrcs  z$PoolManager.connection_from_pool_keycCs t|}|j|j|j|j|dS)a Similar to :func:`urllib3.connectionpool.connection_from_url`. If ``pool_kwargs`` is not provided and a new pool needs to be constructed, ``self.connection_pool_kw`` is used to initialize the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs`` is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided ``pool_kwargs`` are not used. )rZr4ra)rrbr5rZr4)rTurlraurFrFrGconnection_from_url s zPoolManager.connection_from_urlc CsZ|jj}|rVxF|jD]:\}}|dkrJy ||=WqRtk rFYqRXq|||<qW|S)a Merge a dictionary of override values for self.connection_pool_kw. This does not modify self.connection_pool_kw and returns a new dict. Any keys in the override dictionary with a value of ``None`` are removed from the merged dictionary. N)rPr9r<KeyError)rToverrideZbase_pool_kwargsrEvaluerFrFrGr_s   zPoolManager._merge_pool_kwargsTc Kst|}|j|j|j|jd}d|d<d|d<d|krD|jj|d<|jdk rj|jdkrj|j||f|}n|j||j f|}|o|j }|s|St ||}|j dkrd }d|d <t |dj|d<|jd } t| tstj| |d } | jo|j| r %s)rrbr5rZr4r6r9proxyurlopenZ request_uriZget_redirect_locationr ZstatusrZ_prepare_for_method_changer= isinstancerZfrom_intZremove_headers_on_redirectZ is_same_hostr?r Ziterkeysr:rAZ incrementr Zraise_on_redirectloginfo) rTmethodrfrlr\rgZconnrnZredirect_locationrmr6headerrFrFrGrp/sH       zPoolManager.urlopen)rKN)N)NrIN)N)N)T)__name__ __module__ __qualname____doc__rorOrVrYr]rWrbr`rcrhr_rprFrFrFrGr{s     csHeZdZdZdfdd Zdfdd Zdd d Zdfd d ZZS)rax Behaves just like :class:`PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs. :param proxy_url: The URL of the proxy to be used. :param proxy_headers: A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication. Example: >>> proxy = urllib3.ProxyManager('http://localhost:3128/') >>> r1 = proxy.request('GET', 'http://google.com/') >>> r2 = proxy.request('GET', 'http://httpbin.org/') >>> len(proxy.pools) 1 >>> r3 = proxy.request('GET', 'https://httpbin.org/') >>> r4 = proxy.request('GET', 'https://twitter.com/') >>> len(proxy.pools) 3 rKNc st|trd|j|j|jf}t|}|jsFtj|jd}|j|d}|jdkrZt |j||_ |pfi|_ |j |d<|j |d<t t |j||f|dS) Nz %s://%s:%ir^)rZrIrJ_proxyr7)rIrJ)rqrr4r5rZrrr=_replacer ro proxy_headerssuperrrO)rT proxy_urlrUr6r|rProrZ) __class__rFrGrOs          zProxyManager.__init__rIcsD|dkr tt|j||||dStt|j|jj|jj|jj|dS)NrJ)ra)r}rrbror5rZr4)rTr5rZr4ra)rrFrGrbs   z!ProxyManager.connection_from_hostcCs0ddi}t|j}|r||d<|r,|j||S)z Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user. ZAcceptz*/*ZHost)rnetlocupdate)rTrfr6Zheaders_rrFrFrG_set_proxy_headerss  zProxyManager._set_proxy_headersTc sNt|}|jdkr0|jd|j}|j|||d<tt|j||fd|i|S)z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.rIr6rl)rr4r=r6rr}rrp)rTrtrfrlr\rgr6)rrFrGrps  zProxyManager.urlopen)rKNN)NrIN)N)T) rvrwrxryrOrbrrp __classcell__rFrF)rrGrps  cKstfd|i|S)Nr~)r)rfr\rFrFrGrs)rrrrrrr)rrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2))Z __future__r collections functoolsZlogging _collectionsrrZconnectionpoolrrr exceptionsr r r Zpackagesr Zpackages.six.moves.urllib.parser ZrequestrZutil.urlrZ util.retryr__all__Z getLoggerrvrrr[Z _key_fields namedtupler3rHpartialrSrRrrrrFrFrFrGsd          6 vW