ZB @ s d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d l m Z d d l m
Z
d d l m Z m
Z
d d l m Z d d l m Z m Z d d l m Z m Z m Z m Z m Z m Z m Z m Z m Z d d
l m Z d d l m Z m! Z! m" Z" d d l m# Z# m$ Z$ m% Z% m& Z& d d
l' m( Z( d d l) m* Z* y d d l+ Z+ Wn e, k
rd Z+ Yn Xd d d f Z- Gd d d Z. Gd d d Z/ Gd d d Z0 Gd d d Z1 Gd d d e0 Z2 Gd d d e0 Z3 d S) N) defaultdict)suppress) SimpleCookie)cycleislice) monotonic )hdrshelpers) ClientConnectionError ClientConnectorCertificateErrorClientConnectorErrorClientConnectorSSLErrorClientHttpProxyErrorClientProxyConnectionErrorServerFingerprintMismatchcertificate_errors
ssl_errors)ResponseHandler)
ClientRequest Fingerprint_merge_ssl_params)PY_36
is_ip_addressnoopsentinel)EventResultOrError)DefaultResolver
BaseConnector TCPConnector
UnixConnectorc @ s e Z d Z d Z d Z d d Z d d Z e d d Z e d d Z
e d
d Z e d d
Z e d d Z
d d Z d d Z d d Z d d Z d d Z e d d Z d S)
ConnectionNc C sX | | _ | | _ | | _ | | _ g | _ | j rT t j t j d | _
d S)Nr ) _key
_connector_loop _protocol
_callbacks get_debug traceback
extract_stacksys _getframe_source_traceback)self connectorkeyprotocolloop r2
/connector.py__init__) s zConnection.__init__c C s d j | j S)NzConnection<{}>)formatr" )r- r2 r2 r3 __repr__3 s zConnection.__repr__c C s | j d k r t r$ d | i } n i } | j d j | t | | j j rY d S| j j | j | j d d d | d d i } | j
d k r | j
| d <| j j | d S) NsourcezUnclosed connection {!r} should_closeTZclient_connectionmessagezUnclosed connectionsource_traceback) r% r warnr5 ResourceWarningr$ is_closedr# _releaser" r, call_exception_handler)r- _warningskwargscontextr2 r2 r3 __del__6 s
zConnection.__del__c C s | j S)N)r$ )r- r2 r2 r3 r1 K s zConnection.loopc C s
| j j S)N)r% transport)r- r2 r2 r3 rD O s zConnection.transportc C s | j S)N)r% )r- r2 r2 r3 r0 S s zConnection.protocolc C s
| j j S)N)r% writer)r- r2 r2 r3 rE W s zConnection.writerc C s | d k r | j j | d S)N)r& append)r- callbackr2 r2 r3 add_callback[ s zConnection.add_callbackc
C sM | j d d g } | _ x) | D]! } t t | Wd QRXq$ Wd S)N)r& r Exception)r- Z callbackscbr2 r2 r3 _notify_release_ s
zConnection._notify_releasec C sE | j | j d k rA | j j | j | j d d d | _ d S)Nr8 T)rK r% r# r> r" )r- r2 r2 r3 closef s
zConnection.closec C sK | j | j d k rG | j j | j | j d | j j d | _ d S)Nr8 )rK r% r# r> r" r8 )r- r2 r2 r3 releasen s
zConnection.releasec C s9 | j | j d k r, | j j | j d | _ d S)N)rK r% r# _release_acquired)r- r2 r2 r3 detachw s
zConnection.detachc C s | j d k p | j j S)N)r% is_connected)r- r2 r2 r3 closed~ s zConnection.closed)__name__
__module__ __qualname__r,
_transportr4 r6 warningsrC propertyr1 rD r0 rE rH rK rL rM rO rQ r2 r2 r2 r3 r! $ s
r! c @ s" e Z d Z d Z d d Z d S)_TransportPlaceholderz0 placeholder for BaseConnector.connect function c C s d S)Nr2 )r- r2 r2 r3 rL s z_TransportPlaceholder.closeN)rR rS rT __doc__rL r2 r2 r2 r3 rX s rX c @ sK e Z d Z d Z d Z d Z d Z d e d d d d d
d d d d
d d d Z e d d Z
d d Z d d Z e
d d Z e
d d Z e
d d Z d d Z d d Z d d! Z d" d# Z e
d$ d% Z d d& d' Z d( d) Z d* d+ Z d, d- Z d. d d/ d0 Z d d1 d2 Z d S)3r a Base connector class.
keepalive_timeout - (optional) Keep-alive timeout.
force_close - Set to True to force close and do reconnect
after each request (and between redirects).
limit - The total number of simultaneous connections.
limit_per_host - Number of simultaneous connections to one host.
enable_cleanup_closed - Enables clean-up closed ssl transports.
Disabled by default.
loop - Optional event loop.
TNg @keepalive_timeout force_closeFlimitd limit_per_hostr enable_cleanup_closedr1 c C s> | r- | d k r? | t k r? t d n | t k r? d } | d k rW t j } d | _ | j r t j t j d | _
i | _ | | _ | | _
t | _ t t | _ | | _ | | _ t t | _ | | _ t j t d | | _ t | _ d | _ d | _ | | _ g | _ | j! d S)Nz6keepalive_timeout cannot be set if force_close is Trueg .@Fr r1 )"r
ValueErrorasyncioZget_event_loop_closedr' r( r) r* r+ r, _conns_limit_limit_per_hostset _acquiredr _acquired_per_host_keepalive_timeout _force_closelist_waitersr$ functoolspartialr _factoryr Zcookies_cleanup_handle_cleanup_closed_handle_cleanup_closed_disabled_cleanup_closed_transports_cleanup_closed)r- rZ r[ r\ r^ r_ r1 r2 r2 r3 r4 s6
zBaseConnector.__init__c C s | j r
d S| j s d Sd d | j j D } | j t rU d | i } n i } | j d j | t | d | d | d d i } | j d k r | j | d <| j j
| d S)
Nc S s g | ] } t | q Sr2 )repr).0cr2 r2 r3
<listcomp> s z)BaseConnector.__del__.<locals>.<listcomp>r7 zUnclosed connector {!r}r. connectionsr9 zUnclosed connectorr: ) rb rc valuesrL r r; r5 r< r, r$ r? )r- r@ connsrA rB r2 r2 r3 rC s$
zBaseConnector.__del__c C s | S)Nr2 )r- r2 r2 r3 __enter__ s zBaseConnector.__enter__c G s | j d S)N)rL )r- excr2 r2 r3 __exit__ s zBaseConnector.__exit__c C s | j S)z1Ultimately close connection on releasing if True.)rj )r- r2 r2 r3 r[ s zBaseConnector.force_closec C s | j S)zThe total number for simultaneous connections.
If limit is 0 the connector has no limit.
The default limit size is 100.
)rd )r- r2 r2 r3 r\ s zBaseConnector.limitc C s | j S)zThe limit_per_host for simultaneous connections
to the same endpoint.
Endpoints are the same if they are have equal
(host, port, is_ssl) triple.
)re )r- r2 r2 r3 r^ s z BaseConnector.limit_per_hostc C s* | j r | j j | j j } | j } | j r i } | | } x | j j D] \ } } g } xv | D]n \ } } | j rp | | d k r | j }
| d r | j r | j
j |
qp | j | | f qp W| rW | | | <qW W| | _ | j r&t j
| d | | j | _ d S)zCleanup unused transports.r r _cleanupN)rp cancelr$ timeri rc itemsrP rL rr rs rF r
weakref_handle) r- Znowtimeoutry deadliner/ r{ aliveprotoZuse_timerD r2 r2 r3 r s.
zBaseConnector._cleanupc C sD | j } | | k r d S| | } | j | | s@ | j | =d S)N)rh remove)r- r/ valZacquired_per_hostr{ r2 r2 r3 _drop_acquired_per_host s
z%BaseConnector._drop_acquired_per_hostc C sw | j r | j j x' | j D] } | d k r | j q Wg | _ | j ss t j | d | j | j | _ d S)z}Double confirmation for transport close.
Some broken ssl servers may leave socket open without proper close.
Nrt ) rq r rs abortrr r
r _cleanup_closed_periodr$ )r- rD r2 r2 r3 rt ( s
z BaseConnector._cleanup_closedc C s- | j r
d Sd | _ z | j j r/ t S| j rE | j j | j r[ | j j x8 | j j D]' } x | D] \ } } | j qx Wqk Wx | j
D] } | j q Wx' | j D] } | d k r | j q WWd | j j
| j
j
| j j
d | _ | j j
d | _ Xd S)z Close all opened transports.NT)rb r$ r= r rp r rq rc rz rL rg rs r clearrl )r- datar t0rD r2 r2 r3 rL : s0
zBaseConnector.closec C s | j S)z;Is connector closed.
A readonly property.
)rb )r- r2 r2 r3 rQ ` s zBaseConnector.closedc s | j } | j rn | j t | j } | j r | d k r | | j k r | j t | j j | } n@ | j r | | j k r | j t | j j | } n d } | d k r]| j j } | j | } | j
| | r x | D] } | j Id Hq Wz
| Id HWd | j | | s6| j | =X| r]x | D] } | j
Id HqDW| j | } | d k rt } | j j | | j | j | | rx | D] } | j Id HqWz y= | j | d | Id H} | j r
| j t d WnV t k
rc| | j k r\x/ | j | D] }
|
j s8|
j d Pq8W Yn XWd | j s| j j | | j | | X| rxF | D] } | j Id HqWn& | rx | D] } | j Id HqW| j j | | j | j | t | | | | j S)z'Get from pool or create new connection.r r NtraceszConnector is closed.) connection_keyrd lenrg re rh getr$ Z
create_futurerl rF Z send_connection_queued_startr Zsend_connection_queued_end_getrX addZ send_connection_create_start_create_connectionrb rL r
BaseExceptiondone
set_resultr Zsend_connection_create_endZsend_connection_reuseconnr! ) r- reqr r/ Z availableZfutwaiterstracer Z placeholderwaiterr2 r2 r3 connecth sx
zBaseConnector.connectc C s y | j | } Wn t k
r) d SYn X| j j } x | r | j \ } } | j r< | | | j k r | j } | d r | j r | j j
| q< | s | j | =| Sq< W| j | =d S)Nr r ) rc KeyErrorr$ r poprP ri rL rr rs rF )r- r/ r{ t1r r rD r2 r2 r3 r s"
zBaseConnector._getc C s | j rp | j t | j d k r x | j j D]4 \ } } | r5 | d j sh | d j d Pq5 WnQ | j r xE | j j D]4 \ } } | r | d j s | d j d Pq Wd S)Nr )rd r rg rl r r r re )r- r/ r r2 r2 r3 _release_waiter s z BaseConnector._release_waiterc C sT | j r
d Sy$ | j j | | j | | Wn t k
rE Yn X| j d S)N)rb rg r r r r )r- r/ r r2 r2 r3 rN s
z BaseConnector._release_acquiredr8 c C s | j r
d S| j | | | j r, d } | s; | j rn | j } | d r | j r | j j | n{ | j j | } | d k r g } | j | <| j | | j
j f | j d k r t
j | d | j | j
| _ d S)NTr r r )rb rN rj r8 rL rr rs rF rc r r$ r rp r
r ri )r- r/ r0 r8 rD r{ r2 r2 r3 r> s zBaseConnector._releasec s
t d S)N)NotImplementedError)r- r r r2 r2 r3 r s z BaseConnector._create_connection) rR rS rT rY rb r, r r r4 rV rC r| r~ rW r[ r\ r^ r r rt rL rQ r r r rN r> r r2 r2 r2 r3 r s0 ) &X c @ sg e Z d Z d d d Z d d Z d d Z d d Z d
d Z d d
Z d d Z d S)_DNSCacheTableNc C s i | _ i | _ | | _ d S)N) _addrs_rr _timestamps_ttl)r- ttlr2 r2 r3 r4 s z_DNSCacheTable.__init__c C s
| | j k S)N)r )r- hostr2 r2 r3 __contains__! s z_DNSCacheTable.__contains__c C s< t | t | f | j | <| j r8 t | j | <d S)N)r r r r r r )r- r addrsr2 r2 r3 r $ s z_DNSCacheTable.addc C s3 | j j | d | j r/ | j j | d d S)N)r r r r )r- r r2 r2 r3 r * s z_DNSCacheTable.removec C s | j j | j j d S)N)r r r )r- r2 r2 r3 r 0 s
z_DNSCacheTable.clearc C s6 | j | \ } } t t | | } t | | S)N)r rk r next)r- r r1 lengthr r2 r2 r3
next_addrs4 s
z_DNSCacheTable.next_addrsc C s. | j d k r d S| j | | j t k S)NF)r r r )r- r r2 r2 r3 expired; s z_DNSCacheTable.expired)
rR rS rT r4 r r r r r r r2 r2 r2 r3 r s r c ! sZ e Z d Z d Z d d d d d d d d d d
d d d d d
d d d d e d d d d d d
d d d d f d d Z f d d Z e d d Z e d d Z d d d d Z
d d! d" Z d d# d$ Z e
e j d d% d& Z d' d( Z d) d* Z d+ e d, d- Z d+ e d. d d/ d0 Z d d1 d2 Z S)3r a/ TCP connector.
verify_ssl - Set to True to check ssl certifications.
fingerprint - Pass the binary sha256
digest of the expected certificate in DER format to verify
that the certificate the server presents matches. See also
https://en.wikipedia.org/wiki/Transport_Layer_Security#Certificate_pinning
resolver - Enable DNS lookups and use this
resolver
use_dns_cache - Use memory cache for DNS lookups.
ttl_dns_cache - Max seconds having cached a DNS entry, None forever.
family - socket address family
local_addr - local tuple of (host, port) to bind socket to
keepalive_timeout - (optional) Keep-alive timeout.
force_close - Set to True to force close and do reconnect
after each request (and between redirects).
limit - The total number of simultaneous connections.
limit_per_host - Number of simultaneous connections to one host.
enable_cleanup_closed - Enables clean-up closed ssl transports.
Disabled by default.
loop - Optional event loop.
verify_sslT fingerprintN
use_dns_cache
ttl_dns_cache
familyr ssl_contextssl
local_addrresolverrZ r[ Fr\ r] r^ r_ r1 c
s t j d |
d | d | d |
d | d | t | | | | | _ | d k rg t d | j } | | _ | | _ t d | | _ i | _
| | _ | | _ d S)NrZ r[ r\ r^ r_ r1 r )
superr4 r _sslr r$ _resolver_use_dns_cacher
_cached_hosts_throttle_dns_events_family _local_addr)r- r r r r r r r r r rZ r[ r\ r^ r_ r1 ) __class__r2 r3 r4 [ s zTCPConnector.__init__c s5 x! | j j D] } | j q Wt j d S)z Close all ongoing DNS calls.N)r rz r r rL )r- Zev)r r2 r3 rL s s zTCPConnector.closec C s | j S)zSocket family like AF_INET.)r )r- r2 r2 r3 r z s zTCPConnector.familyc C s | j S)z%True if local DNS caching is enabled.)r )r- r2 r2 r3 r s zTCPConnector.use_dns_cachec C si | d k r1 | d k r1 | j j | | f n4 | d k sI | d k rX t d n
| j j d S)z8Remove specified host/port or clear all dns local cache.Nz5either both host and port or none of them are allowed)r r r` r )r- r portr2 r2 r3 clear_dns_cache s
z TCPConnector.clear_dns_cachec s t | r: d | d | d | d | j d d d d i g S| j s | rl x | D] } | j | Id HqP W| j j | | d | j Id H} | r x | D] } | j | Id Hq W| S| | f } | | j k r#| j j | r#| rx | D] } | j | Id Hq W| j j
| S| | j k rt| r[x | D] } | j | Id Hq?W| j | j Id HnH| rx | D] } | j
| Id HqWt | j | j | <z y | rx | D] } | j | Id HqWt j | j j | | d | j d | j Id H} | r@x | D] } | j | Id Hq$W| j j | | | j | j Wn? t k
r} z | j | j d | WYd d } ~ Xn XWd | j j | X| j j
| S)
Nhostnamer r r r r flagsr1 r} )r r r Zsend_dns_resolvehost_startr ZresolveZsend_dns_resolvehost_endr r Zsend_dns_cache_hitr r waitZsend_dns_cache_missr r$ ra Zshieldr rf r r ) r- r r r r resr/ r er2 r2 r3
_resolve_host s^
zTCPConnector._resolve_hostc sP | j r, | j | d d Id H\ } } n | j | d d Id H\ } } | S)zcCreate connection.
Has same keyword arguments as BaseEventLoop.create_connection.
r N)proxy_create_proxy_connection_create_direct_connection)r- r r _r r2 r2 r3 r s z TCPConnector._create_connectionc C sj | r t j St j t j } | j t j O_ | j t j O_ | j t j O_ | j | Sd S)N) r Zcreate_default_context
SSLContextZPROTOCOL_SSLv23ZoptionsZ OP_NO_SSLv2Z OP_NO_SSLv3ZOP_NO_COMPRESSIONZset_default_verify_paths)Zverified
sslcontextr2 r2 r3 _make_ssl_context s
z TCPConnector._make_ssl_contextc C s | j r t d k r$ t d | j } t | t j rC | S| d k r\ | j d S| j } t | t j r{ | S| d k r | j d S| j d Sd Sd S)a* Logic to get the correct SSL context
0. if req.ssl is false, return None
1. if ssl_context is specified in req, use it
2. if _ssl_context is specified in self, use it
3. otherwise:
1. if verify_ssl is not specified in req, use self.ssl_context
(will generate a default context according to self.verify_ssl)
2. if verify_ssl is True in req, generate a default SSL context
3. if verify_ssl is False in req, generate a SSL context that
won't verify
NzSSL is not supported.FT)is_sslr RuntimeError
isinstancer r r )r- r r r2 r2 r3 _get_ssl_context s
z TCPConnector._get_ssl_contextc C s< | j } t | t r | S| j } t | t r8 | Sd S)N)r r r r )r- r Zretr2 r2 r3 _get_fingerprint s z TCPConnector._get_fingerprint client_errorc s y | j j | | Id HSWn t k
rX } z t | j | | WYd d } ~ Xns t k
r } z t | j | | WYd d } ~ Xn: t k
r } z | | j | | WYd d } ~ Xn Xd S)N)r$ create_connectionr r r r r OSError)r- r r argsrA r} r2 r2 r3 _wrap_create_connection s $'z$TCPConnector._wrap_create_connectionr c - s | j | } | j | } y* | j | j j | j d | Id H} Wn: t k
r } z t | j | | WYd d } ~ Xn Xd } xM| D]?} | d }
| d } yv | j | j
|
| d | d | d d | d d | d d | r | d n d d
| j d | d | Id H\ } }
Wn. t k
rR} z | } w WYd d } ~ Xn X| j r| ry | j
| WnQ t k
r} z1 | j | j s| j j | | } w WYd d } ~ Xn X| |
f SW| d S)
Nr r r r r r r server_hostnamer r r r )r r r urlZraw_hostr r r
r r ro r r checkr rL rr rs rF )r- r r r r r Zhostsr} last_excZhinfor r Ztranspr r2 r2 r3 r ) sF (
z&TCPConnector._create_direct_connectionc s i } | j d k r | j } | j t j | t j <t t j | j d | d | j d | j d | j } | j
| d t Id H\ } } | j j t j
d } | d k r | j s | | j t j <n | | j t j <| j r}| j | } t j | _ | j | _ | j | j | j f } t | | | | j }
| j |
} z y | j |
d Id H} Wn) t k
r| j |
j Yn Xd |
_ d |
_ z| | j d k rt | j | j d | j d | j! d | j | j" d
d d }
|
d k rt# d |
j$ }
Wd | j X| j% | j& d | d
|
d | j d | Id H\ } } Wd | j X| | f S)Nheadersauthr1 r r T coder9 socketdefaultz)Transport does not expose socket instancesockr r )'Z
proxy_headersr r ZHOSTr ZMETH_GETr Z
proxy_authr$ r r r r Z
AUTHORIZATIONr ZPROXY_AUTHORIZATIONr Z METH_CONNECTmethodr r r r! sendstartr rL r% rU statusr Z request_infohistoryreasonZget_extra_infor dupr ro )r- r r r Z proxy_reqrD r r r r/ connZ
proxy_respZrespZrawsockr2 r2 r3 r Y sh
z%TCPConnector._create_proxy_connection)rR rS rT rY r r4 rL rW r r r r r staticmethodrm lru_cacher r r r
r r r r2 r2 )r r3 r B s.
E ! .c sX e Z d Z d Z d e d d d f d d Z e d d Z d d
d Z S) r a Unix socket connector.
path - Unix socket path.
keepalive_timeout - (optional) Keep-alive timeout.
force_close - Set to True to force close and do reconnect
after each request (and between redirects).
limit - The total number of simultaneous connections.
limit_per_host - Number of simultaneous connections to one host.
loop - Optional event loop.
Fr] r Nc s8 t j d | d | d | d | d | | | _ d S)Nr[ rZ r\ r^ r1 )r r4 _path)r- pathr[ rZ r\ r^ r1 )r r2 r3 r4 s zUnixConnector.__init__c C s | j S)zPath to unix socket.)r )r- r2 r2 r3 r s zUnixConnector.pathc sk y* | j j | j | j Id H\ } } Wn: t k
rf } z t | j | | WYd d } ~ Xn X| S)N)r$ Zcreate_unix_connectionro r r r
r )r- r r r r r} r2 r2 r3 r s !(z UnixConnector._create_connection) rR rS rT rY r r4 rW r r r2 r2 )r r3 r s
)4ra rm r* r( rV collectionsr
contextlibr Z http.cookiesr itertoolsr r r r r r
Zclient_exceptionsr r r
r r r r r r Z client_protor Z
client_reqrepr r r r r r r Zlocksr r r r ImportError__all__r! rX r r r r r2 r2 r2 r3 <module> s: @ "
_ ( d