Viewing File: /opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pyo

ó
“tĆ_c@sBddlZddlZddlmZdefd„ƒYZdS(i˙˙˙˙Ni(tProbingStatet
CharSetProbercBs€eZdZd
d„Zd„Zed„ƒZd„Zed„ƒZ	d„Z
ed„ƒZed„ƒZ
ed	„ƒZRS(gffffffî?cCs(d|_||_tjtƒ|_dS(N(tNonet_statetlang_filtertloggingt	getLoggert__name__tlogger(tselfR((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pyt__init__'s		cCstj|_dS(N(Rt	DETECTINGR(R	((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytreset,scCsdS(N(R(R	((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytcharset_name/scCsdS(N((R	tbuf((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytfeed3scCs|jS(N(R(R	((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytstate6scCsdS(Ng((R	((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytget_confidence:scCstjdd|ƒ}|S(Ns([-])+t (tretsub(R((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytfilter_high_byte_only=scCsztƒ}tjd|ƒ}xX|D]P}|j|d ƒ|d}|jƒre|dkred}n|j|ƒq"W|S(s5
        We define three types of bytes:
        alphabet: english alphabets [a-zA-Z]
        international: international characters [€-˙]
        marker: everything else [^a-zA-Z€-˙]

        The input buffer can be thought to contain a series of words delimited
        by markers. This function works to filter all words that contain at
        least one international character. All contiguous sequences of markers
        are replaced by a single space ascii character.

        This filter applies to all scripts which do not use English characters.
        s%[a-zA-Z]*[€-˙]+[a-zA-Z]*[^a-zA-Z€-˙]?i˙˙˙˙s€R(t	bytearrayRtfindalltextendtisalpha(Rtfilteredtwordstwordt	last_char((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytfilter_international_wordsBs			

	cCsčtƒ}t}d}x˛tt|ƒƒD]ž}|||d!}|dkrTt}n|dkrit}n|dkr(|jƒr(||krš|rš|j|||!ƒ|jdƒn|d}q(q(W|sä|j||ƒn|S(sČ
        Returns a copy of ``buf`` that retains only the sequences of English
        alphabet and high byte characters that are not between <> characters.
        Also retains English alphabet and high byte characters immediately
        before occurrences of >.

        This filter can be applied to all scripts which contain both English
        characters and extended ASCII characters, but is currently only used by
        ``Latin1Prober``.
        iit>t<s€R(RtFalsetrangetlentTrueRR(RRtin_tagtprevtcurrtbuf_char((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pytfilter_with_english_lettersgs"			N(Rt
__module__tSHORTCUT_THRESHOLDRR
RtpropertyR
RRRtstaticmethodRRR)(((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pyR#s			%(RRtenumsRtobjectR(((sR/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.pyt<module>s
Back to Directory File Manager