U
SW @ sd d Z ddlZddlZddlZddlZddlZddlZddlZz ddlZ W n e
k
rd ddl Z Y nX zddl m Z W n e
k
r Y nX ddl
T ddl
mZ ddl
mZ ddl
mZ ddl
mZ dd l
mZ dd
l
mZ dd l
mZ dd l
mZ dd
l
mZ ddl
mZ ddl
mZ ddl
mZ zddlmZ W n e
k
rN dZ Y nX ej d dkrde Z dZ dZ!ej"dd dkrdpdZ#dZ$G dd deZ%G dd d eZ&G d d d e'Z(G d d d e)Z*G d!d" d"e+Z,G d#d$ d$e-Z.G d%d& d&e.e+Z/G d'd( d(e.e0Z1G d)d* d*e.e2Z3G d+d, d,e.e4Z5G d-d. d.e6Z7G d/d0 d0e7Z8G d1d2 d2e8Z9d3Z:e;d4d5d6d7e<d8d9d:d;d<d=d>d?d@g
e;dAB e;e:B e;e:= B Z>e;dBdC e?dDD Z@eAdEZBG dFdG dGe8ZCd]dHdIZDG dJdK dKeZEejFdLdMdNdO ZGG dPdQ dQeHZIG dRdS dSeZJdTe_KeILejKdUi dVdT ZMdWdX ZNdYdZ ZOd[d\ ZPdS )^a
Sqlite3 extensions
==================
* Define custom aggregates, collations and functions
* Basic support for virtual tables
* Basic support for FTS3/4
* Specify isolation level in transactions
Example usage of the Full-text search:
class Document(FTSModel):
title = TextField() # type affinities are ignored in FTS
content = TextField()
Document.create_table(tokenize='porter') # use the porter stemmer
# populate the documents using normal operations.
for doc in documents:
Document.create(title=doc['title'], content=doc['content'])
# use the "match" operation for FTS queries.
matching_docs = Document.select().where(match(Document.title, 'some query'))
# to sort by best match, use the custom "rank" function.
best_docs = (Document
.select(Document, Document.rank('score'))
.where(match(Document.title, 'some query'))
.order_by(SQL('score')))
# or use the shortcut method.
best_docs = Document.match('some phrase')
N)
TableFunction)*)EnclosedClause)Entity)
Expression)Node)OP)SqliteQueryCompiler)_AutoPrimaryKeyField)sqlite3) transaction)_sqlite_date_part)_sqlite_date_trunc)_sqlite_regexp) _sqlite_ext ZpcnalxZpcx)r ZFTS4ZFTS3)r r c @ s e Zd ZdZdZdS )
RowIDFieldz
Field used to access hidden primary key on FTS5 or any other SQLite
table that does not have a separately-defined primary key.
rowidN__name__
__module__ __qualname____doc__Z _column_name r r /sqlite_ext.pyr O s r c @ s e Zd ZdZdZdS )
DocIDFieldz9Field used to access hidden primary key on FTS3/4 tables.docidNr r r r r r W s r c s e Zd ZdZ fddZ ZS ) PrimaryKeyAutoIncrementFielda
SQLite by default uses MAX(primary key) + 1 to set the ID on a new row.
Using the `AUTOINCREMENT` field, the IDs will increase monotonically
even if rows are deleted. Use this if you need to guarantee IDs are not
re-used in the event of deletion.
c s t t| |}|tdg S )NZ
AUTOINCREMENT)superr __ddl__SQL)selfZ column_typeZddl __class__r r r" c s z$PrimaryKeyAutoIncrementField.__ddl__)r r r r r"
__classcell__r r r% r r \ s r c @ s e Zd Zdd Zdd Zdd Zd dd Zd
d Zd d
Zdd Z dd Z
dd Z dd Z dd Z
d ddZd ddZd!d d ZdS )" JSONFieldc C s6 |d k r2z t |W S ttfk
r0 | Y S X d S N)jsonloads TypeError
ValueErrorr$ valuer r r python_valuei s
zJSONField.python_valuec C s |d k rt |S d S r) )r* dumpsr. r r r db_valuep s zJSONField.db_valuec C s | ds|sd| S d| S )N[z$%sz$.%s)
startswithr$ pathr r r
clean_patht s zJSONField.clean_pathNc C s |rt | | |S t | S r) )fnZjson_array_lengthr7 r5 r r r lengthy s zJSONField.lengthc C s t | | |S r) )r8 Z json_extractr7 r5 r r r extract~ s zJSONField.extractc C s$ t |tttfr tt|S |S r) )
isinstancelisttupledictr8 r* r1 r. r r r _value_for_insertion s z JSONField._value_for_insertionc C sl t |}|d dkr tdg }td|dD ]0}|| || || ||d q,|| f| S )N r z%Mismatched path and value parameters. )lenr- rangeappendr7 r? )r$ r8 pairsZnpairsaccumir r r _insert_like s zJSONField._insert_likec G s | tj|S r) )rH r8 Z json_insertr$ rE r r r insert s zJSONField.insertc G s | tj|S r) )rH r8 Z json_replacerI r r r replace s zJSONField.replacec G s | tj|S r) )rH r8 Zjson_setrI r r r set s z
JSONField.setc s t j f fdd|D S )Nc s g | ]} |qS r )r7 ).0r6 r$ r r
<listcomp> s z$JSONField.remove.<locals>.<listcomp>)r8 Z json_remove)r$ pathsr rN r remove s zJSONField.removec C s |rt | | |S t | S r) )r8 json_typer7 r5 r r r rR s zJSONField.json_typec C s |rt | | |S t | S )a?
Schema of `json_each` and `json_tree`:
key,
value,
type TEXT (object, array, string, etc),
atom (value for primitive/scalar types, NULL for array and object)
id INTEGER (unique identifier for element)
parent INTEGER (unique identifier of parent element or NULL)
fullkey TEXT (full path describing element)
path TEXT (path to the container of the current element)
json JSON hidden (1st input parameter to function)
root TEXT hidden (2nd input parameter, path at which to start)
)r8 Z json_eachr7 r5 r r r children s zJSONField.childrenc C s |rt | | |S t | S r) )r8 Z json_treer7 r5 r r r tree s zJSONField.tree)N)N)N)N)r r r r0 r2 r7 r9 r: r? rH rJ rK rL rQ rR rS rT r r r r r( h s
r( c s. e Zd ZdZd fdd Z fddZ ZS ) SearchFielda-
Field class to be used with full-text search extension. Since the FTS
extensions do not support any field types besides `TEXT`, and furthermore
do not support secondary indexes, using this field will prevent you
from mistakenly creating the wrong kind of field on your FTS table.
FNc s: d||d}|| _ |r$tdg|d< tt| jf | d S )NT)Znull db_columncoerceZ UNINDEXEDZ constraints)
_unindexedr# r! rU __init__)r$ Z unindexedrV rW _kwargsr% r r rY s
zSearchField.__init__c s t t| jf |}| j|_|S r) )r! rU
clone_baserX )r$ r[ cloner% r r r\ s zSearchField.clone_base)FNN)r r r r rY r\ r' r r r% r rU s rU c s e Zd ZdZ fddZ ZS )_VirtualFieldMixinzx
Field mixin to support virtual table attributes that may not correspond
to actual columns in the database.
c s" t t| || |j| d S r) )r! r^ add_to_class_metaZ remove_field)r$ model_classnamer% r r r_ s z _VirtualFieldMixin.add_to_class)r r r r r_ r' r r r% r r^ s r^ c @ s e Zd ZdS ) VirtualFieldNr r r r r r r rc s rc c @ s e Zd ZdS )VirtualIntegerFieldNrd r r r r re s re c @ s e Zd ZdS )VirtualCharFieldNrd r r r r rf s rf c @ s e Zd ZdS )VirtualFloatFieldNrd r r r r rg s rg c @ s4 e Zd ZG dd dZedd Zed ddZdS )
VirtualModelc @ s e Zd ZdZdZi ZdS )zVirtualModel.MetaTN)r r r Z
virtual_tableextension_moduleextension_optionsr r r r Meta s rk c K s |S r) r )clsoptionsr r r
clean_options s zVirtualModel.clean_optionsFc K s. |r| rd S | jjj| |d | d S )N)rm )Z table_existsr` database create_tableZ_create_indexes)rl Z
fail_silentlyrm r r r rp s zVirtualModel.create_tableN)F)r r r rk classmethodrn rp r r r r rh s
rh c @ s e Zd Zedd ZdS ) BaseFTSModelc K sB | d}| d}|r$d| |d< t|tr>|dkr>d|d< |S )Ntokenizecontentz"%s" z'')getr;
basestring)rl rm rs rt r r r rn s
zBaseFTSModel.clean_optionsN)r r r rq rn r r r r rr s rr c @ s e Zd ZdZe ZG dd dZedd Zedd Z edd Z
ed
d Z ed d
Z ed(ddZ
ed)ddZedd Zedd Zedd Zedd Zed d Zed*d"d#Zed+d$d%Zed,d&d'Zd S )-FTSModela_
VirtualModel class for creating tables that use either the FTS3 or FTS4
search extensions. Peewee automatically determines which version of the
FTS extension is supported and will use FTS4 if possible.
Note: because FTS5 is significantly different from FTS3 and FTS4, there is
a separate model class for FTS5 virtual tables.
c @ s e Zd ZeZdS )z
FTSModel.MetaN)r r r FTS_VERri r r r r rk s rk c C s | j jjdkrtdd S )Nr z:FTSModel classes must use the default `docid` primary key.)r` primary_keyrb ImproperlyConfiguredrl r r r validate_model s zFTSModel.validate_modelc C s( | j j}| j jd|||f }| S )Nz INSERT INTO %s(%s) VALUES('%s');)r` db_tablero execute_sqlZfetchone)rl cmdtblresr r r _fts_cmd s
zFTSModel._fts_cmdc C s
| dS )Noptimizer r| r r r r $ s zFTSModel.optimizec C s
| dS )Nrebuildr r| r r r r ( s zFTSModel.rebuildc C s
| dS )Nzintegrity-checkr r| r r r integrity_check, s zFTSModel.integrity_check c C s | d||f S )Nz merge=%s,%sr )rl ZblocksZsegmentsr r r merge0 s zFTSModel.mergeTc C s | d|rdpd S )Nz automerge=%s10r )rl stater r r automerge4 s zFTSModel.automergec C s t | |S zU
Generate a `MATCH` expression appropriate for searching this table.
match as_entityrl termr r r r 8 s zFTSModel.matchc G s t jt | tf| S r) )r8 fts_rank matchinfor FTS_MATCHINFO_FORMAT_SIMPLErl weightsr r r rank? s z
FTSModel.rankc G s t | t}t j|f| S r) )r8 r r FTS_MATCHINFO_FORMATfts_bm25rl r
match_infor r r bm25D s z
FTSModel.bm25c G s t | t}t j|f| S r) )r8 r r r
fts_lucener r r r luceneI s zFTSModel.lucenec
C s |s | }nJt |trNg }| jjD ] } ||| || jd q"|| }n|| }d}
|} |rp| ||f}
|r|st|} | j |
| | | S )N ?r )
r; r> r` declared_fieldsrD rv rb aliasr# selectwherer order_by) rl r r
with_score score_aliasZscore_fnexplicit_orderingr weight_argsfield selectionr r r r _searchN s.
zFTSModel._searchNFscorec C s | ||||| j|S 'Full-text search using selected `term`.)r r rl r r r r r r r r searchh s zFTSModel.searchc C s | ||||| j|S z:Full-text search for selected `term` using BM25 algorithm.)r r r r r r search_bm25t s zFTSModel.search_bm25c C s | ||||| j|S r )r r r r r r
search_lucene s zFTSModel.search_lucene)r r )T)NFr F)NFr F)NFr F)r r r r r r rk rq r} r r r r r r r r r r r r r r r r r r rx sR
rx Zabcdefghijklmnopqrstuvwxyz ," (){}r :rZ +
0123456789c C s g | ]}t |tkrt |qS r )chr _alphanum)rM pr r r rO s rO z(?:[^\s"]|"(?:\.|[^"])*")+c @ s e Zd ZdZe ZG dd dZddddZedd Z ed
d Z
e d d
Z e e
dfddZedd Zedd Zedd Zed-ddZed.d d Zed d Zed d! Zed"d# Zed$d% Zed&d' Zed(d) Zed/d+d,ZdS )0 FTS5Modelan
Requires SQLite >= 3.9.0.
Table options:
content: table name of external content, or empty string for "contentless"
content_rowid: column name of external content primary key
prefix: integer(s). Ex: '2' or '2 3 4'
tokenize: porter, unicode61, ascii. Ex: 'porter unicode61'
The unicode tokenizer supports the following parameters:
* remove_diacritics (1 or 0, default is 1)
* tokenchars (string of characters, e.g. '-_'
* separators (string of characters)
Parameters are passed as alternating parameter name and value, so:
{'tokenize': "unicode61 remove_diacritics 0 tokenchars '-_'"}
Content-less tables:
If you don't need the full-text content in it's original form, you can
specify a content-less table. Searches and auxiliary functions will work
as usual, but the only values returned when SELECT-ing can be rowid. Also
content-less tables do not support UPDATE or DELETE.
External content tables:
You can set up triggers to sync these, e.g.
-- Create a table. And an external content fts5 table to index it.
CREATE TABLE tbl(a INTEGER PRIMARY KEY, b);
CREATE VIRTUAL TABLE ft USING fts5(b, content='tbl', content_rowid='a');
-- Triggers to keep the FTS index up to date.
CREATE TRIGGER tbl_ai AFTER INSERT ON tbl BEGIN
INSERT INTO ft(rowid, b) VALUES (new.a, new.b);
END;
CREATE TRIGGER tbl_ad AFTER DELETE ON tbl BEGIN
INSERT INTO ft(fts_idx, rowid, b) VALUES('delete', old.a, old.b);
END;
CREATE TRIGGER tbl_au AFTER UPDATE ON tbl BEGIN
INSERT INTO ft(fts_idx, rowid, b) VALUES('delete', old.a, old.b);
INSERT INTO ft(rowid, b) VALUES (new.a, new.b);
END;
Built-in auxiliary functions:
* bm25(tbl[, weight_0, ... weight_n])
* highlight(tbl, col_idx, prefix, suffix)
* snippet(tbl, col_idx, prefix, suffix, ?, max_tokens)
c @ s e Zd ZdZdS )zFTS5Model.Metafts5Nr r r ri r r r r rk s rk zQBesides the implicit `rowid` column, all columns must be instances of SearchFieldz3Secondary indexes are not supported for FTS5 modelsz4FTS5 models must use the default `rowid` primary key)
field_typeindexpkc C sd | j jjdkr t| jd | j j D ] }t|tt fs(t| jd q(| j j
r`t| jd d S )Nr r r r ) r` rz rb r{ _error_messagesZfieldsvaluesr; rU r Zindexes)rl r r r r r} s zFTS5Model.validate_modelc C s t jd d tk rdS t d}z\z|d W nH zt d t d W n Y Y W dS X | jj d Y nX W 5 | X dS )Nr Fz:memory:z0CREATE VIRTUAL TABLE fts5test USING fts5 (data);Tr )
r sqlite_version_infoFTS5_MIN_VERSIONZconnectcloseexecuteenable_load_extensionload_extensionr` ro )rl Ztmp_dbr r r fts5_installed s
zFTS5Model.fts5_installedc C s@ t | }|D ],}|dr(|dr(qt|t@ r dS qdS )z
Simple helper function to indicate whether a search query is a
valid FTS5 query. Note: this simply looks at the characters being
used, and is not guaranteed to catch all problematic queries.
r FT) _quote_refindallr4 endswithrL _invalid_ascii)querytokenstokenr r r validate_query s
zFTS5Model.validate_queryr c C s g }d}t | }|D ]\}|dr:|dr:|| qt|}|t@ }|rhd}|D ]}|||}qV|| q|rd|S | S )z2
Clean a query of invalid tokens.
Fr Tr ) r r r4 r rD rL r rK join) r rK rF Z any_invalidr r Z token_setZinvalid_for_tokencr r r clean_query s"
zFTS5Model.clean_queryc C s t | |S r r r r r r r $ s zFTS5Model.matchc G s |r| j | S tdS d S )Nr )r r# )rl argsr r r r + s
zFTS5Model.rankc G s t j| f| S r) )r8 r r r r r r r 2 s zFTS5Model.bm25NFr c C s | t|||||S r )r r r r r r r r 6 s zFTS5Model.searchc
C s |st d}nbt|tr\g }| jjD ] }|||||jd q$tj |
f| }ntj |
f| }d} |}
|r| | |f} |r|st |}
| j |
| t||
S )r r r r )r# r; r> r` r rD rv rb r8 r r r r r r r r r ) rl r r r r r r r r r r r r r r A s.
zFTS5Model.search_bm25c
K sx | }|g}|g}| D ] \}}|t| || q t|}ttd| t| tdt| } | jj | S )Nz INSERT INTOZVALUES)
r itemsrD r r Clauser# r` ro r )
rl r Z extra_paramsr columnsr keyr/ Z inner_clauseclauser r r r \ s zFTS5Model._fts_cmdc C s, d| krdks n t d| jd|dS )Nr z level must be between 0 and 16r r )r- r )rl levelr r r r n s zFTS5Model.automergec C s | j d|dS )Nr r r )rl Znpagesr r r r t s zFTS5Model.mergec C s | j d|dS )Npgszr r )rl r r r r set_pgszx s zFTS5Model.set_pgszc C s | j d|dS )Nr r r )rl Zrank_expressionr r r set_rank| s zFTS5Model.set_rankc C s
| dS )Nz
delete-allr r| r r r
delete_all s zFTS5Model.delete_allrowc s dkrt dd }t |sG fddd}t t t t |d}dkrbt |d< d j }t |t|tf| t |S ) N)r colz)table_type must be either "row" or "col".z_vocab_model_%sc s8 e Zd Z jjZp jjd Ze e Z
dS )z"FTS5Model.VocabModel.<locals>.MetaZ_vN) r r r r` ro r~ r8 Z fts5vocabr r# ri r rl
table_name
table_typer r rk s rk )r docZcntr rk r z%sVocab)
r- hasattr BareField IntegerFieldr r setattrtyperh getattr)rl r r attrrk Zattrs
class_namer r r
VocabModel s
zFTS5Model.VocabModel)NFr F)NFr F)r N)r r r r r r rk r rq r} r staticmethodr r r r r r r r r r r r r r r r r r r r sX 7
r c s dkr4j j D ]}|jkr| q4qtdj jG fdddt}G fddd}dj }t||fd|iS )z3Model factory for the transitive closure extension.Nz,Unable to find self-referential foreign key.c s| e Zd Ze Ze Ze Ze Ze Ze Z
G dd dZ e d fdd Z
e d fdd Ze d
fd d
ZdS )z&ClosureTable.<locals>.BaseClosureTablec @ s e Zd ZdZdS )z+ClosureTable.<locals>.BaseClosureTable.MetaZtransitive_closureNr r r r r rk s rk NFc sb | jdj| | jkd| j|k}|d k rJ|| j|k}n|s^|| jdk}|S Ndepth)Zonr )r r r r idr rootrl noder include_noder ra rz r r descendants s
z2ClosureTable.<locals>.BaseClosureTable.descendantsc s^ | jdj| | jkd| j|k}|rF|| j|k}n|sZ|| jdk}|S r )r r r r r r r r r r r ancestors s
z0ClosureTable.<locals>.BaseClosureTable.ancestorsc s6 |j j} |k}|s2||k}|S r) )_datarv rb r r )rl r r Zfk_valuer foreign_keyra rz r r siblings s
z/ClosureTable.<locals>.BaseClosureTable.siblings)NF)NF)F)r r r re r r idcolumn parentcolumnr rf tablenamerk rq r r r r r r r BaseClosureTable s r c s0 e Zd ZjjZjjjjj jdZdZdS )zClosureTable.<locals>.Meta)r r r
FN) r r r r` ro r~ rz rV rj r )r
ra r r rk s rk z %sClosure) r` Zrelr Z rel_modelr- rz rh r r )ra r
Z field_objr rk rb r r r ClosureTable s
+
r c s4 e Zd ZdZd
fdd Zdd Zd dd Z ZS ) SqliteExtQueryCompilerzQ
Subclass of QueryCompiler that can be used to construct virtual tables.
FNc
s| t t| j||d}t|trpd}|jj}t|trX|j d d t
d|g }t | }qt|j dt
d| nd}|r|d7 }t
||j d< |
|||}|rX|j d } t| D ]\}
} t| tr| |d
k} n\t| rt| tr| } n>t| ttfr"t
d d tt| } nt| ts6t
| } t t
|
| } d
| _| j | qt |jdd rx|j t
d |S )N)safezCREATE VIRTUAL TABLEr@ ZUSINGzUSING %sz CREATE TABLEz IF NOT EXISTSr r z'%s'r =Z
without_rowidz
WITHOUT ROWID) r! r
_create_table
issubclassrh r` ri r; r Znodesr# r rJ rn sortedr ZFieldr inspectZisclassModelr< r= r mapstrgluerD r )
r$ ra r rm r Z statement extensionpartsZ
table_optionsZcolumns_constraintskvZoptionr% r r r sB
z$SqliteExtQueryCompiler._create_tablec C s> t |jdd }|r |jf |}ni }|r:||jf | |S )Nrj )r r` rn update)r$ ra r Z
extra_optionsZ
model_optionsrm r r r rn s z$SqliteExtQueryCompiler.clean_optionsc C s | | |||S r) )Z
parse_noder )r$ ra r rm r r r rp * s z#SqliteExtQueryCompiler.create_table)FN)FN)r r r r r rn rp r' r r r% r r s 6
r F)r] c C s t d| ddS )Nr ru )r )r rN r r r
disqualify. s r" c s e Zd ZdZeZd3 fdd Zedd Zdd Z d d
Z
d d Z d
d Z dd Z
d4ddZd5ddZd6ddZd7ddZd8dd Zd9d d Zd d Zd!d" Zd#d$ Zd%d& Zd'd( Zd)d* Zd:d,d-Zd; fd.d/ Zd<d1d2Z Z S )=SqliteExtDatabasez
Database class which provides additional Sqlite-specific functionality:
* Register custom aggregates, collations and functions
* Specify a row factory
* Advanced transactions (specify isolation level)
Tc s
t t| j|f|| i | _i | _i | _tg | _d | _t r|rd| _
| t j dd | t j
dd | t jdd | t jdd | t jdd | t jd d | t jd
d nLd | _
| tdd | tdd | tdd | tdd | td d d S )
NTZ date_partr@ Z
date_truncZregexpr r r r Z
murmurhashrA F)r! r# rY _aggregates _collations
_functionsrL _extensions _row_factory_c_ext_using_c_extensionsregister_functionZpeewee_date_partZpeewee_date_truncZ
peewee_regexpZ peewee_rankZ
peewee_luceneZ peewee_bm25Zpeewee_murmurhashr
r r r r )r$ ro Z c_extensionsr r[ r% r r rY ? s*
zSqliteExtDatabase.__init__c C s | j S r) )r* rN r r r using_c_extensionsW s z$SqliteExtDatabase.using_c_extensionsc C sJ | | | | | | | | | jr6| j|_| jrF| | d S r) )Z _set_pragmas_load_aggregates_load_collations_load_functionsr( row_factoryr' _load_extensions)r$ connr r r _add_conn_hooks[ s
z!SqliteExtDatabase._add_conn_hooksc C s* | j D ]\}\}}|||| q
d S r) )r$ r Zcreate_aggregate)r$ r2 rb klass
num_paramsr r r r- e s z"SqliteExtDatabase._load_aggregatesc C s$ | j D ]\}}||| q
d S r) )r% r Zcreate_collation)r$ r2 rb r8 r r r r. i s z"SqliteExtDatabase._load_collationsc C s* | j D ]\}\}}|||| q
d S r) )r& r Zcreate_function)r$ r2 rb r8 r5 r r r r/ m s z!SqliteExtDatabase._load_functionsc C s$ | d | jD ]}|| qd S NT)r r' r )r$ r2 r r r r r1 q s
z"SqliteExtDatabase._load_extensionsNr c C s2 ||f| j |p|j < | s.| | d S r) )r$ r lower is_closedr- get_conn)r$ r4 rb r5 r r r register_aggregatev s z$SqliteExtDatabase.register_aggregatec s fdd}|S )Nc s | | S r) )r: )r4 rb r5 r$ r r decorator| s z.SqliteExtDatabase.aggregate.<locals>.decoratorr r$ rb r5 r< r r; r aggregate{ s zSqliteExtDatabase.aggregatec s@ p|j fdd}||_|| j < | s<| | d S )Nc s | t d f }t| S )Nz
collate %s)r# r )r Z expressionsrb r r
_collation s z8SqliteExtDatabase.register_collation.<locals>._collation)r collationr% r8 r. r9 )r$ r8 rb r@ r r? r register_collation s
z$SqliteExtDatabase.register_collationc s fdd}|S )Nc s | | S r) )rB r8 rb r$ r r r< s z.SqliteExtDatabase.collation.<locals>.decoratorr )r$ rb r< r rD r rA s zSqliteExtDatabase.collationc C s. ||f| j |p|j< | s*| | d S r) )r& r r8 r/ r9 )r$ r8 rb r5 r r r r+ s z#SqliteExtDatabase.register_functionc s fdd}|S )Nc s | | S r) )r+ rC r; r r r< s z)SqliteExtDatabase.func.<locals>.decoratorr r= r r; r func s zSqliteExtDatabase.funcc C s4 | j | | s0| }|d || d S r6 )r' addr8 r9 r r )r$ r r2 r r r r s
z SqliteExtDatabase.load_extensionc C s | j |= d S r) )r$ r$ rb r r r unregister_aggregate s z&SqliteExtDatabase.unregister_aggregatec C s | j |= d S r) )r% rG r r r unregister_collation s z&SqliteExtDatabase.unregister_collationc C s | j |= d S r) )r& rG r r r unregister_function s z%SqliteExtDatabase.unregister_functionc C s | j | d S r) )r' rQ )r$ r r r r unload_extension s z"SqliteExtDatabase.unload_extensionc C s
|| _ d S r) )r( )r$ r8 r r r r0 s z SqliteExtDatabase.row_factoryFc C s" | |||\}}| ||S r) )Zcompilerrp r )r$ ra r rm ZsqlZparamsr r r rp s z SqliteExtDatabase.create_tablec s" t |trd S tt| |||S r) )r rx r! r# create_index)r$ ra Z
field_nameuniquer% r r rL s
z SqliteExtDatabase.create_indexdeferredc C s | dkstt| |S )N)rN Z immediateZ exclusive)r7 AssertionErrorgranular_transaction)r$ lock_typer r r rP s z&SqliteExtDatabase.granular_transaction)T)Nr )Nr )N)N)Nr )Nr )FN)F)rN ) r r r r r Zcompiler_classrY propertyr, r3 r- r. r/ r1 r: r> rB rA r+ rE r rH rI rJ rK r0 rp rL rP r' r r r% r r# 5 s2
r# c @ s e Zd ZdddZdd ZdS )rP rN c C s || _ | j | _|| _d S r) )dbr9 r2 rQ )r$ rS rQ r r r rY s z granular_transaction.__init__c C s | j | j d S r) )rS ZbeginrQ rN r r r _begin s zgranular_transaction._beginN)rN )r r r rY rT r r r r rP s
rP r MATCHc C s t | tj|S r) )r r rU )Zlhsrhsr r r r s c s" t } fddtd|dD S )Nc s( g | ] }t d ||d d qS )z@Ir r )structunpack)rM rG bufr r rO s z%_parse_match_info.<locals>.<listcomp>r r )rB rC )rZ bufsizer rY r _parse_match_info s r\ c G s t | }d}|d d \}}|s,dg| }n$dg| }t|D ]\}}|||< q>t|D ]l}d|| d } t|D ]N}
||
}|sqt| |
d } || | d \} }
| dkrt||t| |
7 }qtqX| S )N r@ rA r r )r\ enumeraterC float)raw_match_infor r r r r rG weightZ
phrase_numZphrase_info_idxZcol_numZcol_idxZx1Zx2r r r r s&
r c G s t | }d}d}d}td\}}}} || }
|| } || } | | }
|
| }|s\dg| }n(dg| }t|D ]\}}|| ||< qnt|
D ]}t| D ]}|| }|dkrqt|| | }t||
| }|dkrd}nd| ||| }|d| |d }t|| }t||d }tt| | d |d d}||| }|dkr\d}n||d | }||| | 7 }qq| S )
z
Usage:
# Format string *must* be pcnalx
# Second parameter to bm25 specifies the index of the column, on
# the table being queries.
bm25(matchinfo(document_tbl, 'pcnalx'), 1) AS rank
g333333?g ?r] r rA r r r@ g ?)r\ rC r^ r_ maxmathlog) r` r r KBr ZP_OZC_OZN_OZA_OZ
term_countZ col_countZ
total_docsZL_OZX_Or rG ra jZ
avg_lengthZ
doc_lengthDxZterm_frequencyZdocs_with_termZidfZdenomrV r r r r sT
r )N)Qr Zglobr rc osrerW sysZ
simplejsonr* ImportErrorZvtfuncr Zpeeweer r r r r r r
r r r
r r Z playhouser r) version_infor rw r r r ry r r r ZPrimaryKeyFieldr Z TextFieldr( r rU objectr^ rc r re Z CharFieldrf Z
FloatFieldrg r rh rr rx Z _alphabetrL r upperr rC r compiler r r r extendr" ZSqliteDatabaser# rP rU Z register_opsr r\ r r r r r r <module> s !
R
CH