populse_db.engine package

This Python package contains engines that contains the minimum API to connect an external database engine to a Populse database.

class populse_db.engine.Engine

Bases: object

Base class for database engines. All methods of this base class raise NotImplementedError. This class exists to list all the methods that must be implemented by an engine.

Parameters passed to __init__ may differ between engine classes. It is up to engine_factory to extract appropriate parameter(s) from URL.

add_collection(collection, primary_key)

Create a new collection given the name of a field that will be created with a text type. This field is the primary key of each document. It means that, in a single collection, a value must uniquely identify a document. In other word, two documents cannot have the same value for this field.

Parameters:

• collection – collection name (str)

• primary_key – name of the primary key field (str)

add_field(collection, field, type, description, index)

Adds a new field in a collection.

Parameters:

• collection – collection name (str, must be existing)

• field – new field name (str, must not be existing)

• type – field type, in (‘string’, ‘int’, ‘float’, ‘boolean’, ‘date’, ‘datetime’, ‘time’, ‘json’, ‘list_string’, ‘list_int’, ‘list_float’, ‘list_boolean’, ‘list_date’, ‘list_datetime’, ‘list_time’, ‘list_json’)

• description – field description (str or None)

• index – boolean indicating if a database index must be created for this field to speed-ud queries involving values for this field.

clear()

Erase database data and schema.

collection(collection)

Returns a Row Object with at least the following items: collection_name: name of the collection primary_key: name of the primary key field More engine specific items may be present but they must not be used in the general API. Returns None if the collection does not exist.

Parameters:

collection – collection name (str)

collections()

Return an iterable (e.g. list, generator, etc.) browsing the collections presents in the database. See collection() method for the format of each collection object.

commit()

Store in the database all modifications done since the last call to either __enter__, commit or rollback.

document(collection, document_id, fields=None, as_list=False)

Returns a Row Object corresponding to a document in the collection.

The object has one item per selected fields. If fields is not given, fields returned by the fields() method are used. If as_list is True, a list of values is returned (one value per selected field). Returns None if the collection or document does not exist.

Parameters:

• collection – collection name (str)

• document_id – document identifier: the value of the primary key field (str)

• fields – list of fields to get values from (other fields are ignored) (list of str or None)

• as_list – if True, return a list of values instead of a Row Object (str)

field(collection, field)

Returns a Row Object corresponding to a collection field with at least the following items: collection_name: name of the collection field_name: field name field_type: type of values for this field description: text describing the field usage has_index: boolean indicating if an index was created for this field More engine specific items may be present but they must not be used in the general API. Returns None if the collection or the field does not exist.

Parameters:

• collection – collection name (str)

• field – field name (str)

fields(collection=None)

Return an iterable (e.g. list, generator, etc.) browsing the fields presents in the given collection (or all collections if collection parameter is None). See field() method for the format of each field object.

Parameters:

collection – collection name (str)

filter_documents(parsed_filter, fields=None, as_list=False)

Iterate over document selected by a filter. See document() method for the format of a document object.

Parameters:

• parsed_filter – internal object representing a filter on a collection (filter object returned by parse_filter())

• fields – list of fields to get values from (other fields are ignored) (list of str or None)

• as_list – if True, return a list of values insted of a Row Object (str)

has_collection(collection)

Checks existence of a collection. May be called often, must be fast.

Parameters:

collection – collection name (str)

has_document(collection, document_id)

Checks existence of a document in a collection.

Parameters:

• collection – collection name (str)

• document_id – document identifier: the value of the primary key field (str)

has_field(collection, field)

Checks existence of a field in a collection. May be called often, must be fast. Returns False if collection does not exist.

Parameters:

• collection – collection name (str)

• field – field name (str)

has_value(collection, document_id, field)

Check if a document has a not null value for a given field.

parse_filter(collection, filter)

Given a filter string, return a internal query representation that can be used with filter_documents() to select documents

Parameters:

• collection – the collection for which the filter is intended (str, must be existing)

• filter – the selection string using the populse_db selection language.

primary_key(collection)

Return the name of the primary key of a collection. Returns None if the collection does not exist.

Parameters:

collection – collection name (str)

remove_collection(collection)

Delete a collection and its data.

Parameters:

collection – collection name (str)

remove_document(collection, document_id)

Remove a document from a collection.

Parameters:

• collection – collection name (str)

• document_id – document identifier: the value of the primary key field (str)

remove_fields(collection, fields)

Remove given fields from a collection as well as all corresponding data.

Parameters:

• collection – collection name (str)

• fields – field name (str)

remove_value(collection, document_id, field)

Remove a value from a document (setting its value to None).

Parameters:

• collection – collection name (str)

• document_id – document identifier: the value of the primary key field (str)

• fields – field name (str)

rollback()

Store discards all database modifications done since the last call to either __enter__, commit or rollback.

set_values(collection, document_id, values)

Change some values in an existing document.

Parameters:

• collection – collection name (str)

• document_id – document identifier: the value of the primary key field (str)

• values – dictionary with field/value pairs (dict)

populse_db.engine.engine_factory(database_url)

Method that interprets an URL and creates the corresponding database engine.

Submodules

populse_db.engine.sqlite module

SQLite3 implementation of populse_db engine.

A populse_db engine is created when a DatabaseSession object is created (typically within a “with” statement)

class populse_db.engine.sqlite.FilterToSqliteQuery(engine, collection)

Bases: FilterToQuery

Implements required methods to produce a SQLite query given a document selection filter. This class returns either None (all documents are selected) or an SQL WHERE clause (without the WHERE keyword) as a list of string (that must be joined with spaces). This WHERE clause is useable with a SELECT from the table containing the collection documents. Using a list for combining strings is supposed to be more efficient (especially for long queries).

Create a parser for a givent engine and collection

build_condition_all()

Bla bla bla

build_condition_combine_conditions(left_condition, operator_str, right_condition)

Bla bla bla

build_condition_field_in_list(field, list_value)

Bla bla bla

build_condition_field_in_list_field(field, list_field)

Bla bla bla

build_condition_field_op_field(left_field, operator_str, right_field)

Bla bla bla

build_condition_field_op_value(field, operator_str, value)

BLa bla bla

build_condition_literal_in_list_field(value, list_field)

Bla bla bla

build_condition_negation(condition)

Bla bla bla

build_condition_value_op_field(value, operator_str, field)

Bla bla bla

get_column(field)

Returns:

The SQL representation of a field object.

get_column_value(python_value)

Converts a Python value to a value suitable to put in a database column

no_list_operator = {'<', '<=', '>', '>=', 'ilike', 'like'}

sql_operators = {'!=': 'IS NOT', '==': 'IS', 'ilike': 'LIKE'}

class populse_db.engine.sqlite.SQLiteEngine(database)

Bases: Engine

Bla bla bla

Parameters passed to __init__ may differ between engine classes. It is up to engine_factory to extract appropriate parameter(s) from URL.

add_collection(collection, primary_key)

Bla bla bla

add_document(collection, document, create_missing_fields)

Bla bla bla

add_field(collection, field, type, description, index)

Bla bla bla

clear()

Bla bla bla

collection(collection)

Bla bla bla

collections()

Bla bla bla

static column_to_python(field_type, value)

Bla bla bla

commit()

Bla bla bla

document(collection, document_id, fields=None, as_list=False)

Bla bla bla

field(collection, field)

Bla bla bla

fields(collection=None)

Bla bla bla

filter_documents(parsed_filter, fields=None, as_list=False)

Bla bla bla

has_collection(collection)

Bla bla bla

has_document(collection, document_id)

Bla bla bla

has_field(collection, field)

Bla bla bla

has_table(table)

Bla bla bla

has_value(collection, document_id, field)

Bla bla bla

static list_hash(list)

Bla bla bla

name_to_sql(name)

Transforms the name into a valid and unique SQLite table/column name. Since all names are quoted in SQL with ‘[]’, there is no restriction on character that can be used. However, case is insignificant in SQLite. Therefore, all upper case characters are prefixed with ‘!’.

Parameters:

name – Name (str)

Returns:

Valid and unique table/column name

parse_filter(collection, filter)

Given a filter string, return a internal query representation that can be used with filter_documents() to select documents

Parameters:

• collection – the collection for which the filter is intended (str, must be existing)

• filter – the selection string using the populse_db selection language.

primary_key(collection)

Bla bla bla

static python_to_column(field_type, value)

Converts a python value into a suitable value to put in a database column.

remove_collection(collection)

Bla bla bla

remove_document(collection, document_id)

Bla bla bla

remove_fields(collection, fields)

Bla bla bla

remove_value(collection, document_id, field)

BLa bla bla

rollback()

Bla bla bla

set_values(collection, document_id, values)

Bla bla bla

sql_type(type)

Bla bla bla

type_to_sql = {'boolean': 'BOOLEAN', 'date': 'TEXT', 'datetime': 'TEXT', 'float': 'REAL', 'int': 'INT', 'json': 'STRING', 'string': 'TEXT', 'time': 'TEXT'}