populse_db.engine package

This Python package contains engines that contains the minimum API to connect an external database engine to a Populse database.

class populse_db.engine.Engine

Bases: object

Base class for database engines. All methods of this base class raise NotImplementedError. This class exists to list all the methods that must be implemented by an engine.

Parameters passed to __init__ may differ between engine classes. It is up to engine_factory to extract appropriate parameter(s) from URL.

add_collection(collection, primary_key)

Create a new collection given the name of a field that will be created with a text type. This field is the primary key of each document. It means that, in a single collection, a value must uniquely identify a document. In other word, two documents cannot have the same value for this field.

Parameters:

  • collection – collection name (str)

  • primary_key – name of the primary key field (str)

add_field(collection, field, type, description, index)

Adds a new field in a collection.

Parameters:

  • collection – collection name (str, must be existing)

  • field – new field name (str, must not be existing)

  • type – field type, in (‘string’, ‘int’, ‘float’, ‘boolean’, ‘date’, ‘datetime’, ‘time’, ‘json’, ‘list_string’, ‘list_int’, ‘list_float’, ‘list_boolean’, ‘list_date’, ‘list_datetime’, ‘list_time’, ‘list_json’)

  • description – field description (str or None)

  • index – boolean indicating if a database index must be created for this field to speed-ud queries involving values for this field.

clear()

Erase database data and schema.

collection(collection)

Returns a Row Object with at least the following items: collection_name: name of the collection primary_key: name of the primary key field More engine specific items may be present but they must not be used in the general API. Returns None if the collection does not exist.

Parameters:

collection – collection name (str)

collections()

Return an iterable (e.g. list, generator, etc.) browsing the collections presents in the database. See collection() method for the format of each collection object.

commit()

Store in the database all modifications done since the last call to either __enter__, commit or rollback.

document(collection, document_id, fields=None, as_list=False)

Returns a Row Object corresponding to a document in the collection. The object has one item per selected fields. If fields is not given, fields returned by the fields() method are used. If as_list is True, a list of values is returned (one value per selected field). Returns None if the collection or document does not exists.

Parameters:

  • collection – collection name (str)

  • document_id – document identifier: the value of the primary key field (str)

  • fields – list of fields to get values from (other fields are ignored) (list of str or None)

  • as_list – if True, return a list of values instead of a Row Object (str)

field(collection, field)

Returns a Row Object corresponding to a collection field with at least the following items: collection_name: name of the collection field_name: field name field_type: type of values for this field description: text describing the field usage has_index: boolean indicating if an index was created for this field More engine specific items may be present but they must not be used in the general API. Returns None if the collection or the field does not exist.

Parameters:

  • collection – collection name (str)

  • field – field name (str)

fields(collection=None)

Return an iterable (e.g. list, generator, etc.) browsing the fields presents in the given collection (or all collections if collection parameter is None). See field() method for the format of each field object.

Parameters:

collection – collection name (str)

filter_documents(parsed_filter, fields=None, as_list=False, distinct=False)

Iterate over document selected by a filter. See document() method for the format of a document object.

Parameters:

  • parsed_filter – internal object representing a filter on a collection (filter object returned by parse_filter())

  • fields – list of fields to get values from (other fields are ignored) (list of str or None)

  • as_list – if True, return a list of values instead of a Row Object (str)

  • distinct – if True, return only a series of different values.

has_collection(collection)

Checks existence of a collection. May be called often, must be fast.

Parameters:

collection – collection name (str)

has_document(collection, document_id)

Checks existence of a document in a collection.

Parameters:

  • collection – collection name (str)

  • document_id – document identifier: the value of the primary key field (str)

has_field(collection, field)

Checks existence of a field in a collection. May be called often, must be fast. Returns False if collection does not exists.

Parameters:

  • collection – collection name (str)

  • field – field name (str)

has_value(collection, document_id, field)

Check if a document has a not null value for a given field.

parse_filter(collection, filter)

Given a filter string, return a internal query representation that can be used with filter_documents() to select documents

Parameters:

  • collection – the collection for which the filter is intended (str, must be existing)

  • filter – the selection string using the populse_db selection language.

primary_key(collection)

Return the name of the primary key of a collection. Returns None if the collection does not exist.

Parameters:

collection – collection name (str)

remove_collection(collection)

Delete a collection and its data.

Parameters:

collection – collection name (str)

remove_document(collection, document_id)

Remove a document from a collection.

Parameters:

  • collection – collection name (str)

  • document_id – document identifier: the value of the primary key field (str)

remove_fields(collection, fields)

Remove given fields from a collection as well as all corresponding data.

Parameters:

  • collection – collection name (str)

  • fields – field name (str)

remove_value(collection, document_id, field)

Remove a value from a document (setting its value to None).

Parameters:

  • collection – collection name (str)

  • document_id – document identifier: the value of the primary key field (str)

  • fields – field name (str)

rollback()

Store discards all database modifications done since the last call to either __enter__, commit or rollback.

set_values(collection, document_id, values)

Change some values in an existing document.

Parameters:

  • collection – collection name (str)

  • document_id – document identifier: the value of the primary key field (str)

  • values – dictionary with field/value pairs (dict)

Submodules

populse_db.engine.sqlite module

class populse_db.engine.sqlite.ParsedFilter

Bases: str

class populse_db.engine.sqlite.SQLiteCollection(session, name)

Bases: DatabaseCollection

add(document, replace=False)
add_field(name, field_type, description=None, index=False, bad_json=False)

Adds a field to the database

Parameters:

  • collection – Field collection (str, must be existing)

  • name – Field name (str, must not be existing)

  • field_type – Field type, in (‘string’, ‘int’, ‘float’, ‘boolean’, ‘date’, ‘datetime’, ‘time’, ‘json’, ‘list_string’, ‘list_int’, ‘list_float’, ‘list_boolean’, ‘list_date’, ‘list_datetime’, ‘list_time’, ‘list_json’)

  • description – Field description (str or None) => None by default

  • index – Bool to know if indexing must be done => False by default

Raises:

ValueError –

  • If the collection does not exist

  • If the field already exists

  • If the field name is invalid

  • If the field type is invalid

  • If the field description is invalid

count(filter=None)
delete(filter)

Delete documents corresponding to the given filter

Parameters:

filter_query – Filter query (str)

document(document_id, fields=None, as_list=False)
documents(fields=None, as_list=False, distinct=False)
filter(filter, fields=None, as_list=False, distinct=False)

Iterates over the collection documents selected by filter_query

Each item yield is a row of the collection table returned

filter_query can be the result of self.filter_query() or a string containing a filter (in this case self.fliter_query() is called to get the actual query)

Parameters:

  • filter_query –

    Filter query (str)

    • A filter row must be written this way: {<field>} <operator> “<value>”

    • The operator must be in (‘==’, ‘!=’, ‘<=’, ‘>=’, ‘<’, ‘>’, ‘IN’, ‘ILIKE’, ‘LIKE’)

    • The filter rows can be linked with ‘ AND ‘ or ‘ OR ‘

    • Example: “((({BandWidth} == “50000”)) AND (({FileName} LIKE “%G1%”)))”

  • fields – List of fields to retrieve in the document

  • as_list – If True, document values are returned in a list using fields order

has_document(document_id)
parse_filter(filter)
remove_field(name)

Removes a specified field from the table and updates associated metadata.

Args:

name (str): The name of the field to remove.

Raises:

ValueError: If attempting to remove a primary key field. NotImplementedError: If the SQLite version is below 3.35.0,

which does not support removing columns.

update_document(document_id, partial_document)
class populse_db.engine.sqlite.SQLiteSession(sqlite_file, exclusive=False, timeout=None, echo_sql=None)

Bases: DatabaseSession

add_collection(name, primary_key='primary_key', catchall_column='_catchall')

Adds a collection

Parameters:

  • name – New collection name (str, must not be existing)

  • primary_key – New collection primary_key column (str) => “index” by default

Raises:

ValueError –

  • If the collection is already existing

  • If the collection name is invalid

  • If the primary_key is invalid

clear(keep_settings=False)

Erase the whole database content.

close(rollback=False)
commit()
database_exceptions = (<class 'sqlite3.OperationalError'>, <class 'sqlite3.IntegrityError'>)
execute(sql, data=None)
get_collection(name)

Deprecated since version 3.0: Use db_session[name] instead

has_collection(name)

Check if a collection with the given name exists.

remove_collection(name)

Removes a collection

Parameters:

name – Collection to remove (str, must be existing)

Raises:

ValueError – If the collection does not exist

rollback()
set_settings(category, key, value)
settings(category, key, default=None)
populse_db.engine.sqlite.create_sqlite_session_factory(url)
populse_db.engine.sqlite.sqlite_session_factory(sqlite_file, *args, create=False, **kwargs)