populse_db package

The database API of the populse project.

Subpackages

Submodules

populse_db.database module

Bla bla bla

class populse_db.database.Database(database_url, caches=None, list_tables=None, query_type=None)

Bases: object

Database API

attributes:

  • string_engine: String engine of the database

  • engine: database engine

methods:

  • __enter__: Creates or gets a DatabaseSession instance

  • __exit__: Releases the latest created DatabaseSession

  • clear: Clears the database

Initialization of the database.

Parameters:

  • database_url –

    Database engine. The engine is constructed this way: dialect://user:password@host/dbname[?key=value…]

    The dialect can be sqlite or postgresql.

    For sqlite databases, the file can be not existing yet, it will be created in this case

    Examples:

    • ”sqlite:///foo.db”

    • ”postgresql://scott:tiger@localhost/test”

  • caches – obsolete parameter kept for backward compatibility

  • list_tables – obsolete parameter kept for backward compatibility

  • query_type – obsolete parameter kept for backward compatibility

Raises:

ValueError –

  • If database_url is invalid

  • If the schema is not coherent with the API (the database is not a populse_db database)

clear()

Removes all documents and collections in the database

database_session_class

alias of DatabaseSession

class populse_db.database.DatabaseSession(database)

Bases: object

DatabaseSession API

attributes:

  • database: Database instance

  • session: Session related to the database

  • table_classes: List of all table classes, generated automatically

  • base: Database base

  • metadata: Database metadata

methods:

  • add_collection: Adds a collection

  • remove_collection: Removes a collection

  • get_collection: Gives the collection row

  • get_collections: Gives all collection rows

  • get_collections_names: Gives all collection names

  • add_field: Adds a field to a collection

  • add_fields: Adds a list of fields to a collection

  • remove_field: Removes a field from a collection

  • get_field: Gives all fields rows given a collection

  • get_fields_names: Gives all fields names given a collection to the name

  • get_value: Gives the value of <collection, document, field>

  • set_value: Sets the value of <collection, document, field>

  • set_values: Sets several values of <collection, document, field>

  • remove_value: Removes the value of <collection, document, field>

  • add_value: Adds a value to <collection, document, field>

  • get_document: Gives the document row given a document name and

    a collection

  • get_documents: Gives all document rows given a collection

  • get_documents_names: Gives all document names given a collection

  • add_document: Adds a document to a collection

  • remove_document: Removes a document from a collection

  • save_modifications: Saves the pending modifications

  • unsave_modifications: Unsaves the pending modifications

  • has_unsaved_modifications: To know if there are unsaved modifications

  • filter_documents: Gives the list of documents matching the filter

Creates a session API of the Database instance

Parameters:

database – Database instance to take into account

add_collection(name, primary_key='index')

Adds a collection

Parameters:

  • name – New collection name (str, must not be existing)

  • primary_key – New collection primary_key column (str) => “index” by default

Raises:

ValueError –

  • If the collection is already existing

  • If the collection name is invalid

  • If the primary_key is invalid

add_document(collection, document, create_missing_fields=True, flush=None)

Adds a document to a collection

Parameters:

  • collection – Document collection (str, must be existing)

  • document –

    Dictionary of document values (dict), or document primary_key (str)

    • The primary_key must not be existing

  • create_missing_fields –

    Boolean to know if the missing fields

    must be created

    • If True, fields that are in the document but not in the collection are created if the type can be guessed from the value in the document (possible for all valid values except None and []).

  • flush – ignored obsolete parameter

Raises:

ValueError –

  • If the collection does not exist

  • If the document already exists

  • If document is invalid (invalid name or no primary_key)

add_field(collection, name, field_type, description=None, index=False, flush=None)

Adds a field to the database

Parameters:

  • collection – Field collection (str, must be existing)

  • name – Field name (str, must not be existing)

  • field_type – Field type, in (‘string’, ‘int’, ‘float’, ‘boolean’, ‘date’, ‘datetime’, ‘time’, ‘json’, ‘list_string’, ‘list_int’, ‘list_float’, ‘list_boolean’, ‘list_date’, ‘list_datetime’, ‘list_time’, ‘list_json’)

  • description – Field description (str or None) => None by default

  • index – Bool to know if indexing must be done => False by default

  • flush – obsolete parameter kept for backward compatibility

Raises:

ValueError –

  • If the collection does not exist

  • If the field already exists

  • If the field name is invalid

  • If the field type is invalid

  • If the field description is invalid

add_fields(fields)

Adds the list of fields

Parameters:

fields – List of fields: [collection, name, type, description]

add_value(collection, document_id, field, value, checks=True)

Adds a value for <collection, document_id, field>

Parameters:

  • collection – Document collection (str, must be existing)

  • document_id – Document name (str, must be existing)

  • field – Field name (str, must be existing)

  • value – Value to add

  • checks – if False, do not perform any type or value checking

Raises:

ValueError –

  • If the collection does not exist

  • If the field does not exist

  • If the document does not exist

  • If the value is invalid

  • If <collection, document_id, field> already has a value

classmethod check_value_type(value, field_type)

Checks the type of the value

Parameters:

  • value – Value

  • field_type – Type that the value is supposed to have

Returns:

True if the value is None or has a valid type, False otherwise

commit()

Store in the database all modifications done since the last call

filter_documents(collection, filter_query, fields=None, as_list=False)

Iterates over the collection documents selected by filter_query

Each item yield is a row of the collection table returned

filter_query can be the result of self.filter_query() or a string containing a filter (in this case self.filter_query() is called to get the actual query)

Parameters:

  • collection – Filter collection (str, must be existing)

  • filter_query –

    Filter query (str)

    • A filter row must be written this way: {<field>} <operator> “<value>”

    • The operator must be in (‘==’, ‘!=’, ‘<=’, ‘>=’, ‘<’, ‘>’, ‘IN’, ‘ILIKE’, ‘LIKE’)

    • The filter rows can be linked with ‘ AND ‘ or ‘ OR ‘

    • Example: “((({BandWidth} == “50000”)) AND (({FileName} LIKE “%G1%”)))”

get_collection(name)

Returns the collection row of the collection

Parameters:

name – Collection name (str, must be existing)

Returns:

The collection row if it exists, None otherwise

get_collections()

Gives the list of all collection rows

Returns:

List of all collection rows

get_collections_names()

Gives the list of all collection names

Returns:

List of all collection names

get_document(collection, document_id, fields=None, as_list=False)

Gives a Document instance given a collection and a document identifier

Parameters:

  • collection – Document collection (str, must be existing)

  • document_id – Document name (str, must be existing)

Returns:

The document row if the document exists, None otherwise

get_documents(collection, fields=None, as_list=False, document_ids=None)

Gives the list of all document rows, given a collection

Parameters:

collection – Documents collection (str, must be existing)

Returns:

List of all document rows of the collection if it exists, None otherwise

get_documents_names(collection)

Gives the list of all document names, given a collection

Parameters:

collection – Documents collection (str, must be existing)

Returns:

List of all document names of the collection if it exists, None otherwise

get_field(collection, name)

Gives the field row, given a field name and a collection

Parameters:

  • collection – Document collection (str, must be existing)

  • name – Field name (str, must be existing)

Returns:

The field row if the field exists, None otherwise

get_fields(collection)

Gives the list of all fields rows, given a collection

Parameters:

collection – Fields collection (str, must be existing)

Returns:

List of all fields rows of the collection if it exists, None otherwise

get_fields_names(collection)

Gives the list of all fields, given a collection

Parameters:

collection – Fields collection (str, must be existing)

Returns:

List of all fields names of the collection if it exists, None otherwise

get_value(collection, document_id, field)

Gives the current value of <collection, document, field>

Parameters:

  • collection – Document collection (str, must be existing)

  • document_id – Document name (str, must be existing)

  • field – Field name (str, must be existing)

Returns:

The current value of <collection, document, field> if it exists, None otherwise

has_document(collection, document_id)

Checks existence of a document in a collection.

remove_collection(name)

Removes a collection

Parameters:

name – Collection to remove (str, must be existing)

Raises:

ValueError – If the collection does not exist

remove_document(collection, document_id)

Removes a document in the collection

Parameters:

  • collection – Document collection (str, must be existing)

  • document_id – Document name (str, must be existing)

Raises:

ValueError –

  • If the collection does not exist

  • If the document does not exist

remove_field(collection, fields)

Removes a field in the collection

Parameters:

  • collection – Field collection (str, must be existing)

  • field – Field name (str, must be existing), or list of fields (list of str, must all be existing)

Raises:

ValueError –

  • If the collection does not exist

  • If the field does not exist

remove_value(collection, document_id, field, flush=None)

Removes the value <collection, document, field> if it exists

Parameters:

  • collection – Document collection (str, must be existing)

  • document_id – Document name (str, must be existing)

  • field – Field name (str, must be existing)

  • flush – unused obsolete parameter

Raises:

ValueError –

  • If the collection does not exist

  • If the field does not exist

  • If the document does not exist

rollback()

Rejects all changes made to the database since the last call

set_value(collection, document_id, field, new_value, flush=None)

Sets the value associated to <collection, document, field> if it exists

Parameters:

  • collection – Document collection (str, must be existing)

  • document_id – Document name (str, must be existing)

  • field – Field name (str, must be existing)

  • new_value – New value

  • flush – unused obsolete parameter

Raises:

ValueError –

  • If the collection does not exist

  • If the field does not exist

  • If the document does not exist

  • If the value is invalid

  • If trying to set the primary_key

set_values(collection, document_id, values, flush=None)

Sets the values of a <collection, document, field> if it exists

Parameters:

  • collection – Document collection (str, must be existing)

  • document_id – Document name (str, must be existing)

  • values – Dict of values (key=field, value=value)

  • flush – unused obsolete parameter

Raises:

ValueError –

  • If the collection does not exist

  • If the field does not exist

  • If the document does not exist

  • If the values are invalid

  • If trying to set the primary_key

class populse_db.database.DictList(keys, values)

Bases: ListWithKeys

Bla bla bla

Initialize values with their position (args) or name (kwargs)

class populse_db.database.ListWithKeys(*args, **kwargs)

Bases: object

Represents a list of value of fixed size with a key string for each value. It allows to access to values with their index or with their key. It is also possible to access to values as attributes. The function list_with_keys() is used to create derived classes with a fixed set of item names.

Initialize values with their position (args) or name (kwargs)

classmethod keys()

Return the keys

populse_db.database.list_with_keys(name, keys)

Return a new instance of ListWithNames with a given list of keys

populse_db.database.python_value_type(value)

Returns the field type corresponding to a Python value. This type can be used in add_field(s) method. For list values, only the first item is considered to get the type. Type cannot be determined for empty list. If value is None, the result is None.

populse_db.filter module

Bla bla bla

class populse_db.filter.FilterToQuery(engine, collection)

Bases: Transformer

Instance of this class are passed to Lark parser when parsing a document selection filter string in order to create an object that can be used to select items from the database. FilterToQuery implements methods that are common to all engines and does not produce anything because the query objectis specific to each engine. Therefore, engine class must use a subclass of FilterToQuery that implements the following methods:

build_condition_all build_condition_literal_in_list_field build_condition_field_in_list_field build_condition_field_in_list build_condition_field_op_field build_condition_value_op_field build_condition_negation build_condition_combine_conditions

all(items)

Return a selection query that select all documents.

build_condition_all()

Return a selection query that select all documents. This query is directly given to the engine and never combined with other queries.

build_condition_combine_conditions(left_condition, operator_str, right_condition)

Builds a condition that combines two conditions with an operator.

Parameters:

  • left_condition – condition object returned by one of the build_condition_*() method (except build_condition_all)

  • operator_str – string containing one of the BOOLEAN_OPERATOR defined in the grammar (in lowercase)

  • right_condition – condition object returned by one of the build_condition_*() method (except build_condition_all)

build_condition_field_in_list(field, list_value)

Builds a condition checking if a field value is a constant list value

Parameters:

  • field – field object as returned by Database.get_field

  • list_value – Pyton list containing literals

build_condition_field_in_list_field(field, list_field)

Builds a condition checking if a field value is in another list field value

Parameters:

  • field – field object as returned by Database.get_field

  • list_field – field object as returned by Database.get_field

build_condition_field_op_field(left_field, operator_str, right_field)

Builds a condition comparing the content of two fields with an operator.

Parameters:

  • left_field – field object as returned by Database.get_field

  • operator – string containing one of the CONDITION_OPERATOR defined in the grammar (in lowercase)

  • right_field – field object as returned by Database.get_field

build_condition_field_op_value(field, operator_str, value)

Builds a condition comparing the content of a field with a constant value using an operator.

Parameters:

  • field – field object as returned by Database.get_field

  • operator_str – string containing one of the CONDITION_OPERATOR defined in the grammar (in lowercase)

  • value – Python value (None, string number, boolean or date/time)

build_condition_literal_in_list_field(value, list_field)

Builds a condition checking if a constant value is in a list field

Parameters:

  • value – Python literal

  • list_field – field object as returned by Database.get_field

build_condition_negation(condition)

Builds a condition inverting another condition.

Parameters:

condition – condition object returned by one of the build_condition_*() method (except build_condition_all)

build_condition_value_op_field(value, operator_str, field)

Builds a condition comparing a constant value with the content of a field withusing an operator.

Parameters:

  • value – Python value (None, string number, boolean or date/time)

  • operator_str – string containing one of the CONDITION_OPERATOR defined in the grammar (in lowercase)

  • field – field object as returned by Database.get_field

condition(items)

Bla bla bla

conditions(items)

Bla bla bla

date(items)

Bla bla bla

datetime(items)

Bla bla bla

empty_list(items)

Bla bla bla

field_name(items)

Bla bla bla

static is_field(object)

Checks if an object is a populse_db.database column object

static is_list_field(field)

Check if the field is a list.

keyword_literal(items)

Bla bla bla

keyword_literals = {'false': False, 'null': None, 'true': True}
list(items)

Bla bla bla

negation(items)

Bla bla bla

number(items)

Bla bla bla

quoted_field_name(items)

Bla bla bla

string(items)

Bla bla bla

time(items)

Bla bla bla

populse_db.filter.filter_parser()
Returns:

A singleton instance of Lark grammar parser for filter expression

populse_db.filter.literal_parser()

This is used to test literals parsing

Returns:

An instance of Lark grammar parser for parsing only a literal value (int, string, list, date, etc.) from a filter expression. This is used for testing the parsing of these literals.

populse_db.info module

The module dedicated to the main information on populse_db.

The info.py module is mainly used by the setup.py module.

populse_db.test module

This module is dedicated to populse_db unit tests.

Contains:
Class:

  • TestsSQLiteInMemory

  • TestDatabaseMethods

class populse_db.test.TestsSQLiteInMemory(methodName='runTest')

Bases: TestCase

Bla bla bla

Create an instance of the class that will use the named test method when executed. Raises a ValueError if the instance does not have a method with the specified name.

test_add_get_document()

Bla bla bla

populse_db.test.create_test_case(**database_creation_parameters)

Bla bla bla

populse_db.test.load_tests(loader, standard_tests, pattern)

Prepares the tests parameters

Parameters:

  • loader –

  • standard_tests –

  • pattern –

Returns:

A test suite