Database Backend Interfaces

Generic backend database interfaces expected by BigchainDB.

The interfaces in this module allow BigchainDB to be agnostic about its database backend. One can configure BigchainDB to use different databases as its data store by setting the database.backend property in the configuration or the BIGCHAINDB_DATABASE_BACKEND environment variable.

Generic Interfaces

bigchaindb.backend.connection

bigchaindb.backend.connection.connect(backend=None, host=None, port=None, name=None, max_tries=None, connection_timeout=None, replicaset=None, ssl=None, login=None, password=None, ca_cert=None, certfile=None, keyfile=None, keyfile_passphrase=None, crlfile=None)

Create a new connection to the database backend.

All arguments default to the current configuration’s values if not given.

Parameters:

• backend (str) – the name of the backend to use.
• host (str) – the host to connect to.
• port (int) – the port to connect to.
• name (str) – the name of the database to use.
• replicaset (str) – the name of the replica set (only relevant for MongoDB connections).

Returns:

An instance of Connection based on the given (or defaulted) backend.

Raises:

• ConnectionError – If the connection to the database fails.
• ConfigurationError – If the given (or defaulted) backend is not supported or could not be loaded.
• AuthenticationError – If there is a OperationFailure due to Authentication failure after connecting to the database.

class bigchaindb.backend.connection.Connection(host=None, port=None, dbname=None, connection_timeout=None, max_tries=None, **kwargs)

Connection class interface.

All backend implementations should provide a connection class that inherits from and implements this class.

__init__(host=None, port=None, dbname=None, connection_timeout=None, max_tries=None, **kwargs)

Create a new Connection instance.

Parameters:

• host (str) – the host to connect to.
• port (int) – the port to connect to.
• dbname (str) – the name of the database to use.
• connection_timeout (int, optional) – the milliseconds to wait until timing out the database connection attempt. Defaults to 5000ms.
• max_tries (int, optional) – how many tries before giving up, if 0 then try forever. Defaults to 3.
• **kwargs – arbitrary keyword arguments provided by the configuration’s database settings

run(query)

Run a query.

Parameters:

query – the query to run

Raises:

• DuplicateKeyError – If the query fails because of a duplicate key constraint.
• OperationFailure – If the query fails for any other reason.
• ConnectionError – If the connection to the database fails.

connect()

Try to connect to the database.

Raises:ConnectionError – If the connection to the database fails.

bigchaindb.backend.changefeed

Changefeed interfaces for backends.

class bigchaindb.backend.changefeed.ChangeFeed(table, operation, *, prefeed=None, connection=None)

Create a new changefeed.

It extends multipipes.Node to make it pluggable in other Pipelines instances, and makes usage of self.outqueue to output the data.

A changefeed is a real time feed on inserts, updates, and deletes, and is volatile. This class is a helper to create changefeeds. Moreover, it provides a way to specify a prefeed of iterable data to output before the actual changefeed.

run_forever()

Main loop of the multipipes.Node

This method is responsible for first feeding the prefeed to the outqueue and after that starting the changefeed and recovering from any errors that may occur in the backend.

run_changefeed()

Backend specific method to run the changefeed.

The changefeed is usually a backend cursor that is not closed when all the results are exausted. Instead it remains open waiting for new results.

This method should also filter each result based on the operation and put all matching results on the outqueue of multipipes.Node.

bigchaindb.backend.changefeed.get_changefeed(connection, table, operation, *, prefeed=None)

Return a ChangeFeed.

Parameters:

• connection (Connection) – A connection to the database.
• table (str) – name of the table to listen to for changes.
• operation (int) – can be ChangeFeed.INSERT, ChangeFeed.DELETE, or ChangeFeed.UPDATE. Combining multiple operation is possible with the bitwise | operator (e.g. ChangeFeed.INSERT | ChangeFeed.UPDATE)
• prefeed (iterable) – whatever set of data you want to be published first.

bigchaindb.backend.query

Query interfaces for backends.

bigchaindb.backend.query.write_transaction(connection, signed_transaction)

Write a transaction to the backlog table.

Parameters:signed_transaction (dict) – a signed transaction. Returns:The result of the operation.

bigchaindb.backend.query.update_transaction(connection, transaction_id, doc)

Update a transaction in the backlog table.

Parameters:

• transaction_id (str) – the id of the transaction.
• doc (dict) – the values to update.

Returns:

The result of the operation.

bigchaindb.backend.query.delete_transaction(connection, *transaction_id)

Delete a transaction from the backlog.

Parameters:*transaction_id (str) – the transaction(s) to delete. Returns:The database response.

bigchaindb.backend.query.get_stale_transactions(connection, reassign_delay)

Get a cursor of stale transactions.

Transactions are considered stale if they have been assigned a node, but are still in the backlog after some amount of time specified in the configuration.

Parameters:reassign_delay (int) – threshold (in seconds) to mark a transaction stale. Returns:A cursor of transactions.

bigchaindb.backend.query.get_transaction_from_block(connection, transaction_id, block_id)

Get a transaction from a specific block.

Parameters:

• transaction_id (str) – the id of the transaction.
• block_id (str) – the id of the block.

Returns:

The matching transaction.

bigchaindb.backend.query.get_transaction_from_backlog(connection, transaction_id)

Get a transaction from backlog.

Parameters:transaction_id (str) – the id of the transaction. Returns:The matching transaction.

bigchaindb.backend.query.get_blocks_status_from_transaction(connection, transaction_id)

Retrieve block election information given a secondary index and value.

Parameters:

• value – a value to search (e.g. transaction id string, payload hash string)
• index (str) – name of a secondary index, e.g. ‘transaction_id’

Returns:

A list of blocks with with only election information

Return type:

list of dict

bigchaindb.backend.query.get_asset_by_id(conneciton, asset_id)

Returns the asset associated with an asset_id.

Parameters:asset_id (str) – The asset id. Returns:Returns a rethinkdb cursor.

bigchaindb.backend.query.get_spent(connection, transaction_id, condition_id)

Check if a txid was already used as an input.

A transaction can be used as an input for another transaction. Bigchain needs to make sure that a given txid is only used once.

Parameters:

• transaction_id (str) – The id of the transaction.
• condition_id (int) – The index of the condition in the respective transaction.

Returns:

The transaction that used the txid as an input else None

bigchaindb.backend.query.get_spending_transactions(connection, inputs)

Return transactions which spend given inputs

Parameters:inputs (list) – list of {txid, output} Returns:Iterator of (block_ids, transaction) for transactions that spend given inputs.

bigchaindb.backend.query.get_owned_ids(connection, owner)

Retrieve a list of txids that can we used has inputs.

Parameters:owner (str) – base58 encoded public key. Returns:Iterator of (block_id, transaction) for transactions that list given owner in conditions.

bigchaindb.backend.query.get_votes_by_block_id(connection, block_id)

Get all the votes casted for a specific block.

Parameters:block_id (str) – the block id to use. Returns:A cursor for the matching votes.

bigchaindb.backend.query.get_votes_by_block_id_and_voter(connection, block_id, node_pubkey)

Get all the votes casted for a specific block by a specific voter.

Parameters:

• block_id (str) – the block id to use.
• node_pubkey (str) – base58 encoded public key

Returns:

A cursor for the matching votes.

bigchaindb.backend.query.get_votes_for_blocks_by_voter(connection, block_ids, pubkey)

Return votes for many block_ids

Parameters:

• block_ids (set) – block_ids
• pubkey (str) – public key of voting node

Returns:

A cursor of votes matching given block_ids and public key

bigchaindb.backend.query.write_block(connection, block)

Write a block to the bigchain table.

Parameters:block (dict) – the block to write. Returns:The database response.

bigchaindb.backend.query.get_block(connection, block_id)

Get a block from the bigchain table.

Parameters:block_id (str) – block id of the block to get Returns:the block or None Return type:block (dict)

bigchaindb.backend.query.write_assets(connection, assets)

Write a list of assets to the assets table.

Parameters:assets (list) – a list of assets to write. Returns:The database response.

bigchaindb.backend.query.write_metadata(connection, metadata)

Write a list of metadata to the metadata table.

Parameters:metadata (list) – a list of metadata to write. Returns:The database response.

bigchaindb.backend.query.get_assets(connection, asset_ids)

Get a list of assets from the assets table.

Parameters:

• asset_ids (list) – a list of ids for the assets to be retrieved from
• database. (the) –

Returns:

the list of returned assets.

Return type:

assets (list)

bigchaindb.backend.query.get_metadata(connection, txn_ids)

Get a list of metadata from the metadata table.

Parameters:

• txn_ids (list) – a list of ids for the metadata to be retrieved from
• database. (the) –

Returns:

the list of returned metadata.

Return type:

metadata (list)

bigchaindb.backend.query.count_blocks(connection)

Count the number of blocks in the bigchain table.

Returns:The number of blocks.

bigchaindb.backend.query.count_backlog(connection)

Count the number of transactions in the backlog table.

Returns:The number of transactions in the backlog.

bigchaindb.backend.query.write_vote(connection, vote)

Write a vote to the votes table.

Parameters:vote (dict) – the vote to write. Returns:The database response.

bigchaindb.backend.query.get_genesis_block(connection)

Get the genesis block.

Returns:The genesis block

bigchaindb.backend.query.get_last_voted_block_id(connection, node_pubkey)

Get the last voted block for a specific node.

Parameters:node_pubkey (str) – base58 encoded public key. Returns:The id of the last block the node has voted on. If the node didn’t cast any vote then the genesis block id is returned.

bigchaindb.backend.query.get_txids_filtered(connection, asset_id, operation=None)

Return all transactions for a particular asset id and optional operation.

Parameters:

• asset_id (str) – ID of transaction that defined the asset
• operation (str) (optional) – Operation to filter on

bigchaindb.backend.query.get_new_blocks_feed(connection, start_block_id)

Return a generator that yields change events of the blocks feed

Parameters:start_block_id (str) – ID of block to resume from Returns:Generator of change events

bigchaindb.backend.query.text_search(conn, search, *, language='english', case_sensitive=False, diacritic_sensitive=False, text_score=False, limit=0, table=None)

Return all the assets that match the text search.

The results are sorted by text score. For more information about the behavior of text search on MongoDB see https://docs.mongodb.com/manual/reference/operator/query/text/#behavior

Parameters:

• search (str) – Text search string to query the text index
• language (str, optional) – The language for the search and the rules for stemmer and tokenizer. If the language is None text search uses simple tokenization and no stemming.
• case_sensitive (bool, optional) – Enable or disable case sensitive search.
• diacritic_sensitive (bool, optional) – Enable or disable case sensitive diacritic search.
• text_score (bool, optional) – If True returns the text score with each document.
• limit (int, optional) – Limit the number of returned documents.

Returns:

a list of assets

Return type:

list of dict

Raises:

OperationError – If the backend does not support text search

bigchaindb.backend.schema

Database creation and schema-providing interfaces for backends.

bigchaindb.backend.schema.TABLES

tuple – The three standard tables BigchainDB relies on:

• backlog for incoming transactions awaiting to be put into a block.
• bigchain for blocks.
• votes to store votes for each block by each federation node.

bigchaindb.backend.schema.create_database(connection, dbname)

Create database to be used by BigchainDB.

Parameters:dbname (str) – the name of the database to create. Raises:DatabaseAlreadyExists – If the given dbname already exists as a database.

bigchaindb.backend.schema.create_tables(connection, dbname)

Create the tables to be used by BigchainDB.

Parameters:dbname (str) – the name of the database to create tables for.

bigchaindb.backend.schema.create_indexes(connection, dbname)

Create the indexes to be used by BigchainDB.

Parameters:dbname (str) – the name of the database to create indexes for.

bigchaindb.backend.schema.drop_database(connection, dbname)

Drop the database used by BigchainDB.

Parameters:dbname (str) – the name of the database to drop. Raises:DatabaseDoesNotExist – If the given dbname does not exist as a database.

bigchaindb.backend.schema.init_database(connection=None, dbname=None)

Initialize the configured backend for use with BigchainDB.

Creates a database with dbname with any required tables and supporting indexes.

Parameters:

• connection (Connection) – an existing connection to use to initialize the database. Creates one if not given.
• dbname (str) – the name of the database to create. Defaults to the database name given in the BigchainDB configuration.

Raises:

DatabaseAlreadyExists – If the given dbname already exists as a database.

bigchaindb.backend.schema.validate_language_key(obj, key)

Validate all nested “language” key in obj.

Parameters:obj (dict) – dictionary whose “language” key is to be validated. Returns:validation successful Return type:None

Raises:

ValidationError: will raise exception in case language is not valid.

bigchaindb.backend.schema.validate_language(value)

Check if value is a valid language. https://docs.mongodb.com/manual/reference/text-search-languages/

Args:

value (str): language to validated

Returns:

None: validation successful

Raises:

ValidationError: will raise exception in case language is not valid.

bigchaindb.backend.admin

Database configuration functions.

bigchaindb.backend.utils

exception bigchaindb.backend.utils.ModuleDispatchRegistrationError

Raised when there is a problem registering dispatched functions for a module

RethinkDB Backend

RethinkDB backend implementation.

Contains a RethinkDB-specific implementation of the changefeed, query, and schema interfaces.

You can specify BigchainDB to use RethinkDB as its database backend by either setting database.backend to 'rethinkdb' in your configuration file, or setting the BIGCHAINDB_DATABASE_BACKEND environment variable to 'rethinkdb'.

If configured to use RethinkDB, BigchainDB will automatically return instances of RethinkDBConnection for connect() and dispatch calls of the generic backend interfaces to the implementations in this module.

bigchaindb.backend.rethinkdb.connection

class bigchaindb.backend.rethinkdb.connection.RethinkDBConnection(host=None, port=None, dbname=None, connection_timeout=None, max_tries=None, **kwargs)

This class is a proxy to run queries against the database, it is:

• lazy, since it creates a connection only when needed
• resilient, because before raising exceptions it tries more times to run the query or open a connection.

run(query)

Run a RethinkDB query.

Parameters:query – the RethinkDB query. Raises:rethinkdb.ReqlDriverError – After max_tries.

bigchaindb.backend.rethinkdb.schema

bigchaindb.backend.rethinkdb.query

bigchaindb.backend.rethinkdb.query.unwind_block_transactions(block)

Yield a block for each transaction in given block

bigchaindb.backend.rethinkdb.changefeed

class bigchaindb.backend.rethinkdb.changefeed.RethinkDBChangeFeed(table, operation, *, prefeed=None, connection=None)

This class wraps a RethinkDB changefeed as a multipipes Node.

bigchaindb.backend.rethinkdb.changefeed.run_changefeed(connection, table)

Encapsulate operational logic of tailing changefeed from RethinkDB

bigchaindb.backend.rethinkdb.changefeed.get_changefeed(connection, table, operation, *, prefeed=None)

Return a RethinkDB changefeed.

Returns:An instance of RethinkDBChangeFeed.

bigchaindb.backend.rethinkdb.admin

Database configuration functions.

bigchaindb.backend.rethinkdb.admin.get_config(connection, *, table)

Get the configuration of the given table.

Parameters:

• connection (Connection) – A connection to the database.
• table (str) – The name of the table to get the configuration for.

Returns:

The configuration of the given table

Return type:

dict

bigchaindb.backend.rethinkdb.admin.reconfigure(connection, *, table, shards, replicas, primary_replica_tag=None, dry_run=False, nonvoting_replica_tags=None)

Reconfigures the given table.

Parameters:

• connection (Connection) – A connection to the database.
• table (str) – The name of the table to reconfigure.
• shards (int) – The number of shards, an integer from 1-64.
• replicas (int | dict) –
  • If replicas is an integer, it specifies the number of replicas per shard. Specifying more replicas than there are servers will return an error.
  • If replicas is a dictionary, it specifies key-value pairs of server tags and the number of replicas to assign to those servers:

    {'africa': 2, 'asia': 4, 'europe': 2, ...}
    

• primary_replica_tag (str) – The primary server specified by its server tag. Required if replicas is a dictionary. The tag must be in the replicas dictionary. This must not be specified if replicas is an integer. Defaults to None.
• dry_run (bool) – If True the generated configuration will not be applied to the table, only returned. Defaults to False.
• nonvoting_replica_tags (list of str) – Replicas with these server tags will be added to the nonvoting_replicas list of the resulting configuration. Defaults to None.

Returns:

A dictionary with possibly three keys:

• reconfigured: the number of tables reconfigured. This will be 0 if dry_run is True.
• config_changes: a list of new and old table configuration values.
• status_changes: a list of new and old table status values.

For more information please consult RethinkDB’s documentation ReQL command: reconfigure.

Return type:

dict

Raises:

OperationError – If the reconfiguration fails due to a RethinkDB ReqlOpFailedError or ReqlQueryLogicError.

bigchaindb.backend.rethinkdb.admin.set_shards(connection, *, shards, dry_run=False)

Sets the shards for the tables TABLES.

Parameters:

• connection (Connection) – A connection to the database.
• shards (int) – The number of shards, an integer from 1-64.
• dry_run (bool) – If True the generated configuration will not be applied to the table, only returned. Defaults to False.

Returns:

A dictionary with the configuration and status changes.

For more details please see reconfigure().

Return type:

dict

bigchaindb.backend.rethinkdb.admin.set_replicas(connection, *, replicas, dry_run=False)

Sets the replicas for the tables TABLES.

Parameters:

• connection (Connection) – A connection to the database.
• replicas (int) – The number of replicas per shard. Specifying more replicas than there are servers will return an error.
• dry_run (bool) – If True the generated configuration will not be applied to the table, only returned. Defaults to False.

Returns:

A dictionary with the configuration and status changes.

For more details please see reconfigure().

Return type:

dict

MongoDB Backend

Stay tuned!