Database Backend Interfaces¶
Generic backend database interfaces expected by BigchainDB.
The interfaces in this module allow BigchainDB to be agnostic about its
database backend. One can configure BigchainDB to use different databases as
its data store by setting the database.backend
property in the
configuration or the BIGCHAINDB_DATABASE_BACKEND
environment variable.
Generic Interfaces¶
bigchaindb.backend.connection
¶
-
bigchaindb.backend.connection.
connect
(backend=None, host=None, port=None, name=None, max_tries=None, connection_timeout=None, replicaset=None, ssl=None, login=None, password=None, ca_cert=None, certfile=None, keyfile=None, keyfile_passphrase=None, crlfile=None)[source]¶ Create a new connection to the database backend.
All arguments default to the current configuration’s values if not given.
Parameters: Returns: An instance of
Connection
based on the given (or defaulted)backend
.Raises: ConnectionError
– If the connection to the database fails.ConfigurationError
– If the given (or defaulted)backend
is not supported or could not be loaded.AuthenticationError
– If there is a OperationFailure due to Authentication failure after connecting to the database.
-
class
bigchaindb.backend.connection.
Connection
(host=None, port=None, dbname=None, connection_timeout=None, max_tries=None, **kwargs)[source]¶ Connection class interface.
All backend implementations should provide a connection class that inherits from and implements this class.
-
__init__
(host=None, port=None, dbname=None, connection_timeout=None, max_tries=None, **kwargs)[source]¶ Create a new
Connection
instance.Parameters: - host (str) – the host to connect to.
- port (int) – the port to connect to.
- dbname (str) – the name of the database to use.
- connection_timeout (int, optional) – the milliseconds to wait until timing out the database connection attempt. Defaults to 5000ms.
- max_tries (int, optional) – how many tries before giving up, if 0 then try forever. Defaults to 3.
- **kwargs – arbitrary keyword arguments provided by the
configuration’s
database
settings
-
run
(query)[source]¶ Run a query.
Parameters: query – the query to run
Raises: DuplicateKeyError
– If the query fails because of a duplicate key constraint.OperationFailure
– If the query fails for any other reason.ConnectionError
– If the connection to the database fails.
-
connect
()[source]¶ Try to connect to the database.
Raises: ConnectionError
– If the connection to the database fails.
-
bigchaindb.backend.changefeed
¶
Changefeed interfaces for backends.
-
class
bigchaindb.backend.changefeed.
ChangeFeed
(table, operation, *, prefeed=None, connection=None)[source]¶ Create a new changefeed.
It extends
multipipes.Node
to make it pluggable in other Pipelines instances, and makes usage ofself.outqueue
to output the data.A changefeed is a real time feed on inserts, updates, and deletes, and is volatile. This class is a helper to create changefeeds. Moreover, it provides a way to specify a
prefeed
of iterable data to output before the actual changefeed.-
run_forever
()[source]¶ Main loop of the
multipipes.Node
This method is responsible for first feeding the prefeed to the outqueue and after that starting the changefeed and recovering from any errors that may occur in the backend.
-
run_changefeed
()[source]¶ Backend specific method to run the changefeed.
The changefeed is usually a backend cursor that is not closed when all the results are exausted. Instead it remains open waiting for new results.
This method should also filter each result based on the
operation
and put all matching results on the outqueue ofmultipipes.Node
.
-
-
bigchaindb.backend.changefeed.
get_changefeed
(connection, table, operation, *, prefeed=None)[source]¶ Return a ChangeFeed.
Parameters: - connection (
Connection
) – A connection to the database. - table (str) – name of the table to listen to for changes.
- operation (int) – can be ChangeFeed.INSERT, ChangeFeed.DELETE, or
ChangeFeed.UPDATE. Combining multiple operation is possible
with the bitwise
|
operator (e.g.ChangeFeed.INSERT | ChangeFeed.UPDATE
) - prefeed (iterable) – whatever set of data you want to be published first.
- connection (
bigchaindb.backend.query
¶
Query interfaces for backends.
-
bigchaindb.backend.query.
write_transaction
(connection, signed_transaction)[source]¶ Write a transaction to the backlog table.
Parameters: signed_transaction (dict) – a signed transaction. Returns: The result of the operation.
-
bigchaindb.backend.query.
update_transaction
(connection, transaction_id, doc)[source]¶ Update a transaction in the backlog table.
Parameters: Returns: The result of the operation.
-
bigchaindb.backend.query.
delete_transaction
(connection, *transaction_id)[source]¶ Delete a transaction from the backlog.
Parameters: *transaction_id (str) – the transaction(s) to delete. Returns: The database response.
-
bigchaindb.backend.query.
get_stale_transactions
(connection, reassign_delay)[source]¶ Get a cursor of stale transactions.
Transactions are considered stale if they have been assigned a node, but are still in the backlog after some amount of time specified in the configuration.
Parameters: reassign_delay (int) – threshold (in seconds) to mark a transaction stale. Returns: A cursor of transactions.
-
bigchaindb.backend.query.
get_transaction_from_block
(connection, transaction_id, block_id)[source]¶ Get a transaction from a specific block.
Parameters: Returns: The matching transaction.
-
bigchaindb.backend.query.
get_transaction_from_backlog
(connection, transaction_id)[source]¶ Get a transaction from backlog.
Parameters: transaction_id (str) – the id of the transaction. Returns: The matching transaction.
-
bigchaindb.backend.query.
get_blocks_status_from_transaction
(connection, transaction_id)[source]¶ Retrieve block election information given a secondary index and value.
Parameters: - value – a value to search (e.g. transaction id string, payload hash string)
- index (str) – name of a secondary index, e.g. ‘transaction_id’
Returns: A list of blocks with with only election information
Return type:
-
bigchaindb.backend.query.
get_asset_by_id
(conneciton, asset_id)[source]¶ Returns the asset associated with an asset_id.
Parameters: asset_id (str) – The asset id. Returns: Returns a rethinkdb cursor.
-
bigchaindb.backend.query.
get_spent
(connection, transaction_id, condition_id)[source]¶ Check if a txid was already used as an input.
A transaction can be used as an input for another transaction. Bigchain needs to make sure that a given txid is only used once.
Parameters: Returns: The transaction that used the txid as an input else None
-
bigchaindb.backend.query.
get_spending_transactions
(connection, inputs)[source]¶ Return transactions which spend given inputs
Parameters: inputs (list) – list of {txid, output} Returns: Iterator of (block_ids, transaction) for transactions that spend given inputs.
-
bigchaindb.backend.query.
get_owned_ids
(connection, owner)[source]¶ Retrieve a list of txids that can we used has inputs.
Parameters: owner (str) – base58 encoded public key. Returns: Iterator of (block_id, transaction) for transactions that list given owner in conditions.
-
bigchaindb.backend.query.
get_votes_by_block_id
(connection, block_id)[source]¶ Get all the votes casted for a specific block.
Parameters: block_id (str) – the block id to use. Returns: A cursor for the matching votes.
-
bigchaindb.backend.query.
get_votes_by_block_id_and_voter
(connection, block_id, node_pubkey)[source]¶ Get all the votes casted for a specific block by a specific voter.
Parameters: Returns: A cursor for the matching votes.
-
bigchaindb.backend.query.
get_votes_for_blocks_by_voter
(connection, block_ids, pubkey)[source]¶ Return votes for many block_ids
Parameters: Returns: A cursor of votes matching given block_ids and public key
-
bigchaindb.backend.query.
write_block
(connection, block)[source]¶ Write a block to the bigchain table.
Parameters: block (dict) – the block to write. Returns: The database response.
-
bigchaindb.backend.query.
get_block
(connection, block_id)[source]¶ Get a block from the bigchain table.
Parameters: block_id (str) – block id of the block to get Returns: the block or None Return type: block (dict)
-
bigchaindb.backend.query.
write_assets
(connection, assets)[source]¶ Write a list of assets to the assets table.
Parameters: assets (list) – a list of assets to write. Returns: The database response.
-
bigchaindb.backend.query.
get_assets
(connection, asset_ids)[source]¶ Get a list of assets from the assets table.
Parameters: - asset_ids (list) – a list of ids for the assets to be retrieved from
- database. (the) –
Returns: the list of returned assets.
Return type: assets (list)
-
bigchaindb.backend.query.
count_blocks
(connection)[source]¶ Count the number of blocks in the bigchain table.
Returns: The number of blocks.
-
bigchaindb.backend.query.
count_backlog
(connection)[source]¶ Count the number of transactions in the backlog table.
Returns: The number of transactions in the backlog.
-
bigchaindb.backend.query.
write_vote
(connection, vote)[source]¶ Write a vote to the votes table.
Parameters: vote (dict) – the vote to write. Returns: The database response.
-
bigchaindb.backend.query.
get_genesis_block
(connection)[source]¶ Get the genesis block.
Returns: The genesis block
-
bigchaindb.backend.query.
get_last_voted_block_id
(connection, node_pubkey)[source]¶ Get the last voted block for a specific node.
Parameters: node_pubkey (str) – base58 encoded public key. Returns: The id of the last block the node has voted on. If the node didn’t cast any vote then the genesis block id is returned.
-
bigchaindb.backend.query.
get_txids_filtered
(connection, asset_id, operation=None)[source]¶ Return all transactions for a particular asset id and optional operation.
Parameters:
-
bigchaindb.backend.query.
get_new_blocks_feed
(connection, start_block_id)[source]¶ Return a generator that yields change events of the blocks feed
Parameters: start_block_id (str) – ID of block to resume from Returns: Generator of change events
-
bigchaindb.backend.query.
text_search
(conn, search, *, language='english', case_sensitive=False, diacritic_sensitive=False, text_score=False, limit=0)[source]¶ Return all the assets that match the text search.
The results are sorted by text score. For more information about the behavior of text search on MongoDB see https://docs.mongodb.com/manual/reference/operator/query/text/#behavior
Parameters: - search (str) – Text search string to query the text index
- language (str, optional) – The language for the search and the rules for
stemmer and tokenizer. If the language is
None
text search uses simple tokenization and no stemming. - case_sensitive (bool, optional) – Enable or disable case sensitive search.
- diacritic_sensitive (bool, optional) – Enable or disable case sensitive diacritic search.
- text_score (bool, optional) – If
True
returns the text score with each document. - limit (int, optional) – Limit the number of returned documents.
Returns: a list of assets
Return type: Raises: OperationError
– If the backend does not support text search
bigchaindb.backend.schema
¶
Database creation and schema-providing interfaces for backends.
-
bigchaindb.backend.schema.
TABLES
¶ tuple – The three standard tables BigchainDB relies on:
backlog
for incoming transactions awaiting to be put into a block.bigchain
for blocks.votes
to store votes for each block by each federation node.
-
bigchaindb.backend.schema.
create_database
(connection, dbname)[source]¶ Create database to be used by BigchainDB.
Parameters: dbname (str) – the name of the database to create. Raises: DatabaseAlreadyExists
– If the givendbname
already exists as a database.
-
bigchaindb.backend.schema.
create_tables
(connection, dbname)[source]¶ Create the tables to be used by BigchainDB.
Parameters: dbname (str) – the name of the database to create tables for.
-
bigchaindb.backend.schema.
create_indexes
(connection, dbname)[source]¶ Create the indexes to be used by BigchainDB.
Parameters: dbname (str) – the name of the database to create indexes for.
-
bigchaindb.backend.schema.
drop_database
(connection, dbname)[source]¶ Drop the database used by BigchainDB.
Parameters: dbname (str) – the name of the database to drop. Raises: DatabaseDoesNotExist
– If the givendbname
does not exist as a database.
-
bigchaindb.backend.schema.
init_database
(connection=None, dbname=None)[source]¶ Initialize the configured backend for use with BigchainDB.
Creates a database with
dbname
with any required tables and supporting indexes.Parameters: - connection (
Connection
) – an existing connection to use to initialize the database. Creates one if not given. - dbname (str) – the name of the database to create. Defaults to the database name given in the BigchainDB configuration.
Raises: DatabaseAlreadyExists
– If the givendbname
already exists as a database.- connection (
-
bigchaindb.backend.schema.
validate_language_key
(obj, key)[source]¶ Validate all nested “language” key in obj.
Parameters: obj (dict) – dictionary whose “language” key is to be validated. Returns: validation successful Return type: None - Raises:
- ValidationError: will raise exception in case language is not valid.
-
bigchaindb.backend.schema.
validate_language
(value)[source]¶ Check if value is a valid language. https://docs.mongodb.com/manual/reference/text-search-languages/
- Args:
- value (str): language to validated
- Returns:
- None: validation successful
- Raises:
- ValidationError: will raise exception in case language is not valid.
bigchaindb.backend.admin
¶
Database configuration functions.
RethinkDB Backend¶
RethinkDB backend implementation.
Contains a RethinkDB-specific implementation of the
changefeed
, query
, and
schema
interfaces.
You can specify BigchainDB to use RethinkDB as its database backend by either
setting database.backend
to 'rethinkdb'
in your configuration file, or
setting the BIGCHAINDB_DATABASE_BACKEND
environment variable to
'rethinkdb'
.
If configured to use RethinkDB, BigchainDB will automatically return instances
of RethinkDBConnection
for
connect()
and dispatch calls of the
generic backend interfaces to the implementations in this module.
bigchaindb.backend.rethinkdb.connection
¶
-
class
bigchaindb.backend.rethinkdb.connection.
RethinkDBConnection
(host=None, port=None, dbname=None, connection_timeout=None, max_tries=None, **kwargs)[source]¶ This class is a proxy to run queries against the database, it is:
- lazy, since it creates a connection only when needed
- resilient, because before raising exceptions it tries more times to run the query or open a connection.
bigchaindb.backend.rethinkdb.changefeed
¶
-
class
bigchaindb.backend.rethinkdb.changefeed.
RethinkDBChangeFeed
(table, operation, *, prefeed=None, connection=None)[source]¶ This class wraps a RethinkDB changefeed as a multipipes Node.
bigchaindb.backend.rethinkdb.admin
¶
Database configuration functions.
-
bigchaindb.backend.rethinkdb.admin.
get_config
(connection, *, table)[source]¶ Get the configuration of the given table.
Parameters: - connection (
Connection
) – A connection to the database. - table (str) – The name of the table to get the configuration for.
Returns: The configuration of the given table
Return type: - connection (
-
bigchaindb.backend.rethinkdb.admin.
reconfigure
(connection, *, table, shards, replicas, primary_replica_tag=None, dry_run=False, nonvoting_replica_tags=None)[source]¶ Reconfigures the given table.
Parameters: - connection (
Connection
) – A connection to the database. - table (str) – The name of the table to reconfigure.
- shards (int) – The number of shards, an integer from 1-64.
- replicas (
int
|dict
) –- If replicas is an integer, it specifies the number of replicas per shard. Specifying more replicas than there are servers will return an error.
- If replicas is a dictionary, it specifies key-value pairs
of server tags and the number of replicas to assign to
those servers:
{'africa': 2, 'asia': 4, 'europe': 2, ...}
- primary_replica_tag (str) – The primary server specified by its
server tag. Required if
replicas
is a dictionary. The tag must be in thereplicas
dictionary. This must not be specified ifreplicas
is an integer. Defaults toNone
. - dry_run (bool) – If
True
the generated configuration will not be applied to the table, only returned. Defaults toFalse
. - nonvoting_replica_tags (
list
ofstr
) – Replicas with these server tags will be added to thenonvoting_replicas
list of the resulting configuration. Defaults toNone
.
Returns: A dictionary with possibly three keys:
reconfigured
: the number of tables reconfigured. This will be0
ifdry_run
isTrue
.config_changes
: a list of new and old table configuration values.status_changes
: a list of new and old table status values.
For more information please consult RethinkDB’s documentation ReQL command: reconfigure.
Return type: Raises: OperationError
– If the reconfiguration fails due to a RethinkDBReqlOpFailedError
orReqlQueryLogicError
.- connection (
-
bigchaindb.backend.rethinkdb.admin.
set_shards
(connection, *, shards, dry_run=False)[source]¶ Sets the shards for the tables
TABLES
.Parameters: - connection (
Connection
) – A connection to the database. - shards (int) – The number of shards, an integer from 1-64.
- dry_run (bool) – If
True
the generated configuration will not be applied to the table, only returned. Defaults toFalse
.
Returns: - A dictionary with the configuration and status changes.
For more details please see
reconfigure()
.
Return type: - connection (
-
bigchaindb.backend.rethinkdb.admin.
set_replicas
(connection, *, replicas, dry_run=False)[source]¶ Sets the replicas for the tables
TABLES
.Parameters: - connection (
Connection
) – A connection to the database. - replicas (int) – The number of replicas per shard. Specifying more replicas than there are servers will return an error.
- dry_run (bool) – If
True
the generated configuration will not be applied to the table, only returned. Defaults toFalse
.
Returns: - A dictionary with the configuration and status changes.
For more details please see
reconfigure()
.
Return type: - connection (
MongoDB Backend¶
Stay tuned!