gRPC API Documentation

Table of contents

maintenance.proto

Maintenance

Maintenance service provides methods for maintenance purposes.

Backup

rpc Backup(BackupRequest) .replication.v1.SnapshotChunk

Restore

rpc Restore(RestoreMessage) RestoreResponse

Reset

rpc Reset(ResetRequest) ResetResponse

BackupRequest

BackupRequest requests and opens a stream with backup data.

FieldTypeLabelDescription
tablebytes table is name of the table to stream.

ResetRequest

ResetRequest resets either a single or multiple tables in the cluster, meaning that their data will be repopulated from the Leader.

FieldTypeLabelDescription
tablebytes table is a table name to reset.
reset_allbool reset_all if true all the tables will be reset, use with caution.

ResetResponse

RestoreInfo

RestoreInfo metadata of restore snapshot that is going to be uploaded.

FieldTypeLabelDescription
tablebytes table is name of the table in the stream.

RestoreMessage

RestoreMessage contains either info of the table being restored or chunk of a backup data.

FieldTypeLabelDescription
infoRestoreInfo  
chunkreplication.v1.SnapshotChunk  

RestoreResponse

mvcc.proto

Command

FieldTypeLabelDescription
tablebytes table name of the table
typeCommand.CommandType type is the kind of event. If type is a PUT, it indicates new data has been stored to the key. If type is a DELETE, it indicates the key was deleted.
kvKeyValue kv holds the KeyValue for the event. A PUT event contains current kv pair. A PUT event with kv.Version=1 indicates the creation of a key. A DELETE/EXPIRE event contains the deleted key with its modification revision set to the revision of deletion.
leader_indexuint64optionalleader_index holds the value of the log index of a leader cluster from which this command was replicated from.
batchKeyValuerepeatedbatch is an atomic batch of KVs to either PUT or DELETE. (faster, no read, no mix of types, no conditions).
txnTxnoptionaltxn is an atomic transaction (slow, supports reads and conditions).
range_endbytesoptionalrange_end is the key following the last key to affect for the range [kv.key, range_end). If range_end is not given, the range is defined to contain only the kv.key argument. If range_end is one bit larger than the given kv.key, then the range is all the keys with the prefix (the given key). If range_end is ‘\0’, the range is all keys greater than or equal to the key argument.
prev_kvsbool prev_kvs if to fetch previous KVs.
sequenceCommandrepeatedsequence is the sequence of commands to be applied as a single FSM step.
countbool count if to count number of records affected by a command.

CommandResult

FieldTypeLabelDescription
responsesResponseOprepeatedresponses are the responses (if any) in order of application.
revisionuint64 revision is the key-value store revision when the request was applied.

Compare

Compare property target for every KV from DB in [key, range_end) with target_union using the operation result. e.g. DB[key].target result target_union.target, that means that for asymmetric operations LESS and GREATER the target property of the key from the DB is the left-hand side of the comparison. Examples:

  • DB[key][value] EQUAL target_union.value
  • DB[key][value] GREATER target_union.value
  • DB[key...range_end][value] GREATER target_union.value
  • DB[key][value] LESS target_union.value
FieldTypeLabelDescription
resultCompare.CompareResult result is logical comparison operation for this comparison.
targetCompare.CompareTarget target is the key-value field to inspect for the comparison.
keybytes key is the subject key for the comparison operation.
valuebytes value is the value of the given key, in bytes.
range_endbytes range_end compares the given target to all keys in the range [key, range_end). See RangeRequest for more details on key ranges.
TODO: fill out with most of the rest of RangeRequest fields when needed.

KeyValue

FieldTypeLabelDescription
keybytes key is the key in bytes. An empty key is not allowed.
create_revisionint64 create_revision is the revision of last creation on this key.
mod_revisionint64 mod_revision is the revision of last modification on this key.
valuebytes value is the value held by the key, in bytes.

RequestOp

FieldTypeLabelDescription
request_rangeRequestOp.Range  
request_putRequestOp.Put  
request_delete_rangeRequestOp.DeleteRange  

RequestOp.DeleteRange

FieldTypeLabelDescription
keybytes key is the first key to delete in the range.
range_endbytes range_end is the key following the last key to delete for the range [key, range_end). If range_end is not given, the range is defined to contain only the key argument. If range_end is one bit larger than the given key, then the range is all the keys with the prefix (the given key). If range_end is ‘\0’, the range is all keys greater than or equal to the key argument.
prev_kvbool If prev_kv is set, regatta gets the previous key-value pairs before deleting it. The previous key-value pairs will be returned in the delete response. Beware that getting previous records could have serious performance impact on a delete range spanning a large dataset.
countbool If count is set, regatta gets the count of previous key-value pairs before deleting it. The deleted field will be set to number of deleted key-value pairs in the response. Beware that counting records could have serious performance impact on a delete range spanning a large dataset.

RequestOp.Put

FieldTypeLabelDescription
keybytes key is the key, in bytes, to put into the key-value store.
valuebytes value is the value, in bytes, to associate with the key in the key-value store.
prev_kvbool prev_kv if true the previous key-value pair will be returned in the put response.

RequestOp.Range

FieldTypeLabelDescription
keybytes key is the first key for the range. If range_end is not given, the request only looks up key.
range_endbytes range_end is the upper bound on the requested range [key, range_end). If range_end is ‘\0’, the range is all keys >= key. If range_end is key plus one (e.g., “aa”+1 == “ab”, “a\xff”+1 == “b”), then the range request gets all keys prefixed with key. If both key and range_end are ‘\0’, then the range request returns all keys.
limitint64 limit is a limit on the number of keys returned for the request. When limit is set to 0, it is treated as no limit.
keys_onlybool keys_only when set returns only the keys and not the values.
count_onlybool count_only when set returns only the count of the keys in the range.

ResponseOp

FieldTypeLabelDescription
response_rangeResponseOp.Range  
response_putResponseOp.Put  
response_delete_rangeResponseOp.DeleteRange  

ResponseOp.DeleteRange

FieldTypeLabelDescription
deletedint64 deleted is the number of keys deleted by the delete range request.
prev_kvsKeyValuerepeatedif prev_kv is set in the request, the previous key-value pairs will be returned.

ResponseOp.Put

FieldTypeLabelDescription
prev_kvKeyValue if prev_kv is set in the request, the previous key-value pair will be returned.

ResponseOp.Range

FieldTypeLabelDescription
kvsKeyValuerepeatedkvs is the list of key-value pairs matched by the range request. kvs is empty when count is requested.
morebool more indicates if there are more keys to return in the requested range.
countint64 count is set to the number of keys within the range when requested.

Txn

FieldTypeLabelDescription
compareComparerepeatedcompare is a list of predicates representing a conjunction of terms. If the comparisons succeed, then the success requests will be processed in order, and the response will contain their respective responses in order. If the comparisons fail, then the failure requests will be processed in order, and the response will contain their respective responses in order.
successRequestOprepeatedsuccess is a list of requests which will be applied when compare evaluates to true.
failureRequestOprepeatedfailure is a list of requests which will be applied when compare evaluates to false.

Command.CommandType

NameNumberDescription
PUT0 
DELETE1 
DUMMY2 
PUT_BATCH3 
DELETE_BATCH4 
TXN5 
SEQUENCE6 

Compare.CompareResult

NameNumberDescription
EQUAL0 
GREATER1 
LESS2 
NOT_EQUAL3 

Compare.CompareTarget

NameNumberDescription
VALUE0 

regatta.proto

Cluster

Cluster service ops.

MemberList

rpc MemberList(MemberListRequest) MemberListResponse

MemberList lists all the members in the cluster.

Status

rpc Status(StatusRequest) StatusResponse

Status gets the status of the member.

KV

KV for handling the read/put requests

Range

rpc Range(RangeRequest) RangeResponse

Range gets the keys in the range from the key-value store.

IterateRange

rpc IterateRange(RangeRequest) RangeResponse

IterateRange gets the keys in the range from the key-value store.

Put

rpc Put(PutRequest) PutResponse

Put puts the given key into the key-value store.

DeleteRange

rpc DeleteRange(DeleteRangeRequest) DeleteRangeResponse

DeleteRange deletes the given range from the key-value store.

Txn

rpc Txn(TxnRequest) TxnResponse

Txn processes multiple requests in a single transaction. A txn request increments the revision of the key-value store and generates events with the same revision for every completed request. It is allowed to modify the same key several times within one txn (the result will be the last Op that modified the key).

Tables

API for managing tables.

Create

rpc Create(CreateTableRequest) CreateTableResponse

Create a table. All followers will automatically replicate the table. This procedure is available only in the leader cluster.

Delete

rpc Delete(DeleteTableRequest) DeleteTableResponse

Delete a table. All followers will automatically delete the table. This procedure is available only in the leader cluster.

List

rpc List(ListTablesRequest) ListTablesResponse

Get names of all the tables present in the cluster. This procedure is available in both leader and follower clusters.

CreateTableRequest

CreateTableRequest describes the table to be created.

FieldTypeLabelDescription
namestring Name of the table to be created.
configgoogle.protobuf.Struct config the table configuration values.

CreateTableResponse

CreateTableResponse describes the newly created table.

FieldTypeLabelDescription
idstring id the created table.

DeleteRangeRequest

FieldTypeLabelDescription
tablebytes table name of the table
keybytes key is the first key to delete in the range.
range_endbytes range_end is the key following the last key to delete for the range [key, range_end). If range_end is not given, the range is defined to contain only the key argument. If range_end is one bit larger than the given key, then the range is all the keys with the prefix (the given key). If range_end is ‘\0’, the range is all keys greater than or equal to the key argument.
prev_kvbool If prev_kv is set, regatta gets the previous key-value pairs before deleting it. The previous key-value pairs will be returned in the delete response. Beware that getting previous records could have serious performance impact on a delete range spanning a large dataset.
countbool If count is set, regatta gets the count of previous key-value pairs before deleting it. The deleted field will be set to number of deleted key-value pairs in the response. Beware that counting records could have serious performance impact on a delete range spanning a large dataset.

DeleteRangeResponse

FieldTypeLabelDescription
headerResponseHeader  
deletedint64 deleted is the number of keys deleted by the delete range request.
prev_kvsmvcc.v1.KeyValuerepeatedif prev_kv is set in the request, the previous key-value pairs will be returned.

DeleteTableRequest

DeleteTableRequest describes the table to be deleted.

FieldTypeLabelDescription
namestring name of the table to be deleted.

DeleteTableResponse

DeleteTableResponse when the table was successfully deleted.

ListTablesRequest

ListTablesRequest requests the list of currently registered tables.

ListTablesResponse

FollowerGetTablesResponse contains information about tables stored in the cluster.

FieldTypeLabelDescription
tablesTableInforepeated 

Member

FieldTypeLabelDescription
idstring id is the member ID of this member.
namestring name is the human-readable name of the member. If the member is not started, the name will be an empty string.
peerURLsstringrepeatedpeerURLs is the list of URLs the member exposes to the cluster for communication.
clientURLsstringrepeatedclientURLs is the list of URLs the member exposes to clients for communication. If the member is not started, clientURLs will be empty.

MemberListRequest

MemberListResponse

FieldTypeLabelDescription
clusterstring cluster is a name of the cluster.
membersMemberrepeatedmembers is a list of all members associated with the cluster.

PutRequest

FieldTypeLabelDescription
tablebytes table name of the table
keybytes key is the key, in bytes, to put into the key-value store.
valuebytes value is the value, in bytes, to associate with the key in the key-value store.
prev_kvbool prev_kv if true the previous key-value pair will be returned in the put response.

PutResponse

FieldTypeLabelDescription
headerResponseHeader  
prev_kvmvcc.v1.KeyValue if prev_kv is set in the request, the previous key-value pair will be returned.

RangeRequest

FieldTypeLabelDescription
tablebytes table name of the table
keybytes key is the first key for the range. If range_end is not given, the request only looks up key.
range_endbytes range_end is the upper bound on the requested range [key, range_end). If range_end is ‘\0’, the range is all keys >= key. If range_end is key plus one (e.g., “aa”+1 == “ab”, “a\xff”+1 == “b”), then the range request gets all keys prefixed with key. If both key and range_end are ‘\0’, then the range request returns all keys.
limitint64 limit is a limit on the number of keys returned for the request. When limit is set to 0, it is treated as no limit.
linearizablebool linearizable sets the range request to use linearizable read. Linearizable requests have higher latency and lower throughput than serializable requests but reflect the current consensus of the cluster. For better performance, in exchange for possible stale reads, a serializable range request is served locally without needing to reach consensus with other nodes in the cluster. The serializable request is default option.
keys_onlybool keys_only when set returns only the keys and not the values.
count_onlybool count_only when set returns only the count of the keys in the range.
min_mod_revisionint64 min_mod_revision is the lower bound for returned key mod revisions; all keys with lesser mod revisions will be filtered away.
max_mod_revisionint64 max_mod_revision is the upper bound for returned key mod revisions; all keys with greater mod revisions will be filtered away.
min_create_revisionint64 min_create_revision is the lower bound for returned key create revisions; all keys with lesser create revisions will be filtered away.
max_create_revisionint64 max_create_revision is the upper bound for returned key create revisions; all keys with greater create revisions will be filtered away.

RangeResponse

FieldTypeLabelDescription
headerResponseHeader  
kvsmvcc.v1.KeyValuerepeatedkvs is the list of key-value pairs matched by the range request. kvs is empty when count is requested.
morebool more indicates if there are more keys to return in the requested range.
countint64 count is set to the number of keys within the range when requested.

ResponseHeader

FieldTypeLabelDescription
shard_iduint64 shard_id is the ID of the shard which sent the response.
replica_iduint64 replica_id is the ID of the member which sent the response.
revisionuint64 revision is the key-value store revision when the request was applied.
raft_termuint64 raft_term is the raft term when the request was applied.
raft_leader_iduint64 raft_leader_id is the ID of the actual raft quorum leader.

StatusRequest

FieldTypeLabelDescription
configbool config controls if the configuration values should be fetched as well.

StatusResponse

FieldTypeLabelDescription
idstring id is the member ID of this member.
versionstring version is the semver version used by the responding member.
infostring info is the additional server info.
tablesStatusResponse.TablesEntryrepeatedtables is a status of tables of the responding member.
configgoogle.protobuf.Struct config the node configuration values.
errorsstringrepeatederrors contains alarm/health information and status.

StatusResponse.TablesEntry

FieldTypeLabelDescription
keystring  
valueTableStatus  

TableInfo

TableInfo describes a single table.

FieldTypeLabelDescription
namestring name of the table.
idstring id of the table.
configgoogle.protobuf.Struct config the table configuration values.

TableStatus

FieldTypeLabelDescription
logSizeint64 dbSize is the size of the raft log, in bytes, of the responding member.
dbSizeint64 dbSize is the size of the backend database physically allocated, in bytes, of the responding member.
leaderstring leader is the member ID which the responding member believes is the current leader.
raftIndexuint64 raftIndex is the current raft committed index of the responding member.
raftTermuint64 raftTerm is the current raft term of the responding member.
raftAppliedIndexuint64 raftAppliedIndex is the current raft applied index of the responding member.

TxnRequest

From google paxosdb paper: Our implementation hinges around a powerful primitive which we call MultiOp. All other database operations except for iteration are implemented as a single call to MultiOp. A MultiOp is applied atomically and consists of three components:

  1. A list of tests called guard. Each test in guard checks a single entry in the database. It may check for the absence or presence of a value, or compare with a given value. Two different tests in the guard may apply to the same or different entries in the database. All tests in the guard are applied and MultiOp returns the results. If all tests are true, MultiOp executes t op (see item 2 below), otherwise it executes f op (see item 3 below).
  2. A list of database operations called t op. Each operation in the list is either an insert, delete, or lookup operation, and applies to a database entry(ies). Two different operations in the list may apply to the same or different entries in the database. These operations are executed if guard evaluates to true.
  3. A list of database operations called f op. Like t op, but executed if guard evaluates to false.
FieldTypeLabelDescription
tablebytes table name of the table
comparemvcc.v1.Comparerepeatedcompare is a list of predicates representing a conjunction of terms. If the comparisons succeed, then the success requests will be processed in order, and the response will contain their respective responses in order. If the comparisons fail, then the failure requests will be processed in order, and the response will contain their respective responses in order.
successmvcc.v1.RequestOprepeatedsuccess is a list of requests which will be applied when compare evaluates to true.
failuremvcc.v1.RequestOprepeatedfailure is a list of requests which will be applied when compare evaluates to false.

TxnResponse

FieldTypeLabelDescription
headerResponseHeader  
succeededbool succeeded is set to true if the compare evaluated to true or false otherwise.
responsesmvcc.v1.ResponseOprepeatedresponses is a list of responses corresponding to the results from applying success if succeeded is true or failure if succeeded is false.

replication.proto

Log

Log service provides methods to replicate data from Regatta leader’s log to Regatta followers’ logs.

Replicate

rpc Replicate(ReplicateRequest) ReplicateResponse

Replicate is method to ask for data of specified table from the specified index.

Metadata

Metadata service provides method to get Regatta metadata, e.g. tables.

Get

rpc Get(MetadataRequest) MetadataResponse

Snapshot

Stream

rpc Stream(SnapshotRequest) SnapshotChunk

MetadataRequest

MetadataResponse

FieldTypeLabelDescription
tablesTablerepeated 

ReplicateCommand

FieldTypeLabelDescription
leader_indexuint64 leaderIndex represents the leader raft index of the given command
commandmvcc.v1.Command command holds the leader raft log command at leaderIndex

ReplicateCommandsResponse

ReplicateCommandsResponse sequence of replication commands

FieldTypeLabelDescription
commandsReplicateCommandrepeatedcommands represent the

ReplicateErrResponse

FieldTypeLabelDescription
errorReplicateError  

ReplicateRequest

ReplicateRequest request of the replication data at given leader_index

FieldTypeLabelDescription
tablebytes table is name of the table to replicate
leader_indexuint64 leader_index is the index in the leader raft log of the last stored item in the follower

ReplicateResponse

ReplicateResponse response to the ReplicateRequest

FieldTypeLabelDescription
commands_responseReplicateCommandsResponse  
error_responseReplicateErrResponse  
leader_indexuint64 leader_index is the largest applied leader index at the time of the client RPC.

SnapshotChunk

FieldTypeLabelDescription
databytes data is chunk of snapshot
lenuint64 len is a length of data bytes
indexuint64 index the index for which the snapshot was created

SnapshotRequest

FieldTypeLabelDescription
tablebytes table is name of the table to stream

Table

FieldTypeLabelDescription
namestring  
typeTable.Type  

ReplicateError

NameNumberDescription
USE_SNAPSHOT0USE_SNAPSHOT occurs when leader has no longer the specified leader_index in the log. Follower must use GetSnapshot to catch up.
LEADER_BEHIND1LEADER_BEHIND occurs when the index of the leader is smaller than requested leader_index. This should never happen. Manual intervention needed.

Table.Type

NameNumberDescription
REPLICATED0 
LOCAL1 

Scalar Value Types

.proto TypeNotesC++JavaPythonGoC#PHPRuby
double doubledoublefloatfloat64doublefloatFloat
float floatfloatfloatfloat32floatfloatFloat
int32Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.int32intintint32intintegerBignum or Fixnum (as required)
int64Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.int64longint/longint64longinteger/stringBignum
uint32Uses variable-length encoding.uint32intint/longuint32uintintegerBignum or Fixnum (as required)
uint64Uses variable-length encoding.uint64longint/longuint64ulonginteger/stringBignum or Fixnum (as required)
sint32Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.int32intintint32intintegerBignum or Fixnum (as required)
sint64Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.int64longint/longint64longinteger/stringBignum
fixed32Always four bytes. More efficient than uint32 if values are often greater than 2^28.uint32intintuint32uintintegerBignum or Fixnum (as required)
fixed64Always eight bytes. More efficient than uint64 if values are often greater than 2^56.uint64longint/longuint64ulonginteger/stringBignum
sfixed32Always four bytes.int32intintint32intintegerBignum or Fixnum (as required)
sfixed64Always eight bytes.int64longint/longint64longinteger/stringBignum
bool boolbooleanbooleanboolboolbooleanTrueClass/FalseClass
stringA string must always contain UTF-8 encoded or 7-bit ASCII text.stringStringstr/unicodestringstringstringString (UTF-8)
bytesMay contain any arbitrary sequence of bytes.stringByteStringstr[]byteByteStringstringString (ASCII-8BIT)