gravitino.client.relational_catalog.RelationalCatalog¶

class gravitino.client.relational_catalog.RelationalCatalog(catalog_namespace: Namespace, name: str, catalog_type: Type, provider: str, audit: AuditDTO, rest_client: HTTPClient, comment: str | None = None, properties: dict[str, str] | None = None)¶

Bases: BaseSchemaCatalog, TableCatalog

Relational catalog is a catalog implementation

The RelationalCatalog supports relational database like metadata operations, for example, schemas and tables list, creation, update and deletion. A Relational catalog is under the metalake.

__init__(catalog_namespace: Namespace, name: str, catalog_type: Type, provider: str, audit: AuditDTO, rest_client: HTTPClient, comment: str | None = None, properties: dict[str, str] | None = None)¶

Methods

`__init__`(catalog_namespace, name, ...[, ...])
`alter_function`(ident, *changes)	Applies FunctionChange changes to a function in the catalog.
`alter_schema`(schema_name, *changes)	Alter the schema with specified identifier by applying the changes.
`alter_table`(identifier, *changes)	Alter a table in the catalog.
`as_fileset_catalog`()	Raises:
`as_function_catalog`()	Returns:
`as_model_catalog`()	Returns:
`as_schemas`()	Return the {@link SupportsSchemas} if the catalog supports schema operations.
`as_table_catalog`()	Return this relational catalog as a `TableCatalog`.
`as_topic_catalog`()	Returns:
`associate_tags`(tags_to_add, tags_to_remove)	Associate tags to the specific object.
`audit_info`()
`builder`([name, catalog_type, provider, ...])
`comment`()	The comment of the catalog.
`create_schema`([schema_name, comment, properties])	Create a new schema with specified identifier, comment and metadata.
`create_table`(identifier, columns[, comment, ...])	Create a table in the catalog.
`drop_function`(ident)	Drop a function by name.
`drop_schema`(schema_name, cascade)	Drop the schema with specified identifier.
`drop_table`(identifier)	Drop the table with specified identifier.
`format_schema_request_path`(ns)
`function_exists`(ident)	Check if a function with the given name exists in the catalog.
`get_function`(ident)	Get a function by NameIdentifier from the catalog.
`get_tag`(name)	Get a tag by its name for the specific object.
`list_function_infos`(namespace)	List the functions with details in a namespace from the catalog.
`list_functions`(namespace)	List the functions in a namespace from the catalog.
`list_schemas`([parent_schema])	List the schemas under the given catalog namespace, or directly under the given parent schema when `parent_schema` is provided.
`list_tables`(namespace)	List the tables in a namespace from the catalog.
`list_tags`()	List all the tag names for the specific object.
`list_tags_info`()	List all the tags with details for the specific object.
`load_schema`(schema_name)	Load the schema with specified identifier.
`load_table`()	Load table metadata by NameIdentifier from the catalog with required privileges.
`name`()	Returns:
`properties`()	The properties of the catalog.
`provider`()	Returns:
`purge_table`(identifier)	Purge the table with specified identifier.
`register_function`(ident, comment, ...)	Register a function with one or more definitions (overloads).
`schema_exists`(schema_name)	Check if a schema exists.
`supports_tags`()	Return the {@link SupportsTags} if the catalog supports tag operations.
`table_exists`(identifier)	Check if a table exists using an NameIdentifier from the catalog.
`to_schema_update_request`(change)
`type`()	Returns:
`validate`()

Attributes

`PRIVILEGES`
`PROPERTY_PACKAGE`	A reserved property to specify the package location of the catalog.
`rest_client`

PROPERTY_PACKAGE = 'package'¶

A reserved property to specify the package location of the catalog. The “package” is a string of path to the folder where all the catalog related dependencies is located. The dependencies under the “package” will be loaded by Gravitino to create the catalog.

The property “package” is not needed if the catalog is a built-in one, Gravitino will search the proper location using “provider” to load the dependencies. Only when the folder is in different location, the “package” property is needed.

class Type(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)¶

Bases: Enum

The type of the catalog.

FILESET = ('fileset', True)¶: Catalog Type for Fileset System (including HDFS, S3, etc.), like path/to/file

MESSAGING = ('messaging', False)¶: Catalog Type for Message Queue, like kafka://topic

MODEL = ('model', True)¶: Catalog Type for ML model

RELATIONAL = ('relational', False)¶: “Catalog Type for Relational Data Structure, like db.table, catalog.db.table.

UNSUPPORTED = ('unsupported', False)¶: Catalog Type for test only.

property supports_managed_catalog¶: A flag to indicate if the catalog type supports managed catalog. Managed catalog is a concept in Gravitino, which means Gravitino will manage the lifecycle of the catalog and its subsidiaries. If the catalog type supports managed catalog, users can create managed catalog of this type without specifying the catalog provider, Gravitino will use the type as the provider to create the managed catalog. If the catalog type does not support managed catalog, users need to specify the provider to create the catalog.

property type_name¶: The name of the catalog type.

alter_function(ident: NameIdentifier, *changes: FunctionChange) → Function¶

Applies FunctionChange changes to a function in the catalog.

Implementations may reject the changes. If any change is rejected, no changes should be applied to the function.

Args:: ident: The NameIdentifier instance of the function to alter. changes: The FunctionChange instances to apply to the function.
Returns:: The updated Function instance.
Raises:: NoSuchFunctionException: If the function does not exist. IllegalArgumentException: If the change is rejected by the implementation.

alter_schema(schema_name: str, *changes: SchemaChange) → Schema¶

Alter the schema with specified identifier by applying the changes.

Args:: schema_name: The name of the schema. changes: The metadata changes to apply.
Raises:: NoSuchSchemaException if the schema with specified identifier does not exist.
Returns:: The altered Schema.

alter_table(identifier: NameIdentifier, *changes) → Table¶

Alter a table in the catalog.

Args:

identifier (NameIdentifier): A table identifier. *changes:

Table changes (defined in class TableChange) to apply to the table.

Returns:

Table: The updated table metadata.

Raises:

NoSuchTableException:: If the table does not exist.
IllegalArgumentException:: If the change is rejected by the implementation.

as_fileset_catalog() → FilesetCatalog¶

Raises:: UnsupportedOperationException if the catalog does not support fileset operations.
Returns:: the FilesetCatalog if the catalog supports fileset operations.

as_function_catalog()¶

Returns:: the {@link FunctionCatalog} if the catalog supports function operations.
Raises:: UnsupportedOperationException if the catalog does not support function operations.

as_model_catalog() → GenericModelCatalog¶

Returns:: the {@link GenericModelCatalog} if the catalog supports model operations.
Raises:: UnsupportedOperationException if the catalog does not support model operations.

as_schemas()¶

Return the {@link SupportsSchemas} if the catalog supports schema operations.

Raises:: UnsupportedOperationException if the catalog does not support schema operations.
Returns:: The {@link SupportsSchemas} if the catalog supports schema operations.

as_table_catalog() → TableCatalog¶

Return this relational catalog as a TableCatalog.

This method returns self to provide access to table-related operations defined by the TableCatalog interface.

Returns:: TableCatalog: The current catalog instance as a TableCatalog.

as_topic_catalog() → TopicCatalog¶

Returns:: the {@link TopicCatalog} if the catalog supports topic operations.
Raises:: UnsupportedOperationException if the catalog does not support topic operations.

associate_tags(tags_to_add: List[str], tags_to_remove: List[str]) → List[str]¶

Associate tags to the specific object.

The tags_to_add will be added to the object, and the tags_to_remove will be removed from the object.

Note that: 1. Adding or removing tags that are not existed will be ignored. 2. If the same name tag is in both tags_to_add and tags_to_remove, it will be ignored. 3. If the tag is already associated with the object, it will raise TagAlreadyAssociatedException.

Args:: tags_to_add (list[str]): The arrays of tag name to be added to the object. tags_to_remove (list[str]): The array of tag name to be removed from the object.
Raises:: TagAlreadyAssociatedException: If the tag is already associated with the object.
Returns:: list[str]: The array of tag names that are associated with the object.

comment() → str¶

The comment of the catalog. Note. this method will return null if the comment is not set for this catalog.

Returns:: The provider of the catalog.

create_schema(schema_name: str = None, comment: str = None, properties: Dict[str, str] = None) → Schema¶

Create a new schema with specified identifier, comment and metadata.

Args:: schema_name: The name of the schema. comment: The comment of the schema. properties: The properties of the schema.
Raises:: NoSuchCatalogException if the catalog with specified namespace does not exist. SchemaAlreadyExistsException if the schema with specified identifier already exists.
Returns:: The created Schema.

create_table(identifier: NameIdentifier, columns: list[gravitino.api.rel.column.Column], comment: str | None = None, properties: dict[str, str] | None = None, partitioning: list[gravitino.api.rel.expressions.transforms.transform.Transform] | None = None, distribution: Distribution | None = None, sort_orders: list[gravitino.api.rel.expressions.sorts.sort_order.SortOrder] | None = None, indexes: list[gravitino.api.rel.indexes.index.Index] | None = None) → Table¶

Create a table in the catalog.

Args:

identifier (NameIdentifier):: A table identifier.
columns (list[Column]):: The columns of the new table.
comment (str, optional):: The table comment. Defaults to None.
properties (dict[str, str], optional):: The table properties. Defaults to None.
partitioning (Optional[list[Transform]], optional):: The table partitioning. Defaults to None.
distribution (Optional[Distribution], optional):: The distribution of the table. Defaults to None.
sort_orders (Optional[list[SortOrder]], optional):: The sort orders of the table. Defaults to None.
indexes (Optional[list[Index]], optional):: The table indexes. Defaults to None.

Raises:

NoSuchSchemaException:: If the schema does not exist.
TableAlreadyExistsException:: If the table already exists.

Returns:

Table:: The created table metadata.

drop_function(ident: NameIdentifier) → bool¶

Drop a function by name.

Args:: ident: The name identifier of the function.
Returns:: True if the function is deleted, False if the function does not exist.

drop_schema(schema_name: str, cascade: bool) → bool¶

Drop the schema with specified identifier.

Args:: schema_name: The name of the schema. cascade: Whether to drop all the tables under the schema.
Raises:: NonEmptySchemaException if the schema is not empty and cascade is false.
Returns:: true if the schema is dropped successfully, false otherwise.

drop_table(identifier: NameIdentifier) → bool¶

Drop the table with specified identifier.

Args:

identifier (NameIdentifier):: The identifier of the table, which should be “schema.table” format.

Returns:

bool:: True if the table is dropped successfully, False if the table does not exist.

function_exists(ident: NameIdentifier) → bool¶

Check if a function with the given name exists in the catalog.

Args:: ident: The function identifier.
Returns:: True if the function exists, False otherwise.

get_function(ident: NameIdentifier) → Function¶

Get a function by NameIdentifier from the catalog.

The identifier only contains the schema and function name. A function may include multiple definitions (overloads) in the result.

Args:: ident: A function identifier.
Returns:: The function with the given name.
Raises:: NoSuchFunctionException: If the function does not exist.

get_tag(name: str) → Tag¶

Get a tag by its name for the specific object.

Args:: name (str): The name of the tag.
Raises:: NoSuchTagException: If the tag does not associate with the object.
Returns:: Tag: The tag.

list_function_infos(namespace: Namespace) → List[Function]¶

List the functions with details in a namespace from the catalog.

Args:: namespace: A namespace.
Returns:: A list of functions in the namespace.
Raises:: NoSuchSchemaException: If the schema does not exist.

list_functions(namespace: Namespace) → List[NameIdentifier]¶

List the functions in a namespace from the catalog.

Args:: namespace: A namespace.
Returns:: A list of function identifiers in the namespace.
Raises:: NoSuchSchemaException: If the schema does not exist.

list_schemas(parent_schema: str | None = None) → List[str]¶

List the schemas under the given catalog namespace, or directly under the given parent schema when parent_schema is provided.

Args:

parent_schema: The parent (possibly hierarchical) schema name whose direct children are: listed, e.g. "a" or "a:b". When None, all schemas under the catalog are listed. Must not be blank when provided.

Raises:

IllegalArgumentException if parent_schema is provided but blank. NoSuchCatalogException if the catalog with specified namespace does not exist. NoSuchSchemaException if parent_schema is provided but does not exist.

Returns:

A list of schema names under the given catalog namespace or parent schema.

list_tables(namespace: Namespace) → list[gravitino.name_identifier.NameIdentifier]¶

List the tables in a namespace from the catalog.

Args:: namespace (Namespace): A namespace.
Returns:: list[NameIdentifier]: An array of table identifiers in the namespace.
Raises:: NoSuchSchemaException: If the schema does not exist.

list_tags() → List[str]¶

List all the tag names for the specific object.

Returns:: list[str]: The list of tag names.

list_tags_info() → List[Tag]¶

List all the tags with details for the specific object.

Returns:: list[Tag]: The list of tags.

load_schema(schema_name: str) → Schema¶

Load the schema with specified identifier.

Args:: schema_name: The name of the schema.
Raises:: NoSuchSchemaException if the schema with specified identifier does not exist.
Returns:: The Schema with specified identifier.

load_table(identifier: NameIdentifier) → Table¶

load_table(identifier: NameIdentifier, required_privilege_names: set[Privilege.Name]) → Table

Load table metadata by NameIdentifier from the catalog with required privileges.

Args:: identifier (NameIdentifier): A table identifier. required_privilege_names (Optional[set[Privilege.Name]], optional):

The required privilege names to access the table. Defaults to None.
Returns:: Table: The table metadata.
Raises:: NoSuchTableException: If the table does not exist.

name() → str¶

Returns:: The name of the catalog.

properties() → Dict[str, str]¶

The properties of the catalog. Note, this method will return null if the properties are not set.

Returns:: The properties of the catalog.

provider() → str¶

Returns:: The provider of the catalog.

purge_table(identifier: NameIdentifier) → bool¶

Purge the table with specified identifier.

Args:

identifier (NameIdentifier):: The identifier of the table, which should be “schema.table” format.

Returns:

bool:: True if the table is purged successfully, False if the table does not exist.

register_function(ident: NameIdentifier, comment: str | None, function_type: FunctionType, deterministic: bool, definitions: List[FunctionDefinition]) → Function¶

Each definition contains its own return type (for scalar/aggregate functions) or return columns (for table-valued functions).

Args:: ident: The function identifier. comment: The optional function comment. function_type: The function type (SCALAR, AGGREGATE, or TABLE). deterministic: Whether the function is deterministic. definitions: The function definitions, each containing parameters,

return type/columns, and implementations.
Returns:: The registered function.
Raises:: NoSuchSchemaException: If the schema does not exist. FunctionAlreadyExistsException: If the function already exists.

schema_exists(schema_name: str) → bool¶

Check if a schema exists.

If an entity such as a table, view exists, its parent namespaces must also exist. For example, if table a.b.t exists, this method invoked as schema_exists(a.b) must return true.

Args:: schema_name: The name of the schema.
Returns:: True if the schema exists, false otherwise.

supports_tags() → SupportsTags¶

Return the {@link SupportsTags} if the catalog supports tag operations.

Raises:: UnsupportedOperationException: if the catalog does not support tag operations.
Returns:: SupportsTags: the {@link SupportsTags} instance

table_exists(identifier: NameIdentifier) → bool¶

Check if a table exists using an NameIdentifier from the catalog.

Args:: identifier (NameIdentifier): A table identifier.
Returns:: bool: True If the table exists, False otherwise.

type() → Type¶

Returns:: The type of the catalog.