gravitino.client.gravitino_metalake.GravitinoMetalake

class gravitino.client.gravitino_metalake.GravitinoMetalake(metalake: MetalakeDTO = None, client: HTTPClient = None)

Bases: MetalakeDTO, SupportsJobs, TagOperations

Gravitino Metalake is the top-level metadata repository for users. It contains a list of catalogs as sub-level metadata collections. With GravitinoMetalake, users can list, create, load, alter and drop a catalog with specified identifier.

__init__(metalake: MetalakeDTO = None, client: HTTPClient = None)

Methods

__init__([metalake, client])
add_group(group)	Add a group to this metalake.
add_user(user)	Add a user to this metalake.
alter_catalog(name, *changes)	Alter the catalog with specified name by applying the changes.
alter_job_template(job_template_name, *changes)	Alter the job template with specified name by applying the changes.
alter_tag(tag_name, *changes)	Alter a tag under a metalake.
audit_info()
cancel_job(job_id)	Cancels a job by its ID.
comment()	The comment of the metalake.
create_catalog(name, catalog_type, provider, ...)	Create a new catalog with specified name, catalog type, comment and properties.
create_role(role_name[, properties, ...])	Create a new role under this metalake.
create_tag(tag_name, comment, properties)	Create a new tag under a metalake.
delete_job_template(job_template_name)	Deletes a job template by its name.
delete_role(role_name)	Delete a role from this metalake.
delete_tag(tag_name)	Delete a tag under a metalake.
disable_catalog(name)	Disable the catalog with specified name.
drop_catalog(name[, force])	Drop the catalog with specified name.
enable_catalog(name)	Enable the catalog with specified name.
equals(other)
from_dict(kvs, *[, infer_missing])
from_json(s, *[, parse_float, parse_int, ...])
get_group(group)	Get a group by name from this metalake.
get_job(job_id)	Retrieves a job by its ID.
get_job_template(job_template_name)	Retrieves a job template by its name.
get_owner(metadata_object)	Get the owner of a metadata object.
get_role(role_name)	Get a role by name from this metalake.
get_tag(tag_name)	Get a tag by its name under a metalake.
get_user(user)	Get a user by name from this metalake.
grant_privileges_to_role(role_name, ...)	Grant privileges to a role on a securable object.
grant_roles_to_group(role_names, group_name)	Grant roles to a group.
grant_roles_to_user(role_names, user_name)	Grant roles to a user.
list_catalogs()	List all the catalogs under this metalake.
list_catalogs_info()	List all the catalogs with their information under this metalake.
list_group_names()	List all group names under this metalake.
list_groups()	List all groups with details under this metalake.
list_job_templates()	List all the registered job templates in Gravitino.
list_jobs([job_template_name])	List all the jobs under this metalake.
list_role_names()	List all role names under this metalake.
list_tags()	List all the tag names under a metalake.
list_tags_info()	List tags information under a metalake.
list_user_names()	List all user names under this metalake.
list_users()	List all users with details under this metalake.
load_catalog(name)	Load the catalog with specified name.
name()	The name of the metalake.
properties()	The properties of the metalake.
property_equal(p1, p2)
register_job_template(job_template)	Register a job template with the specified job template to Gravitino.
remove_group(group)	Remove a group from this metalake.
remove_user(user)	Remove a user from this metalake.
revoke_privileges_from_role(role_name, ...)	Revoke privileges from a role on a securable object.
revoke_roles_from_group(role_names, group_name)	Revoke roles from a group.
revoke_roles_from_user(role_names, user_name)	Revoke roles from a user.
run_job(job_template_name, job_conf)	Runs a job based on the specified job template and configuration.
schema(*[, infer_missing, only, exclude, ...])
set_owner(metadata_object, owner_name, ...)	Set the owner of a metadata object.
to_dict([encode_json])
to_json(*[, skipkeys, ensure_ascii, ...])

Attributes

API_METALAKES_CATALOGS_PATH
API_METALAKES_GROUPS_PATH
API_METALAKES_GROUP_PATH
API_METALAKES_JOB_RUNS_PATH
API_METALAKES_JOB_TEMPLATES_PATH
API_METALAKES_OWNERS_PATH
API_METALAKES_ROLES_PATH
API_METALAKES_ROLE_PATH
API_METALAKES_TAGS_PATH
API_METALAKES_TAG_PATH
API_METALAKES_USERS_PATH
API_METALAKES_USER_PATH
API_PERMISSIONS_GROUP_GRANT_PATH
API_PERMISSIONS_GROUP_REVOKE_PATH
API_PERMISSIONS_ROLE_GRANT_PATH
API_PERMISSIONS_ROLE_REVOKE_PATH
API_PERMISSIONS_USER_GRANT_PATH
API_PERMISSIONS_USER_REVOKE_PATH
dataclass_json_config
rest_client

add_group(group: str) → Group

Add a group to this metalake.

Args:

group: The name of the group.

Returns:

The added Group object.

Raises:

GroupAlreadyExistsException: If a group with the same name already exists. NoSuchMetalakeException: If the metalake does not exist.

add_user(user: str) → User

Add a user to this metalake.

Args:

user: The name of the user.

Returns:

The added User object.

Raises:

UserAlreadyExistsException: If a user with the same name already exists. NoSuchMetalakeException: If the metalake does not exist.

alter_catalog(name: str, *changes: CatalogChange) → Catalog

Alter the catalog with specified name by applying the changes.

Args:

name: the name of the catalog. changes: the changes to apply to the catalog.

Raises:

NoSuchCatalogException if the catalog with specified name does not exist. IllegalArgumentException if the changes are invalid.

Returns:

the altered Catalog.

alter_job_template(job_template_name: str, *changes: JobTemplateChange) → JobTemplate

Alter the job template with specified name by applying the changes.

Args:

job_template_name: the name of the job template. changes: the changes to apply to the job template.

Raises:

NoSuchJobTemplateException if the job template with specified name does not exist. IllegalArgumentException if the changes are invalid.

Returns:

the altered JobTemplate.

alter_tag(tag_name, *changes) → Tag

Alter a tag under a metalake.

Args:

tag_name (str): The name of the tag. changes (TagChange): The changes to apply to the tag.

Returns:

Tag: The altered tag.

Raises:

NoSuchTagException: If the tag does not exist. NoSuchMetalakeException: If the metalake does not exist. IllegalArgumentException: If the changes cannot be applied to the tag. TagAlreadyExistsException: If a tag with the new name already exists.

cancel_job(job_id: str) → JobHandle

Cancels a job by its ID.

Args:

job_id: The ID of the job to cancel.

Returns:

A JobHandle representing the cancelled job.

Raises:

NoSuchJobException: If no job with the specified ID exists.

comment() → str

The comment of the metalake. Note. this method will return None if the comment is not set for this metalake.

Returns:

Optional[str]: The comment of the metalake.

create_catalog(name: str, catalog_type: Type, provider: str, comment: str, properties: Dict[str, str]) → Catalog

Create a new catalog with specified name, catalog type, comment and properties.

Args:

name: The name of the catalog. catalog_type: The type of the catalog. provider: The provider of the catalog. This parameter can be None if the catalog provides a managed implementation. Currently, the model and fileset catalog support None provider. For the details, please refer to the Catalog.Type. comment: The comment of the catalog. properties: The properties of the catalog.

Raises:

NoSuchMetalakeException if the metalake does not exist. CatalogAlreadyExistsException if the catalog with specified name already exists.

Returns:

The created Catalog.

create_role(role_name: str, properties: Dict[str, str] | None = None, securable_objects: List[SecurableObject] | None = None) → Role

Create a new role under this metalake.

Args:

role_name: The name of the role. properties: The properties of the role. securable_objects: The securable objects of the role.

Returns:

The created Role object.

Raises:

RoleAlreadyExistsException: If a role with the same name already exists. NoSuchMetalakeException: If the metalake does not exist.

create_tag(tag_name, comment, properties) → Tag

Create a new tag under a metalake.

Raises:

NoSuchMetalakeException: If the metalake does not exist. TagAlreadyExistsException: If the tag already exists.

Args:

tag_name (str): The name of the tag. comment (str): The comment of the tag. properties (dict[str, str]): The properties of the tag.

Returns:

Tag: The tag information.

delete_job_template(job_template_name: str) → bool

Deletes a job template by its name. This will remove the job template from Gravitino, and it will no longer be available for execution. Only when all the jobs associated with this job template are completed, failed or cancelled, the job template can be deleted successfully, otherwise it will throw InUseException. Returns false if the job template to be deleted does not exist.

The deletion of a job template will also delete all the jobs associated with this template.

Args:

job_template_name: The name of the job template to delete.

Returns:

bool: True if the job template was deleted successfully, False if the job template does not exist.

Raises:

InUseException: If the job template is currently in use by any jobs, it cannot be deleted.

delete_role(role_name: str) → bool

Delete a role from this metalake.

Args:

role_name: The name of the role.

Returns:

True if the role was deleted, False otherwise.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

delete_tag(tag_name) → bool

Delete a tag under a metalake.

Args:

tag_name (str): The name of the tag.

Returns:

bool: True if the tag was deleted, False otherwise.

Raises:

NoSuchTagException: If the tag does not exist. NoSuchMetalakeException: If the metalake does not exist.

disable_catalog(name: str)

Disable the catalog with specified name. If the catalog is already disabled, this method does nothing.

Args:

name: the name of the catalog.

Raises:

NoSuchCatalogException if the catalog with specified name does not exist.

drop_catalog(name: str, force: bool = False) → bool

Drop the catalog with specified name.

Args:

name: the name of the catalog. force: whether to force drop the catalog.

Returns:

true if the catalog is dropped successfully, false if the catalog does not exist.

enable_catalog(name: str)

Enable the catalog with specified name. If the catalog is already in use, this method does nothing.

Args:

name: the name of the catalog.

Raises:

NoSuchCatalogException if the catalog with specified name does not exist.

get_group(group: str) → Group

Get a group by name from this metalake.

Args:

group: The name of the group.

Returns:

The Group object.

Raises:

NoSuchGroupException: If the group does not exist. NoSuchMetalakeException: If the metalake does not exist.

get_job(job_id: str) → JobHandle

Retrieves a job by its ID.

Args:

job_id: The ID of the job to retrieve.

Returns:

The JobHandle representing the job if found, otherwise raises an exception.

Raises:

NoSuchJobException: If no job with the specified ID exists.

get_job_template(job_template_name: str) → JobTemplate

Retrieves a job template by its name.

Args:

job_template_name: The name of the job template to retrieve.

Returns:

The job template if found, otherwise raises an exception.

Raises:

NoSuchJobTemplateException: If no job template with the specified name exists.

get_owner(metadata_object: MetadataObject) → Owner | None

Get the owner of a metadata object.

Args:

metadata_object: The metadata object to get the owner for.

Returns:

Optional[Owner]: The owner of the metadata object, or None if no owner is set.

Raises:

NoSuchMetadataObjectException: If the metadata object does not exist. NotFoundException: If a related resource is not found. MetalakeNotInUseException: If the metalake is not in use.

get_role(role_name: str) → Role

Get a role by name from this metalake.

Args:

role_name: The name of the role.

Returns:

The Role object.

Raises:

NoSuchRoleException: If the role does not exist. NoSuchMetalakeException: If the metalake does not exist.

get_tag(tag_name) → Tag

Get a tag by its name under a metalake.

Args:

tag_name (str): The name of the tag.

Returns:

Tag: The tag information.

Raises:

NoSuchTagException: If the tag does not exist.

get_user(user: str) → User

Get a user by name from this metalake.

Args:

user: The name of the user.

Returns:

The User object.

Raises:

NoSuchUserException: If the user does not exist. NoSuchMetalakeException: If the metalake does not exist.

grant_privileges_to_role(role_name: str, securable_object: SecurableObject, privileges: List[Privilege]) → Role

Grant privileges to a role on a securable object.

Args:

role_name: The name of the role. securable_object: The securable object. privileges: The privileges to grant.

Returns:

The updated Role object.

Raises:

NoSuchRoleException: If the role does not exist. NoSuchMetalakeException: If the metalake does not exist. NoSuchMetadataObjectException: If the securable object does not exist. IllegalPrivilegeException: If a privilege is invalid.

grant_roles_to_group(role_names: List[str], group_name: str) → Group

Grant roles to a group.

Args:

role_names: The names of the roles to grant. group_name: The name of the group.

Returns:

The updated Group object.

Raises:

NoSuchRoleException: If the role does not exist. NoSuchGroupException: If the group does not exist. NoSuchMetalakeException: If the metalake does not exist.

grant_roles_to_user(role_names: List[str], user_name: str) → User

Grant roles to a user.

Args:

role_names: The names of the roles to grant. user_name: The name of the user.

Returns:

The updated User object.

Raises:

NoSuchRoleException: If the role does not exist. NoSuchUserException: If the user does not exist. NoSuchMetalakeException: If the metalake does not exist.

list_catalogs() → List[str]

List all the catalogs under this metalake.

Raises:

NoSuchMetalakeException if the metalake with specified namespace does not exist.

Returns:

A list of the catalog names under this metalake.

list_catalogs_info() → List[Catalog]

List all the catalogs with their information under this metalake.

Raises:

NoSuchMetalakeException if the metalake with specified namespace does not exist.

Returns:

A list of Catalog under the specified namespace.

list_group_names() → list[str]

List all group names under this metalake.

Returns:

A list of group name strings.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

list_groups() → list[gravitino.api.authorization.group.Group]

List all groups with details under this metalake.

Returns:

A list of Group objects.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

list_job_templates() → List[JobTemplate]

List all the registered job templates in Gravitino.

Returns:

List of job templates.

list_jobs(job_template_name: str = None) → List[JobHandle]

List all the jobs under this metalake.

Args:

job_template_name: The name of the job template to filter jobs by. If None, all jobs are listed.

Returns:

A list of JobHandle objects representing the jobs.

list_role_names() → list[str]

List all role names under this metalake.

Returns:

A list of role name strings.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

list_tags() → list[str]

List all the tag names under a metalake.

Returns:

list[str]: The list of tag names.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

list_tags_info() → List[Tag]

List tags information under a metalake.

Returns:

list[Tag]: The list of tag information.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

list_user_names() → list[str]

List all user names under this metalake.

Returns:

A list of user name strings.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

list_users() → list[gravitino.api.authorization.user.User]

List all users with details under this metalake.

Returns:

A list of User objects.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

load_catalog(name: str) → Catalog

Load the catalog with specified name.

Args:

name: The name of the catalog to load.

Raises:

NoSuchCatalogException if the catalog with specified name does not exist.

Returns:

The Catalog with specified name.

name() → str

The name of the metalake.

Returns:

str: The name of the metalake.

properties() → Dict[str, str]

The properties of the metalake. Note, this method will return None if the properties are not set.

Returns:

Optional[Dict[str, str]]: The properties of the metalake.

register_job_template(job_template: JobTemplate) → None

Register a job template with the specified job template to Gravitino. The registered job template will be maintained in Gravitino, allowing it to be executed later.

Args:

job_template: The job template to register.

Raises:

JobTemplateAlreadyExists: If a job template with the same name already exists.

remove_group(group: str) → bool

Remove a group from this metalake.

Args:

group: The name of the group.

Returns:

True if the group was removed, False if the group did not exist.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

remove_user(user: str) → bool

Remove a user from this metalake.

Args:

user: The name of the user.

Returns:

True if the user was removed, False if the user did not exist.

Raises:

NoSuchMetalakeException: If the metalake does not exist.

revoke_privileges_from_role(role_name: str, securable_object: SecurableObject, privileges: List[Privilege]) → Role

Revoke privileges from a role on a securable object.

Args:

role_name: The name of the role. securable_object: The securable object. privileges: The privileges to revoke.

Returns:

The updated Role object.

Raises:

NoSuchRoleException: If the role does not exist. NoSuchMetalakeException: If the metalake does not exist. NoSuchMetadataObjectException: If the securable object does not exist. IllegalPrivilegeException: If a privilege is invalid.

revoke_roles_from_group(role_names: List[str], group_name: str) → Group

Revoke roles from a group.

Args:

role_names: The names of the roles to revoke. group_name: The name of the group.

Returns:

The updated Group object.

Raises:

NoSuchRoleException: If the role does not exist. NoSuchGroupException: If the group does not exist. NoSuchMetalakeException: If the metalake does not exist.

revoke_roles_from_user(role_names: List[str], user_name: str) → User

Revoke roles from a user.

Args:

role_names: The names of the roles to revoke. user_name: The name of the user.

Returns:

The updated User object.

Raises:

NoSuchRoleException: If the role does not exist. NoSuchUserException: If the user does not exist. NoSuchMetalakeException: If the metalake does not exist.

run_job(job_template_name: str, job_conf: Dict[str, str]) → JobHandle

Runs a job based on the specified job template and configuration.

Args:

job_template_name: The name of the job template to use for running the job. job_conf: A dictionary containing the configuration for the job.

Returns:

A JobHandle representing the started job.

Raises:

NoSuchJobTemplateException: If no job template with the specified name exists.

set_owner(metadata_object: MetadataObject, owner_name: str, owner_type: Type) → None

Set the owner of a metadata object.

Args:

metadata_object: The metadata object to set the owner for. owner_name: The name of the owner. owner_type: The type of the owner (USER or GROUP).

Raises:

NoSuchMetadataObjectException: If the metadata object does not exist. NotFoundException: If a related resource is not found. MetalakeNotInUseException: If the metalake is not in use. UnsupportedOperationException: If the operation is not supported.