Manage tags in Gravitino

Introduction

Starting from 0.6.0, Gravitino introduces a new tag system that allows you to manage tags for metadata objects. Tags are a way to categorize and organize metadata objects in Gravitino.

This document briefly introduces how to use tags in Gravitino by both Gravitino Java client and REST APIs. If you want to know more about the tag system in Gravitino, please refer to the Javadoc and REST API documentation.

Note that current tag system is a basic implementation, some advanced features will be added in the future versions.

info

1. Metadata objects are objects that are managed in Gravitino, such as CATALOG, SCHEMA, TABLE, COLUMN, FILESET, TOPIC, COLUMN, etc. A metadata object is combined by a type and a comma-separated name. For example, a CATAGLOG object has a name "catalog1" with type "CATALOG", a SCHEMA object has a name "catalog1.schema1" with type "SCHEMA", a TABLE object has a name "catalog1.schema1.table1" with type "TABLE".
2. Currently, only CATALOG, SCHEMA, TABLE, FILESET, TOPIC objects can be tagged, tagging on COLUMN will be supported in the future.
3. Tags in Gravitino is inheritable, so listing tags of a metadata object will also list the tags of its parent metadata objects. For example, listing tags of a Table will also list the tags of its parent Schema and Catalog.
4. Same tag can be associated to both parent and child metadata objects. When you list the associated tags of a child metadata object, this tag will be included twice in the result list with different inherited values.

Tag operations

Create new tags

The first step to manage tags is to create new tags. You can create a new tag by providing a tag name, optional comment and properties.

Shell

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "name": "tag1",
  "comment": "This is a tag",
  "properties": {
    "key1": "value1",
    "key2": "value2"
  }
}' http://localhost:8090/api/metalakes/test/tags

Java

GravitinoClient client = ...
Tag tag =
    client.createTag("tag1", "This is a tag", ImmutableMap.of("key1", "value1", "key2", "value2"));

List created tags

You can list all the created tag names as well as tag objects in a metalake in Gravitino.

Shell

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags?details=true

Java

GravitinoClient client = ...
String[] tagNames = client.listTags();

Tag[] tags = client.listTagsInfo();

Get a tag by name

You can get a tag by its name.

Shell

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/tag1

Java

GravitinoClient client = ...
Tag tag = client.getTag("tag1");

Update a tag

Gravitino allows you to update a tag by providing a new tag name, comment and properties.

Shell

curl -X PUT -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "updates": [
    {
      "@type": "rename",
      "newName": "tag2"
    },
    {
      "@type": "updateComment",
      "newComment": "This is an updated tag"
    },
    {
      "@type": "setProperty",
      "property": "key3",
      "value": "value3"
    },
    {
      "@type": "removeProperty",
      "property": "key1"
    }
  ]
}' http://localhost:8090/api/metalakes/test/tags/tag1

Java

GravitinoClient client = ...
Tag tag = client.alterTag(
    "tag1",
    TagChange.rename("tag2"),
    TagChange.updateComment("This is an updated tag"),
    TagChange.setProperty("key3", "value3"),
    TagChange.removeProperty("key1"));

Currently, Gravitino support the following tag changes:

Supported modification	JSON	Java
Rename a tag	{"@type":"rename","newName":"tag_renamed"}	TagChange.rename("tag_renamed")
Update a comment	{"@type":"updateComment","newComment":"new_comment"}	TagChange.updateComment("new_comment")
Set a tag property	{"@type":"setProperty","property":"key1","value":"value1"}	TagChange.setProperty("key1", "value1")
Remove a tag property	{"@type":"removeProperty","property":"key1"}	TagChange.removeProperty("key1")

Delete a tag

You can delete a tag by its name.

Shell

curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/tag2

Java

GravitinoClient client = ...
client.deleteTag("tag2");

Tag associations

Gravitino allows you to associate and disassociate tags with metadata objects. Currently, only CATALOG, SCHEMA, TABLE, FILESET, TOPIC objects can be tagged.

Associate and disassociate tags with a metadata object

You can associate and disassociate tags with a metadata object by providing the object type, object name and tag names.

The request path for REST API is /api/metalakes/{metalake}/tags/{metadataObjectType}/{metadataObjectName}.

Shell

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "tagsToAdd": ["tag1", "tag2"],
  "tagsToRemove": ["tag3"]
}' http://localhost:8090/api/metalakes/test/tags/catalog/catalog1

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "tagsToAdd": ["tag1"]
}' http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1

Java

Catalog catalog1 = ...
catalog1.supportsTags().associateTags(
    new String[] {"tag1", "tag2"},
    new String[] {"tag3"});

Schema schema1 = ...
schema1.supportsTags().associateTags(new String[] {"tag1"}， null);

List associated tags for a metadata object

You can list all the tags associated with a metadata object. The tags in Gravitino are inheritable, so listing tags of a metadata object will also list the tags of its parent metadata objects.

The request path for REST API is /api/metalakes/{metalake}/tags/{metadataObjectType}/{metadataObjectName}.

Shell

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/catalog/catalog1

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/catalog/catalog1?details=true

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1?details=true

Java

Catalog catalog1 = ...
String[] tags = catalog1.supportsTags().listTags();
Tag[] tagsInfo = catalog1.supportsTags().listTagsInfo();

Schema schema1 = ...
String[] tags = schema1.supportsTags().listTags();
Tag[] tagsInfo = schema1.supportsTags().listTagsInfo();

Get an associated tag by name for a metadata object

You can get an associated tag by its name for a metadata object.

The request path for REST API is /api/metalakes/{metalake}/tags/{metadataObjectType}/{metadataObjectName}/{tagName}.

Shell

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/catalog/catalog1/tag1

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/schema/catalog1.schema1/tag1

Java

Catalog catalog1 = ...
Tag tag = catalog1.supportsTags().getTag("tag1");

Schema schema1 = ...
Tag tag = schema1.supportsTags().getTag("tag1");

List metadata objects associated with a tag

You can list all the metadata objects associated with a tag.

Shell

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
http://localhost:8090/api/metalakes/test/tags/tag1/objects

Java

Tag tag = ...
MetadataObject[] objects = tag.associatedObjects().objects();
int count = tag.associatedObjects().count();