gravitino.api.job.job_template.JobTemplate

class gravitino.api.job.job_template.JobTemplate(builder: BaseBuilder)

Bases: ABC

JobTemplate is a class to define all the configuration parameters for a job.

JobType is enum to define the type of the job, because Gravitino needs different runtime environments to execute different types of jobs, so the job type is required.

Some parameters can be templated, which means that they can be replaced with actual values when running the job, for example, arguments can be { “{input_path}}”, “{{output_path}” }, environment variables can be { “foo”: “{{foo_value}}”, “bar”: “{{bar_value}}” }. the parameters support templating are: arguments, environments.

__init__(builder: BaseBuilder)

Methods

__init__(builder)
job_type()	Returns:

Attributes

arguments	Returns:
comment	Returns:
custom_fields	Returns:
environments	Returns:
executable	Returns:
name	Returns:

class BaseBuilder

Bases: Generic[B, T]

BaseBuilder is a generic class to build a JobTemplate.

abstract build() → T

Builds the JobTemplate instance.

Returns:

An instance of JobTemplate with the configured parameters.

validate()

Validates the job template parameters.

with_arguments(arguments: List[str]) → B

Sets the arguments of the job template.

Args:

arguments: The arguments of the job template.

Returns:

The builder instance for method chaining.

with_comment(comment: str) → B

Sets the comment of the job template.

Args:

comment: The comment of the job template. It can be null or empty.

Returns:

The builder instance for method chaining.

with_custom_fields(custom_fields: Dict[str, str]) → B

Sets the custom fields of the job template.

Args:

custom_fields: The custom fields of the job template.

Returns:

The builder instance for method chaining.

with_environments(environments: Dict[str, str]) → B

Sets the environment variables of the job template.

Args:

environments: The environment variables of the job template.

Returns:

The builder instance for method chaining.

with_executable(executable: str) → B

Sets the executable of the job template.

Args:

executable: The executable of the job template. It must not be null or empty.

Returns:

The builder instance for method chaining.

with_name(name: str) → B

Sets the name of the job template.

Args:

name: The name of the job template. It must not be null or empty.

Returns:

The builder instance for method chaining.

property arguments: List[str]

Returns:

The arguments of the job template, which are the parameters that will be passed to the executable when the job is run.

property comment: str | None

Returns:

The comment of the job template, which can be used to describe the job. This field is optional and can be None.

property custom_fields: Dict[str, str]

Returns:

Custom fields of the job template, which are additional key-value pairs that can be used to store any other information related to the job. This field is optional and can be empty.

property environments: Dict[str, str]

Returns:

The environment variables of the job template, which are key-value pairs that will be set in the environment when the job is run. This can include variables that can be templated with actual values when running the job.

property executable: str

Returns:

The executable of the job template, which is the command or script that will be executed when the job is run. This field is required and must not be empty.

abstract job_type() → JobType

Returns:

The type of the job template, which is an enum value indicating the type of job (e.g., SPARK, SHELL). This is required to determine the runtime environment for executing the job.

property name: str

Returns:

The name of the job template.