gravitino.api.job.spark_job_template.SparkJobTemplate¶

class gravitino.api.job.spark_job_template.SparkJobTemplate(builder: Builder)¶

Bases: JobTemplate

Represents a job template for executing Spark jobs.

__init__(builder: Builder)¶: Initializes a SparkJobTemplate instance.

Methods

`__init__`(builder)	Initializes a SparkJobTemplate instance.
`builder`()	Creates a new builder instance for constructing a SparkJobTemplate.
`job_type`()	Returns the type of the job, which is SPARK for this template.

Attributes

`archives`	Returns the list of archives associated with this Spark job template.
`arguments`	Returns:
`class_name`	Returns the class name of the Spark job.
`comment`	Returns:
`configs`	Returns the configuration settings for this Spark job template.
`custom_fields`	Returns:
`environments`	Returns:
`executable`	Returns:
`files`	Returns the list of files associated with this Spark job template.
`jars`	Returns the list of JAR files associated with this Spark job template.
`name`	Returns:

class BaseBuilder¶

Bases: Generic[B, T]

BaseBuilder is a generic class to build a JobTemplate.

abstract build() → T¶

Builds the JobTemplate instance.

Returns:: An instance of JobTemplate with the configured parameters.

validate()¶: Validates the job template parameters.

with_arguments(arguments: List[str]) → B¶

Sets the arguments of the job template.

Args:: arguments: The arguments of the job template.
Returns:: The builder instance for method chaining.

with_comment(comment: str) → B¶

Sets the comment of the job template.

Args:: comment: The comment of the job template. It can be null or empty.
Returns:: The builder instance for method chaining.

with_custom_fields(custom_fields: Dict[str, str]) → B¶

Sets the custom fields of the job template.

Args:: custom_fields: The custom fields of the job template.
Returns:: The builder instance for method chaining.

with_environments(environments: Dict[str, str]) → B¶

Sets the environment variables of the job template.

Args:: environments: The environment variables of the job template.
Returns:: The builder instance for method chaining.

with_executable(executable: str) → B¶

Sets the executable of the job template.

Args:: executable: The executable of the job template. It must not be null or empty.
Returns:: The builder instance for method chaining.

with_name(name: str) → B¶

Sets the name of the job template.

Args:: name: The name of the job template. It must not be null or empty.
Returns:: The builder instance for method chaining.

class Builder¶

Bases: BaseBuilder[SparkJobTemplate.Builder, SparkJobTemplate]

Builder class for creating SparkJobTemplate instances.

build() → SparkJobTemplate¶: Builds and returns a SparkJobTemplate instance.

validate()¶: Validates the SparkJobTemplate properties.

with_archives(archives: List[str]) → Builder¶

Sets the archives for this Spark job template.

Args:: archives: A list of archive file paths to be included in the Spark job.
Returns:: The builder instance for method chaining.

with_arguments(arguments: List[str]) → B¶

Sets the arguments of the job template.

Args:: arguments: The arguments of the job template.
Returns:: The builder instance for method chaining.

with_class_name(class_name: str) → Builder¶

Sets the class name for this Spark job template.

Args:: class_name: The fully qualified name of the Spark job class.
Returns:: The builder instance for method chaining.

with_comment(comment: str) → B¶

Sets the comment of the job template.

Args:: comment: The comment of the job template. It can be null or empty.
Returns:: The builder instance for method chaining.

with_configs(configs: Dict[str, str]) → Builder¶

Sets the configuration settings for this Spark job template.

Args:: configs: A dictionary of configuration key-value pairs to be applied to the Spark job.
Returns:: The builder instance for method chaining.

with_custom_fields(custom_fields: Dict[str, str]) → B¶

Sets the custom fields of the job template.

Args:: custom_fields: The custom fields of the job template.
Returns:: The builder instance for method chaining.

with_environments(environments: Dict[str, str]) → B¶

Sets the environment variables of the job template.

Args:: environments: The environment variables of the job template.
Returns:: The builder instance for method chaining.

with_executable(executable: str) → B¶

Sets the executable of the job template.

Args:: executable: The executable of the job template. It must not be null or empty.
Returns:: The builder instance for method chaining.

with_files(files: List[str]) → Builder¶

Sets the files for this Spark job template.

Args:: files: A list of file paths to be included in the Spark job.
Returns:: The builder instance for method chaining.

with_jars(jars: List[str]) → Builder¶

Sets the JAR files for this Spark job template.

Args:: jars: A list of JAR file paths to be included in the Spark job.
Returns:: The builder instance for method chaining.

with_name(name: str) → B¶

Sets the name of the job template.

Args:: name: The name of the job template. It must not be null or empty.
Returns:: The builder instance for method chaining.

property archives: List[str]¶

Returns the list of archives associated with this Spark job template.

Returns:: List of archive file paths.

property arguments: List[str]¶

Returns:: The arguments of the job template, which are the parameters that will be passed to the executable when the job is run.

static builder() → Builder¶

Creates a new builder instance for constructing a SparkJobTemplate.

Returns:: SparkJobTemplate.Builder: A new builder instance.

property class_name: str¶

Returns the class name of the Spark job.

Returns:: the class name as a string.

property comment: str | None¶

Returns:: The comment of the job template, which can be used to describe the job. This field is optional and can be None.

property configs: Dict[str, str]¶

Returns the configuration settings for this Spark job template.

Returns:: A dictionary of configuration key-value pairs.

property custom_fields: Dict[str, str]¶

Returns:: Custom fields of the job template, which are additional key-value pairs that can be used to store any other information related to the job. This field is optional and can be empty.

property environments: Dict[str, str]¶

Returns:: The environment variables of the job template, which are key-value pairs that will be set in the environment when the job is run. This can include variables that can be templated with actual values when running the job.

property executable: str¶

Returns:: The executable of the job template, which is the command or script that will be executed when the job is run. This field is required and must not be empty.

property files: List[str]¶

Returns the list of files associated with this Spark job template.

Returns:: List of file paths.

property jars: List[str]¶

Returns the list of JAR files associated with this Spark job template.

Returns:: List of JAR file paths.

job_type() → JobType¶

Returns the type of the job, which is SPARK for this template.

Returns:: JobType.SPARK: Indicates that this is a Spark job template.

property name: str¶

Returns:: The name of the job template.