Package org.apache.gravitino.policy

Class IcebergDataCompactionContent

java.lang.Object
org.apache.gravitino.policy.IcebergDataCompactionContent
All Implemented Interfaces:
PolicyContent
public class IcebergDataCompactionContent extends Object implements PolicyContent
Built-in policy content for Iceberg compaction strategy.

  • Field Details

    • STRATEGY_TYPE_KEY

      public static final String STRATEGY_TYPE_KEY
      Property key for strategy type.
      See Also:

    • STRATEGY_TYPE_VALUE

      public static final String STRATEGY_TYPE_VALUE
      Strategy type value for iceberg data compaction.
      See Also:

    • JOB_TEMPLATE_NAME_KEY

      public static final String JOB_TEMPLATE_NAME_KEY
      Property key for job template name.
      See Also:

    • JOB_TEMPLATE_NAME_VALUE

      public static final String JOB_TEMPLATE_NAME_VALUE
      Built-in job template name for Iceberg rewrite data files.
      See Also:

    • JOB_OPTIONS_PREFIX

      public static final String JOB_OPTIONS_PREFIX
      Prefix for rewrite options propagated to job options.
      See Also:

    • TRIGGER_EXPR_KEY

      public static final String TRIGGER_EXPR_KEY
      Rule key for trigger expression.
      See Also:

    • SCORE_EXPR_KEY

      public static final String SCORE_EXPR_KEY
      Rule key for score expression.
      See Also:

    • MIN_DATA_FILE_MSE_KEY

      public static final String MIN_DATA_FILE_MSE_KEY
      Rule key for minimum data file MSE threshold.
      See Also:

    • MIN_DELETE_FILE_NUMBER_KEY

      public static final String MIN_DELETE_FILE_NUMBER_KEY
      Rule key for minimum delete file count threshold.
      See Also:

    • DATA_FILE_MSE_WEIGHT_KEY

      public static final String DATA_FILE_MSE_WEIGHT_KEY
      Rule key for data file MSE score weight.
      See Also:

    • DELETE_FILE_NUMBER_WEIGHT_KEY

      public static final String DELETE_FILE_NUMBER_WEIGHT_KEY
      Rule key for delete file number score weight.
      See Also:

    • MAX_PARTITION_NUM_KEY

      public static final String MAX_PARTITION_NUM_KEY
      Rule key for max partition number selected for compaction.
      See Also:

    • DATA_FILE_MSE_METRIC

      public static final String DATA_FILE_MSE_METRIC
      Metric name for data file MSE.
      See Also:

    • DELETE_FILE_NUMBER_METRIC

      public static final String DELETE_FILE_NUMBER_METRIC
      Metric name for delete file number.
      See Also:

    • DEFAULT_MIN_DATA_FILE_MSE

      public static final long DEFAULT_MIN_DATA_FILE_MSE
      Default minimum threshold for data file MSE metric, equals (128 MiB * 0.15)^2.
      See Also:

    • DEFAULT_MIN_DELETE_FILE_NUMBER

      public static final long DEFAULT_MIN_DELETE_FILE_NUMBER
      Default minimum threshold for delete file number metric.
      See Also:

    • DEFAULT_DATA_FILE_MSE_WEIGHT

      public static final long DEFAULT_DATA_FILE_MSE_WEIGHT
      Default score weight for data file MSE.
      See Also:

    • DEFAULT_DELETE_FILE_NUMBER_WEIGHT

      public static final long DEFAULT_DELETE_FILE_NUMBER_WEIGHT
      Default score weight for delete file number.
      See Also:

    • DEFAULT_MAX_PARTITION_NUM

      public static final long DEFAULT_MAX_PARTITION_NUM
      Default max partition number for compaction.
      See Also:

    • DEFAULT_REWRITE_OPTIONS

      public static final Map<String,String> DEFAULT_REWRITE_OPTIONS
      Default rewrite options for Iceberg rewrite data files.

  • Method Details

    • minDataFileMse

      public Long minDataFileMse()
      Returns the minimum threshold for "custom-data-file-mse".
      Returns:
      minimum data file MSE threshold

    • minDeleteFileNumber

      public Long minDeleteFileNumber()
      Returns the minimum threshold for "custom-delete-file-number".
      Returns:
      minimum delete file number threshold

    • dataFileMseWeight

      public Long dataFileMseWeight()
      Returns the weight used by "custom-data-file-mse" in score expression.
      Returns:
      data file MSE score weight

    • deleteFileNumberWeight

      public Long deleteFileNumberWeight()
      Returns the weight used by "custom-delete-file-number" in score expression.
      Returns:
      delete file number score weight

    • maxPartitionNum

      public Long maxPartitionNum()
      Returns max partition number selected for compaction.
      Returns:
      max partition number

    • rewriteOptions

      public Map<String,String> rewriteOptions()
      Returns rewrite options that are expanded to job.options.* rule entries.
      Returns:
      rewrite options

    • supportedObjectTypes

      public Set<MetadataObject.Type> supportedObjectTypes()
      Specified by:
      supportedObjectTypes in interface PolicyContent
      Returns:
      the set of metadata object types that the policy can be applied to

    • properties

      public Map<String,String> properties()
      Specified by:
      properties in interface PolicyContent
      Returns:
      The additional properties of the policy.

    • rules

      public Map<String,Object> rules()
      Description copied from interface: PolicyContent
      A convenience method to get all rules in the policy content.
      Specified by:
      rules in interface PolicyContent
      Returns:
      A map of rule names to their corresponding rule objects.

    • validate

      public void validate() throws IllegalArgumentException
      Description copied from interface: PolicyContent
      Validates the policy content.
      Specified by:
      validate in interface PolicyContent
      Throws:
      IllegalArgumentException - if the content is invalid.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object