Creates a definition for a job that monitors data quality and drift¶

Description¶

Creates a definition for a job that monitors data quality and drift. For information about model monitor, see Amazon SageMaker AI Model Monitor.

Usage¶

sagemaker_create_data_quality_job_definition(JobDefinitionName,
  DataQualityBaselineConfig, DataQualityAppSpecification,
  DataQualityJobInput, DataQualityJobOutputConfig, JobResources,
  NetworkConfig, RoleArn, StoppingCondition, Tags)

Arguments¶

JobDefinitionName

[required] The name for the monitoring job definition.
DataQualityBaselineConfig

Configures the constraints and baselines for the monitoring job.
DataQualityAppSpecification

[required] Specifies the container that runs the monitoring job.
DataQualityJobInput

[required] A list of inputs for the monitoring job. Currently endpoints are supported as monitoring inputs.
DataQualityJobOutputConfig

[required] The output configuration for monitoring jobs.
JobResources

[required] Identifies the resources to deploy for a monitoring job.
NetworkConfig

Specifies networking configuration for the monitoring job.
RoleArn

[required] The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker AI can assume to perform tasks on your behalf.
StoppingCondition

Specifies a limit to how long a job can run. When the job reaches the time limit, SageMaker ends the job. Use this API to cap costs.

To stop a training job, SageMaker sends the algorithm the SIGTERM signal, which delays job termination for 120 seconds. Algorithms can use this 120-second window to save the model artifacts, so the results of training are not lost.

The training algorithms provided by SageMaker automatically save the intermediate results of a model training job when possible. This attempt to save artifacts is only a best effort case as model might not be in a state from which it can be saved. For example, if training has just started, the model might not be ready to save. When saved, this intermediate data is a valid model artifact. You can use it to create a model with create_model.

The Neural Topic Model (NTM) currently does not support saving intermediate model artifacts. When training NTMs, make sure that the maximum runtime is sufficient for the training job to complete.
Tags

(Optional) An array of key-value pairs. For more information, see Using Cost Allocation Tags in the Amazon Web Services Billing and Cost Management User Guide.

Value¶

A list with the following syntax:

list(
  JobDefinitionArn = "string"
)

Request syntax¶

svc$create_data_quality_job_definition(
  JobDefinitionName = "string",
  DataQualityBaselineConfig = list(
    BaseliningJobName = "string",
    ConstraintsResource = list(
      S3Uri = "string"
    ),
    StatisticsResource = list(
      S3Uri = "string"
    )
  ),
  DataQualityAppSpecification = list(
    ImageUri = "string",
    ContainerEntrypoint = list(
      "string"
    ),
    ContainerArguments = list(
      "string"
    ),
    RecordPreprocessorSourceUri = "string",
    PostAnalyticsProcessorSourceUri = "string",
    Environment = list(
      "string"
    )
  ),
  DataQualityJobInput = list(
    EndpointInput = list(
      EndpointName = "string",
      LocalPath = "string",
      S3InputMode = "Pipe"|"File",
      S3DataDistributionType = "FullyReplicated"|"ShardedByS3Key",
      FeaturesAttribute = "string",
      InferenceAttribute = "string",
      ProbabilityAttribute = "string",
      ProbabilityThresholdAttribute = 123.0,
      StartTimeOffset = "string",
      EndTimeOffset = "string",
      ExcludeFeaturesAttribute = "string"
    ),
    BatchTransformInput = list(
      DataCapturedDestinationS3Uri = "string",
      DatasetFormat = list(
        Csv = list(
          Header = TRUE|FALSE
        ),
        Json = list(
          Line = TRUE|FALSE
        ),
        Parquet = list()
      ),
      LocalPath = "string",
      S3InputMode = "Pipe"|"File",
      S3DataDistributionType = "FullyReplicated"|"ShardedByS3Key",
      FeaturesAttribute = "string",
      InferenceAttribute = "string",
      ProbabilityAttribute = "string",
      ProbabilityThresholdAttribute = 123.0,
      StartTimeOffset = "string",
      EndTimeOffset = "string",
      ExcludeFeaturesAttribute = "string"
    )
  ),
  DataQualityJobOutputConfig = list(
    MonitoringOutputs = list(
      list(
        S3Output = list(
          S3Uri = "string",
          LocalPath = "string",
          S3UploadMode = "Continuous"|"EndOfJob"
        )
      )
    ),
    KmsKeyId = "string"
  ),
  JobResources = list(
    ClusterConfig = list(
      InstanceCount = 123,
      InstanceType = "ml.t3.medium"|"ml.t3.large"|"ml.t3.xlarge"|"ml.t3.2xlarge"|"ml.m4.xlarge"|"ml.m4.2xlarge"|"ml.m4.4xlarge"|"ml.m4.10xlarge"|"ml.m4.16xlarge"|"ml.c4.xlarge"|"ml.c4.2xlarge"|"ml.c4.4xlarge"|"ml.c4.8xlarge"|"ml.p2.xlarge"|"ml.p2.8xlarge"|"ml.p2.16xlarge"|"ml.p3.2xlarge"|"ml.p3.8xlarge"|"ml.p3.16xlarge"|"ml.c5.xlarge"|"ml.c5.2xlarge"|"ml.c5.4xlarge"|"ml.c5.9xlarge"|"ml.c5.18xlarge"|"ml.m5.large"|"ml.m5.xlarge"|"ml.m5.2xlarge"|"ml.m5.4xlarge"|"ml.m5.12xlarge"|"ml.m5.24xlarge"|"ml.r5.large"|"ml.r5.xlarge"|"ml.r5.2xlarge"|"ml.r5.4xlarge"|"ml.r5.8xlarge"|"ml.r5.12xlarge"|"ml.r5.16xlarge"|"ml.r5.24xlarge"|"ml.g4dn.xlarge"|"ml.g4dn.2xlarge"|"ml.g4dn.4xlarge"|"ml.g4dn.8xlarge"|"ml.g4dn.12xlarge"|"ml.g4dn.16xlarge"|"ml.g5.xlarge"|"ml.g5.2xlarge"|"ml.g5.4xlarge"|"ml.g5.8xlarge"|"ml.g5.16xlarge"|"ml.g5.12xlarge"|"ml.g5.24xlarge"|"ml.g5.48xlarge"|"ml.r5d.large"|"ml.r5d.xlarge"|"ml.r5d.2xlarge"|"ml.r5d.4xlarge"|"ml.r5d.8xlarge"|"ml.r5d.12xlarge"|"ml.r5d.16xlarge"|"ml.r5d.24xlarge"|"ml.g6.xlarge"|"ml.g6.2xlarge"|"ml.g6.4xlarge"|"ml.g6.8xlarge"|"ml.g6.12xlarge"|"ml.g6.16xlarge"|"ml.g6.24xlarge"|"ml.g6.48xlarge"|"ml.g6e.xlarge"|"ml.g6e.2xlarge"|"ml.g6e.4xlarge"|"ml.g6e.8xlarge"|"ml.g6e.12xlarge"|"ml.g6e.16xlarge"|"ml.g6e.24xlarge"|"ml.g6e.48xlarge"|"ml.m6i.large"|"ml.m6i.xlarge"|"ml.m6i.2xlarge"|"ml.m6i.4xlarge"|"ml.m6i.8xlarge"|"ml.m6i.12xlarge"|"ml.m6i.16xlarge"|"ml.m6i.24xlarge"|"ml.m6i.32xlarge"|"ml.c6i.xlarge"|"ml.c6i.2xlarge"|"ml.c6i.4xlarge"|"ml.c6i.8xlarge"|"ml.c6i.12xlarge"|"ml.c6i.16xlarge"|"ml.c6i.24xlarge"|"ml.c6i.32xlarge"|"ml.m7i.large"|"ml.m7i.xlarge"|"ml.m7i.2xlarge"|"ml.m7i.4xlarge"|"ml.m7i.8xlarge"|"ml.m7i.12xlarge"|"ml.m7i.16xlarge"|"ml.m7i.24xlarge"|"ml.m7i.48xlarge"|"ml.c7i.large"|"ml.c7i.xlarge"|"ml.c7i.2xlarge"|"ml.c7i.4xlarge"|"ml.c7i.8xlarge"|"ml.c7i.12xlarge"|"ml.c7i.16xlarge"|"ml.c7i.24xlarge"|"ml.c7i.48xlarge"|"ml.r7i.large"|"ml.r7i.xlarge"|"ml.r7i.2xlarge"|"ml.r7i.4xlarge"|"ml.r7i.8xlarge"|"ml.r7i.12xlarge"|"ml.r7i.16xlarge"|"ml.r7i.24xlarge"|"ml.r7i.48xlarge"|"ml.p5.4xlarge"|"ml.g7e.2xlarge"|"ml.g7e.4xlarge"|"ml.g7e.8xlarge"|"ml.g7e.12xlarge"|"ml.g7e.24xlarge"|"ml.g7e.48xlarge"|"ml.g7.2xlarge"|"ml.g7.4xlarge"|"ml.g7.8xlarge"|"ml.g7.12xlarge"|"ml.g7.24xlarge"|"ml.g7.48xlarge"|"ml.r6i.large"|"ml.r6i.xlarge"|"ml.r6i.2xlarge"|"ml.r6i.4xlarge"|"ml.r6i.8xlarge"|"ml.r6i.12xlarge"|"ml.r6i.16xlarge"|"ml.r6i.24xlarge"|"ml.r6i.32xlarge"|"ml.m8i.large"|"ml.m8i.xlarge"|"ml.m8i.2xlarge"|"ml.m8i.4xlarge"|"ml.m8i.8xlarge"|"ml.m8i.12xlarge"|"ml.m8i.16xlarge"|"ml.m8i.24xlarge"|"ml.m8i.32xlarge"|"ml.m8i.48xlarge"|"ml.m8i.96xlarge"|"ml.c8i.xlarge"|"ml.c8i.2xlarge"|"ml.c8i.4xlarge"|"ml.c8i.8xlarge"|"ml.c8i.12xlarge"|"ml.c8i.16xlarge"|"ml.c8i.24xlarge"|"ml.c8i.32xlarge"|"ml.c8i.48xlarge"|"ml.c8i.96xlarge"|"ml.r8i.large"|"ml.r8i.xlarge"|"ml.r8i.2xlarge"|"ml.r8i.4xlarge"|"ml.r8i.8xlarge"|"ml.r8i.12xlarge"|"ml.r8i.16xlarge"|"ml.r8i.24xlarge"|"ml.r8i.32xlarge"|"ml.r8i.48xlarge"|"ml.r8i.96xlarge"|"ml.m7i-flex.large"|"ml.m7i-flex.xlarge"|"ml.m7i-flex.2xlarge"|"ml.m7i-flex.4xlarge"|"ml.m7i-flex.8xlarge"|"ml.m7i-flex.12xlarge"|"ml.m7i-flex.16xlarge"|"ml.c7i-flex.xlarge"|"ml.c7i-flex.2xlarge"|"ml.c7i-flex.4xlarge"|"ml.c7i-flex.8xlarge"|"ml.c7i-flex.12xlarge"|"ml.c7i-flex.16xlarge"|"ml.m8i-flex.large"|"ml.m8i-flex.xlarge"|"ml.m8i-flex.2xlarge"|"ml.m8i-flex.4xlarge"|"ml.m8i-flex.8xlarge"|"ml.m8i-flex.12xlarge"|"ml.m8i-flex.16xlarge"|"ml.c8i-flex.xlarge"|"ml.c8i-flex.2xlarge"|"ml.c8i-flex.4xlarge"|"ml.c8i-flex.8xlarge"|"ml.c8i-flex.12xlarge"|"ml.c8i-flex.16xlarge"|"ml.r8i-flex.large"|"ml.r8i-flex.xlarge"|"ml.r8i-flex.2xlarge"|"ml.r8i-flex.4xlarge"|"ml.r8i-flex.8xlarge"|"ml.r8i-flex.12xlarge"|"ml.r8i-flex.16xlarge",
      VolumeSizeInGB = 123,
      VolumeKmsKeyId = "string"
    )
  ),
  NetworkConfig = list(
    EnableInterContainerTrafficEncryption = TRUE|FALSE,
    EnableNetworkIsolation = TRUE|FALSE,
    VpcConfig = list(
      SecurityGroupIds = list(
        "string"
      ),
      Subnets = list(
        "string"
      )
    )
  ),
  RoleArn = "string",
  StoppingCondition = list(
    MaxRuntimeInSeconds = 123
  ),
  Tags = list(
    list(
      Key = "string",
      Value = "string"
    )
  )
)