Skip to content

Create Evaluation Job

bedrock_create_evaluation_job R Documentation

API operation for creating and managing Amazon Bedrock automatic model evaluation jobs and model evaluation jobs that use human workers

Description

API operation for creating and managing Amazon Bedrock automatic model evaluation jobs and model evaluation jobs that use human workers. To learn more about the requirements for creating a model evaluation job see, Model evaluation.

Usage

bedrock_create_evaluation_job(jobName, jobDescription,
  clientRequestToken, roleArn, customerEncryptionKeyId, jobTags,
  evaluationConfig, inferenceConfig, outputDataConfig)

Arguments

jobName

[required] The name of the model evaluation job. Model evaluation job names must unique with your AWS account, and your account's AWS region.

jobDescription

A description of the model evaluation job.

clientRequestToken

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.

roleArn

[required] The Amazon Resource Name (ARN) of an IAM service role that Amazon Bedrock can assume to perform tasks on your behalf. The service role must have Amazon Bedrock as the service principal, and provide access to any Amazon S3 buckets specified in the EvaluationConfig object. To pass this role to Amazon Bedrock, the caller of this API must have the iam:PassRole permission. To learn more about the required permissions, see Required permissions.

customerEncryptionKeyId

Specify your customer managed key ARN that will be used to encrypt your model evaluation job.

jobTags

Tags to attach to the model evaluation job.

evaluationConfig

[required] Specifies whether the model evaluation job is automatic or uses human worker.

inferenceConfig

[required] Specify the models you want to use in your model evaluation job. Automatic model evaluation jobs support a single model, and model evaluation job that use human workers support two models.

outputDataConfig

[required] An object that defines where the results of model evaluation job will be saved in Amazon S3.

Value

A list with the following syntax: 

list(
  jobArn = "string"
)

Request syntax

svc$create_evaluation_job(
  jobName = "string",
  jobDescription = "string",
  clientRequestToken = "string",
  roleArn = "string",
  customerEncryptionKeyId = "string",
  jobTags = list(
    list(
      key = "string",
      value = "string"
    )
  ),
  evaluationConfig = list(
    automated = list(
      datasetMetricConfigs = list(
        list(
          taskType = "Summarization"|"Classification"|"QuestionAndAnswer"|"Generation"|"Custom",
          dataset = list(
            name = "string",
            datasetLocation = list(
              s3Uri = "string"
            )
          ),
          metricNames = list(
            "string"
          )
        )
      )
    ),
    human = list(
      humanWorkflowConfig = list(
        flowDefinitionArn = "string",
        instructions = "string"
      ),
      customMetrics = list(
        list(
          name = "string",
          description = "string",
          ratingMethod = "string"
        )
      ),
      datasetMetricConfigs = list(
        list(
          taskType = "Summarization"|"Classification"|"QuestionAndAnswer"|"Generation"|"Custom",
          dataset = list(
            name = "string",
            datasetLocation = list(
              s3Uri = "string"
            )
          ),
          metricNames = list(
            "string"
          )
        )
      )
    )
  ),
  inferenceConfig = list(
    models = list(
      list(
        bedrockModel = list(
          modelIdentifier = "string",
          inferenceParams = "string"
        )
      )
    )
  ),
  outputDataConfig = list(
    s3Uri = "string"
  )
)