AWS Batch Compute Environment

Deploys a MANAGED AWS Batch compute environment with bundled job queues and an optional fair-share scheduling policy. Supports EC2, SPOT, FARGATE, and FARGATE_SPOT resource types with automatic vCPU scaling, VPC networking, and multi-queue priority routing. The component provisions the compute infrastructure, one or more job queues, and an optional scheduling policy in a single resource definition.

What Gets Created

When you deploy an AwsBatchComputeEnvironment resource, OpenMCF provisions:

• Compute Environment — a batch.ComputeEnvironment of type MANAGED with the specified resource type (EC2/SPOT/FARGATE/FARGATE_SPOT), vCPU limits, VPC subnets, security groups, and optional update policy
• Job Queues — one batch.JobQueue per entry in jobQueues, each referencing the compute environment with configurable priority, state, and optional job-state time-limit actions for automatic cancellation of stuck jobs
• Scheduling Policy (optional) — a batch.SchedulingPolicy with fair-share configuration when schedulingPolicy is provided, attached to all bundled job queues for capacity distribution across share identifiers

Prerequisites

• AWS credentials configured via environment variables or OpenMCF provider config
• At least one VPC subnet (private subnets recommended) — use AwsVpc to provision
• A security group allowing outbound access for containers — use AwsSecurityGroup to provision
• For EC2/SPOT types: an ECS instance profile IAM role with AmazonEC2ContainerServiceforEC2Role policy
• For SPOT type: a Spot Fleet IAM role with AmazonEC2SpotFleetTaggingRole policy

Quick Start

Create a file batch.yaml:

apiVersion: aws.openmcf.org/v1
kind: AwsBatchComputeEnvironment
metadata:
  name: my-batch
  labels:
    openmcf.org/provisioner: pulumi
    pulumi.openmcf.org/organization: my-org
    pulumi.openmcf.org/project: my-project
    pulumi.openmcf.org/stack.name: dev.AwsBatchComputeEnvironment.my-batch
spec:
  region: us-west-2
  computeResources:
    type: FARGATE
    maxVcpus: 256
    subnetIds:
      - value: subnet-0a1b2c3d4e5f00001
      - value: subnet-0a1b2c3d4e5f00002
    securityGroupIds:
      - value: sg-0a1b2c3d4e5f00001
  jobQueues:
    - name: default
      priority: 1

Deploy:

openmcf apply -f batch.yaml

This creates a serverless Fargate compute environment with up to 256 vCPUs of capacity and a single default job queue. AWS manages all compute infrastructure — no EC2 instances, patching, or AMI management required.

Configuration Reference

Required Fields

Optional Fields — Top Level

Optional Fields — Compute Resources

Optional Fields — Job Queue

Optional Fields — Scheduling Policy

Outputs

Presets

Design Decisions

Bundling scope. The compute environment and job queues are bundled because a compute environment without a queue is incomplete infrastructure — you cannot submit jobs without a queue. Job definitions are excluded because they represent application-level workloads with independent lifecycles (versioned, frequently updated, reusable across queues).

MANAGED only. UNMANAGED compute environments (where the user manages compute) and EKS-based compute environments are deferred to v2 as they are niche use cases that add significant complexity.

Scheduling policy as top-level. The scheduling policy is defined at the spec level and attached to all bundled queues. Per-queue scheduling policies with external references are deferred to v2.

State defaults. Both compute environment and job queue states default to ENABLED via the OpenMCF middleware default mechanism. State validation is delegated to the AWS API to keep the proto schema simple and forward-compatible.