Start from locked, public inputs.
The tested dependency is GADE-Tech/OSWorld at the commit above. That revision is an upstream OSWorld commit with no GADE-specific source changes. The guest comes from the public Ubuntu.qcow2.zip.
Prepare the host checkout.
Python 3.12 is recommended for the pinned OSWorld environment. Keep the benchmark checkout adjacent to this repository or set OSWORLD_ROOT explicitly.
git clone https://github.com/GADE-Tech/OSWorld.git
git -C OSWorld checkout b7db4d8c85d9e95e0b1db44de5bec954cf37f0cf
git clone https://github.com/GADE-Tech/GADE-CUA-Evolve.git
cd GADE-CUA-Evolve
uv sync --extra google --extra dev
uv pip install -e ../OSWorld
export OSWORLD_ROOT="$(cd ../OSWorld && pwd)"
cp .env.example .env
Secrets belong only in the process environment or local untracked .env. YAML contains non-sensitive configuration.
Create the network and image prerequisites.
Enable ECS, snapshots, and TOS in one region. Create a VPC, subnet, and security group. The TOS bucket used for image import must be in the target ECS image region, and ECS needs explicit service-account authorization to read the object.
| Port | Allowed source | Purpose |
|---|---|---|
| TCP 5000 | Runner security group / private CIDR | OSWorld control API |
| TCP 9222 | Runner security group / private CIDR | Chrome DevTools proxy |
| TCP 5910 | Optional trusted operator only | noVNC debugging |
| TCP 22 | Optional trusted operator only | Image provisioning |
Default disposable-run path
Download, verify, and upload QCOW2.
Install curl, unzip, qemu-img, and Volcengine tosutil. Use a volume with at least 40 GiB free.
./scripts/prepare_volcengine_image.sh \
--work-dir /data/osworld-image \
--upload \
--tos-bucket YOUR_PRIVATE_BUCKET \
--tos-key osworld-v1/Ubuntu.qcow2
The script resumes partial downloads, enforces SHA-256, runs qemu-img info/check, uploads with tosutil cp -vchecksum, and prints the object URL required by the ECS import wizard. Use --dry-run to inspect locked inputs without downloading.
Import the base image.
- Open ECS → Images → Custom images → Import image.
- Select the same region as the TOS bucket.
- Paste the object URL printed by the script.
- Select Linux/Ubuntu and the OS version matching the guest.
- Import as a system-disk image, enable detection, and wait for both tasks to complete.
Image metadata selection and the first ECS/TOS authorization intentionally remain manual console gates.
Provision Coder dependencies.
Create one temporary ECS instance from the imported base, connect through a trusted private path or temporarily restricted SSH, and run:
./scripts/provision_osworld_coder.sh
The script installs curl, jq, sqlite3, ImageMagick, xdotool, and common Office/PDF Python packages. It verifies Python modules, Bash tooling, the OSWorld service, and guest ports 5000/9222.
systemctl status osworld_server.service
curl --fail http://127.0.0.1:5000/probe
curl --fail http://127.0.0.1:9222/json/version
Sanitize, stop, and create the image.
Remove task artifacts and inspect browser profiles, home directories, /root, and temporary storage. Never use a guest that has held real benchmark accounts as a public reproduction base.
./scripts/provision_osworld_coder.sh --finalize
The guarded finalizer refuses common .env, OAuth credential, or private-key files and credential assignments. It clears histories, Coder temporary files, pip cache, and Cloud-Init instance state.
Scale private workers through controlled egress.
For high-volume inference and trajectory collection, remove per-worker EIPs. Place runner hosts and guests in the same VPC, address workers over private IPs, and route required HTTP(S) traffic through hardened Squid gateways.
Required infrastructure edges
- Runner security group → workers on TCP 5000/9222 only.
- Worker security group → Squid on TCP 3128 only.
- Squid → one controlled EIP or managed NAT path.
- Runners → model endpoints and TOS, preferably over private endpoints.
- Persist trajectories before queue acknowledgement and worker deletion.
HTTP_PROXY=http://squid.service.private:3128
HTTPS_PROXY=http://squid.service.private:3128
NO_PROXY=127.0.0.1,localhost,.service.private
Chromium and desktop applications may require system or application-specific proxy policy. Environment variables alone do not guarantee GUI traffic uses Squid. Restrict CONNECT ports and destinations, redact sensitive log data, end Squid policy with http_access deny all, and use redundant gateways for long jobs.
Shard the runner layer
gadecua batch --env osworldv1 \
--manifest "$OSWORLD_ROOT/evaluation_examples/test_nogdrive.json" \
--shard-index 0 --num-shards 8 --workers 4 \
--resume --infra-retries 2 --evaluate \
--output-dir results/rollouts/shard-00
Run shard indices 0..7 on separate supervised runners. Size concurrency from the lowest subnet-IP, ECS, disk, model-QPS, proxy-throughput, or budget limit. This is an operator topology rather than a current --no-eip switch; the pinned OSWorld dependency remains unmodified.
Configure GADE CUA Evolve.
Copy .env.example to local .env and replace placeholders without committing the result.
GEMINI_AK=...
GEMINI_MODEL=gemini-3.5-flash-lite
VOLCENGINE_ACCESS_KEY_ID=...
VOLCENGINE_SECRET_ACCESS_KEY=...
VOLCENGINE_REGION=...
VOLCENGINE_ZONE_ID=...
VOLCENGINE_IMAGE_ID=...
VOLCENGINE_INSTANCE_TYPE=...
VOLCENGINE_SUBNET_ID=...
VOLCENGINE_SECURITY_GROUP_ID=...
VOLCENGINE_DEFAULT_PASSWORD=...
The VM password is resolved only at runtime and supplied consistently to OSWorld, the Planner, and Coder redaction. It must not appear in YAML.
Probe, smoke-test, and clean up.
gade-cua env probe \
--config configs/volcengine_gta15_gemini.yaml \
--check-code --check-services \
--output probe.png
gadecua --env osworldv1 \
--task chrome/2ae9ba84-3a0d-4d4c-8338-3a1478dc5fe3 \
--set loop.max_steps=3 --verbose
The probe captures a screenshot, runs harmless Python and Bash in the guest, checks services, and closes the environment in finally. After interrupted runs, verify temporary ECS instances and EIPs are gone.
Reproduction record
Record the GADE commit, OSWorld commit, image revision/checksum, private final image ID, region, instance type, models, manifest, overrides, and ARM/Coder status. Never include secrets.