1
Training time
Bayesian Optimization-guided fine-tuning
The success-rate landscape over initial states is unknown, non-convex, non-differentiable, and expensive to evaluate — and it moves as the policy learns. We model it with a Gaussian Process over θ, the initial xyz position of the plug relative to its goal:
Ĵi(θ) ~ 𝒢𝒫( μ(θ), k(θ, θ′) )
Each iteration re-evaluates the current policy to update the GP, then an acquisition function proposes where to fine-tune next:
θ′ = arg maxθ A(θ)
We compared three acquisition functions across all 26 steps of the 5 multi-part assemblies (5 seeds, 1000 trials each). UCB wins — exploring high-uncertainty states beats chasing the current best.
-
UCB
A = μ(θ) + βσ(θ) · 94.41% (β = 2.0)
-
PI
A = Φ( (μ(θ) − J⁺) ∕ σ(θ) ) · 91.82%
-
EI
A = (μ(θ) − J⁺)Φ(Z) + σ(θ)φ(Z) · 91.12%
Algorithm 1 — per assembly step
- Train a specialist policy πi with on-policy RL + an imitation objective (AutoMate)
- Evaluate the success-rate distribution Ji(θ) over plug initializations
- Propose samples θ′ by maximizing the acquisition function
- Fine-tune with episodes initialized at θ′; repeat until convergence
2
Deployment time
GMM sampling over initial conditions
Fine-tuning flattens the success landscape; deployment exploits whatever structure remains. Offline, we roll out the policy from N = 1000 uniformly-sampled initial poses, keep the successful ones, and fit a Gaussian Mixture Model to that set:
pGMM(x) = Σk ξk 𝒩( x | μk, Σk )
Online, we draw M = 1000 candidates from the mixture and start the robot at the highest-density one — no brute-force search over the initialization space:
x* = arg maxj pGMM(x̃j)
This costs nothing at training time and needs no online adaptation. On its own it lifts mean per-step success from 83.61% to 94.50%; combined with fine-tuning it reaches 96.35%. For the real robot, the GMM is fit entirely from simulation rollouts and transferred zero-shot.
Algorithm 2 — offline, then online
- Sample N initializations uniformly from the initialization space
- Execute π from each; record binary success yi
- Fit a GMM with K components to the successful set
- Sample M candidates, select the max-density one, execute