The "Strict Optimization" or "Inner Loop" is the defining feature of the Zampini method. Unlike standard guidance, which merely "nudges" the generation once per step, this method essentially pauses the diffusion process at each timestep to fix the image until it meets the rules.
1. How the Inner Loop Works
In standard diffusion, you move from timestep $t$ to $t-1$ in one go. In this work, the authors introduce a Proximal Langevin Dynamics scheme that splits the process into an "Outer Loop" (the diffusion steps) and an "Inner Loop" (the constraint correction).
The Pause
At timestep $t$, the model predicts a noisy latent $\mathbf{z}_t$.
Instead of immediately moving to the next step, the algorithm enters a
while
loop.
The Check
It decodes the latent to image space and checks the constraint violation $g(\mathcal{D}(\mathbf{z}))$.
The Loop
If violation > tolerance $\varepsilon$, it performs Gradient Descent on the latent variable:
- Calculates the gradient of the violation.
- Updates $\mathbf{z}_t$ to reduce that violation.
- Crucially: Repeats this check-update cycle inside the same timestep.
The Resume
Only once the latent is "fixed" (strictly feasible within tolerance) does it proceed to step $t-1$.
2. Is it possible to correct until "Strictly Feasible"?
Yes, but the paper distinguishes between two types of constraints.
Case A: Convex Constraints
Guaranteed Strictness
If the constraint is convex (e.g., microstructure porosity), the method uses a closed-form projection. Because the constraint set is convex, there is a guaranteed single "closest point".
Result: The paper proves (Theorem 4.1) convergence to the feasible set. In experiments, they achieved 0% of samples having >10% error.
Case B: Non-Convex / Black-Box
Arbitrary Precision
For complex constraints (e.g., physics simulators), strict mathematical feasibility is harder, but the "inner loop" allows for arbitrary precision.
You set a tolerance $\varepsilon$. The loop continues optimizing until error < $\varepsilon$. You can force the error to be incredibly small by running the inner loop longer, essentially trading computational time for strictness.
3. The Augmented Lagrangian Method
For non-convex constraints where a simple projection is impossible (Appendix D), the paper employs an Augmented Lagrangian Method (ALM). This involves tracking "Dual Variables" ($\lambda, \mu$) that strictly enforce equality constraints.
The Lagrangian Objective
The method minimizes an objective that combines distance from the original sample with penalty terms:
Dual Variable Updates
The dual variables are updated in an outer loop (Dual Ascent) surrounding the gradient descent on $y$:
-
1. Primal Update ($y$):
$y \leftarrow y - \gamma \nabla_y \mathcal{L}_{ALM}(y, \lambda, \mu)$ -
2. Dual Ascent ($\lambda$):
$\lambda \leftarrow \lambda + \mu \tilde{\phi}(y)$
Accumulates penalty for persistent violations within the step. -
3. Penalty Hardening ($\mu$):
$\mu \leftarrow \min(\alpha \mu, \mu_{max})$
Increases the quadratic weight to force strict satisfaction.
Algorithm Logic (Inner Loop)
Input: x_t, λ, μ, γ, α, δ
y ← D(z_t)
while φ(y) > δ do # Until strictly feasible
for j ← 1 to max_inner_iter do # Primal Optimization
L ← ||y - D(z_t)|| + λg(y) + (μ/2)g(y)²
y ← y - γ ∇_y L
end
# Dual Updates
λ ← λ + μ φ(y) # Update Lagrange Multiplier
μ ← min(αμ, μ_max) # Increase Penalty
end
return y
This dual update mechanism distinguishes the method from simple "Prop-Guidance", allowing it to solve hard, non-convex physical constraints by progressively tightening the solution space.
Summary: Guidance vs. Optimization
Standard Guidance
Is a "soft" pressure. It says, "try to look more like X." If the model resists, the guidance might fail.
Zampini's Strict Optimization
Is a "hard" wall. It says, "Do not leave this timestep until you look like X." By iterating the gradient update repeatedly before moving on, it forces the latent vector into the valid region.