MedPhyGraph | ECCV 2026

Abstract

Dynamic digital twins must update semantic relations and geometry when objects change. We present MedPhyGraph, a framework for maintaining SupportedBy graphs across adjacent digital-twin states. CF-SupportNet scores candidate edges using static geometry and counterfactual evidence from analytic host-removal rollouts. Deterministic state and transition consistency enforce graph validity and identify supporter changes. MedPhyGraph operates downstream of perception on structured object states and requires neither ground-truth destinations nor transfer metadata at inference. We evaluate MedPhyGraph in controlled healthcare digital-twin environments realized and visualized using NVIDIA Isaac for Healthcare assets; these assets are not model inputs or label sources. Under the controlled core protocol, MedPhyGraph achieves Transition-Macro, Pooled Delta Micro, and Transfer Dyn-F1 of 1.000. Across 217 transfers and 19 semantic templates, it recovers all required transfer edges and reaches pooled Transfer Dyn-F1 of 0.998.

Pipeline

Candidate edges are scored by CF-SupportNet from static geometry and analytic AABB host-removal evidence. State Consistency and Union-Based Transition-Aware Consistency update the support graph across adjacent states.

Candidate generation

Geometric and structural cues define a plausible subject–host search space without GT edges.

CF-SupportNet

GRU + geometry fusion scores each candidate from analytic host-removal trajectories.

Transition consistency

Dual-state scoring, previous-host decline, Direct-Support Gate, and graph refinement.

Demo

Isaac Sim clips visualize a tray support transfer in a healthcare digital-twin room. Renders are visual context only — not model inputs or training labels.

Isaac Sim — wide room view

Hero camera over the operating / procedure room layout while the tray moves from the cabinet area toward the side table beside the monitor cart.

Isaac Sim — transfer close-up

Demo camera focused on the support transfer: the tray leaves the previous host surface and settles on the destination support used by MedPhyGraph evaluation.

Support-graph update

Corresponding graph edit after transition-aware inference: remove the previous SupportedBy edge and add the new host relation for the transferred subject.

Results

Support-graph recovery and adjacent-state maintenance on Procedural Scenes and Isaac for Healthcare Scenes, with Isaac as the primary setting unless noted otherwise. Primary results use the frozen inference policy with observation ratio ρ=1.0 and seed 0.

1.000Transition-Macro Dyn-F1 (Isaac / Procedural core)

1.000Transfer Dyn-F1 (core protocol)

0.997±0.001Expanded Transfer Dyn-F1 (seeds 0–4)

Results on the expanded 217-case transfer suite

Required success indicates recovery of both required transfer edges; additional errors are reflected in the pooled metrics. Best and second-best results are highlighted.

Method	Req. Success	Transfer	Add	Remove
Geometry Rule	0.862	0.931	1.000	0.862
Logistic Regression	0.654	0.791	0.791	0.791
Random Forest	0.000	0.000	0.000	0.000
MLP	0.000	0.000	0.000	0.000
CF-SupportNet + SC	0.000	0.000	0.000	0.000
MedPhyGraph	1.000	0.998	1.000	0.995

Complete dynamic results on Isaac for Healthcare Scenes

Under the full transition operator and verified operation-consistent targets.

Method	Trans-Macro	Pooled Micro	Add	Remove	Transfer
Geometry Rule	0.583	0.600	1.000	0.333	0.667
Logistic Regression	0.833	0.846	1.000	1.000	0.333
Random Forest	0.750	0.750	1.000	1.000	0.000
MLP	0.601	0.545	0.606	0.900	0.000
MedPhyGraph	1.000	1.000	1.000	1.000	1.000

Component analysis on Isaac for Healthcare Scenes under the core protocol

Viol.: mean structural violations; T-Mac.: Transition-Macro Dyn-F1. Best transfer results and the strongest partial configuration are highlighted.

Configuration	Viol.	T-Mac.	Transfer
Independent CF-SupportNet	1.361	0.729	0.556
+ State Consistency	0.000	0.750	0.000
Union rescoring only	0.000	0.750	0.000
Direct-Support Gate only	0.000	0.712	0.778
Union and Gate	0.000	0.750	0.000
MedPhyGraph	0.000	1.000	1.000

Figures

Figure 3. Method-by-operation dynamic performance on Procedural and Isaac for Healthcare Scenes under the core 60-transition protocol (15 eligible support transfers). Methods differ primarily on support transfer rather than ordinary addition or removal. Complete operation-level results are reported in the Supplementary Material. PDF

Figure S1. Isaac refinement analysis under the frozen core protocol. State Consistency removes structural violations, while complete transition-aware selection is required for reliable supporter replacement. Exact values are reported in the Supplementary Material. PDF

Paired bootstrap differences with 95 percent confidence intervals — Figure S2. Paired bootstrap differences (MedPhyGraph minus comparator) with 95% confidence intervals from 10,000 seed-0 replicates using base-scene resampling. Columns show Procedural and Isaac for Healthcare Scenes; rows show Transition-Macro, Pooled Delta Micro, and Transfer Dyn-F1. PDF

Supplementary Material

Complete static and dynamic baselines, component matrices, multi-seed stability analysis, evidence-channel diagnostics, and paired bootstrap comparisons.

Core and expanded evaluation inventories

Quantity	Core	Expanded
Base layouts / scene states	30 / 90	—
Dynamic transitions	60	—
Eligible transfers	15	217
Procedural / Isaac transfers	9 / 6	136 / 81
Semantic templates	2	19
Ambiguous / invalid transfers	0 / 0	0 / 0
Destination metadata at inference	none	none

Candidate-availability stress test on the expanded 217-case suite using MedPhyGraph

Condition	Required success	Transfer Dyn-F1
Native candidates	217/217	0.998
Destination edge removed	0/217	0.000

Complete static results on Procedural Scenes after State Consistency

Method	Pooled GE-F1	Scene-Macro	Precision
Geometry Rule	0.805	0.828	0.884
Logistic Regression	0.974	0.979	1.000
Random Forest	0.974	0.979	1.000
MLP	0.974	0.979	1.000
CF-SupportNet + SC	0.974	0.979	1.000

Complete static results on Isaac for Healthcare Scenes after State Consistency

Method	Pooled GE-F1	Scene-Macro	Precision
Geometry Rule	0.788	0.795	0.883
Logistic Regression	0.932	0.938	1.000
Random Forest	0.915	0.923	1.000
MLP	0.883	0.892	0.965
CF-SupportNet + SC	0.915	0.923	1.000

MedPhyGraph full-population static GE-F1: Procedural 0.957 (n=54), Isaac 0.899 (n=36).

Complete dynamic results on Procedural Scenes

Method	T-Mac.	Pooled	Add	Remove	Transfer
Geometry Rule	0.528	0.600	1.000	0.111	0.889
Logistic Regression	0.944	0.953	1.000	1.000	0.778
Random Forest	0.750	0.750	1.000	1.000	0.000
MLP	0.750	0.750	1.000	1.000	0.000
MedPhyGraph	1.000	1.000	1.000	1.000	1.000

Complete dynamic results on Isaac for Healthcare Scenes

Method	T-Mac.	Pooled	Add	Remove	Transfer
Geometry Rule	0.583	0.600	1.000	0.333	0.667
Logistic Regression	0.833	0.846	1.000	1.000	0.333
Random Forest	0.750	0.750	1.000	1.000	0.000
MLP	0.601	0.545	0.606	0.900	0.000
MedPhyGraph	1.000	1.000	1.000	1.000	1.000

Complete inference-component analysis on Isaac for Healthcare Scenes under the core protocol

✓ enabled · ✗ disabled · SC: State Consistency; Un.: union dual-state scoring; Dec.: previous-host decline; Sw.: gain/switch selection; Ga.: Direct-Support Gate; Ref.: post-selection refinement

Configuration	SC	Un.	Dec.	Sw.	Ga.	Ref.	Viol.	T-Mac.	P-Mic.	Transfer
Independent CF-SupportNet	✗	✗	✗	✗	✗	✗	1.361	0.729	0.686	0.556
+ State Consistency	✓	✗	✗	✗	✗	✗	0.000	0.750	0.750	0.000
Union rescoring only	✗	✓	✗	✗	✗	✓	0.000	0.750	0.750	0.000
Direct-Support Gate only	✗	✗	✗	✗	✓	✓	0.000	0.712	0.625	0.778
Union and Gate	✗	✓	✗	✗	✓	✓	0.000	0.750	0.750	0.000
MedPhyGraph	✓	✓	✓	✓	✓	✓	0.000	1.000	1.000	1.000

Inference-component analysis on the expanded 217-case suite

Configuration	SC	Un.	Dec.	Sw.	Ga.	Ref.	Viol.	T-Mac.	P-Mic.
Independent CF	✗	✗	✗	✗	✗	✗	2.281	0.642	0.635
+ State Consistency	✓	✗	✗	✗	✗	✗	0.000	0.000	0.000
Union only	✗	✓	✗	✗	✗	✓	0.000	0.000	0.000
Gate only	✗	✗	✗	✗	✓	✓	0.000	0.756	0.682
Union + Gate	✗	✓	✗	✗	✓	✓	0.000	0.000	0.000
MedPhyGraph	✓	✓	✓	✓	✓	✓	0.000	0.998	0.998

Inference-component analysis on Procedural Scenes

Configuration	SC	Un.	Dec.	Sw.	Ga.	Ref.	Viol.	T-Mac.	P-Mic.	Transfer
Independent CF	✗	✗	✗	✗	✗	✗	1.463	0.759	0.727	0.630
+ State Consistency	✓	✗	✗	✗	✗	✗	0.000	0.750	0.750	0.000
Union only	✗	✓	✗	✗	✗	✓	0.000	0.750	0.750	0.000
Gate only	✗	✗	✗	✗	✓	✓	0.000	0.727	0.643	0.926
Union + Gate	✗	✓	✗	✗	✓	✓	0.000	0.750	0.750	0.000
MedPhyGraph	✓	✓	✓	✓	✓	✓	0.000	1.000	1.000	1.000

Multi-seed audit on Isaac under the core 15-transfer protocol

Transfer Dyn-F1 unless noted.

Seed	Indep.	SC	MedPhyGraph	T-Mac.
0	0.545	0.000	1.000	1.000
1	0.545	0.000	1.000	1.000
2	0.444	0.000	1.000	1.000
3	0.444	0.000	1.000	1.000
4	0.462	0.000	1.000	1.000
mean	0.488	0.000	1.000	1.000
std	±0.053	±0.000	±0.000	±0.000

Incomplete temporal evidence on the frozen test split

ρ	Edge F1
1.00	0.957
0.75	0.957
0.50	0.945
0.25	0.957
0.10	0.909

Inference-time channel masking at ρ=1.0

Edge F1.

Variant	Edge F1
Full features	0.957
cf_only	0.909
geometry_only	0.778
no_cf_motion	0.909

Paired bootstrap differences (MedPhyGraph minus comparator)

95% confidence intervals from 10,000 seed-0 replicates using base-scene resampling.

Corpus	Comparator	Metric	Δ mean	CI low	CI high
Isaac	LR	T-Mac.	0.167	0.063	0.250
Isaac	LR	P-Mic.	0.154	0.056	0.267
Isaac	LR	Transfer	0.667	0.333	1.000
Isaac	RF	T-Mac.	0.250	0.143	0.357
Isaac	RF	P-Mic.	0.250	0.143	0.357
Isaac	RF	Transfer	1.000	1.000	1.000
Isaac	MLP	T-Mac.	0.399	0.312	0.494
Isaac	MLP	P-Mic.	0.455	0.357	0.526
Isaac	MLP	Transfer	1.000	1.000	1.000
Isaac	Indep. CF	T-Mac.	0.271	0.214	0.333
Isaac	Indep. CF	P-Mic.	0.314	0.256	0.364
Isaac	Indep. CF	Transfer	0.444	0.389	0.500
Isaac	CF+SC	T-Mac.	0.250	0.143	0.357
Isaac	CF+SC	P-Mic.	0.250	0.143	0.357
Isaac	CF+SC	Transfer	1.000	1.000	1.000
Procedural	LR	T-Mac.	0.056	0.000	0.100
Procedural	LR	P-Mic.	0.047	0.000	0.087
Procedural	LR	Transfer	0.222	0.000	0.400
Procedural	RF	T-Mac.	0.250	0.167	0.333
Procedural	RF	P-Mic.	0.250	0.167	0.333
Procedural	RF	Transfer	1.000	1.000	1.000
Procedural	MLP	T-Mac.	0.250	0.167	0.333
Procedural	MLP	P-Mic.	0.250	0.167	0.346
Procedural	MLP	Transfer	1.000	1.000	1.000
Procedural	Indep. CF	T-Mac.	0.241	0.197	0.288
Procedural	Indep. CF	P-Mic.	0.273	0.231	0.310
Procedural	Indep. CF	Transfer	0.370	0.333	0.400
Procedural	CF+SC	T-Mac.	0.250	0.154	0.346
Procedural	CF+SC	P-Mic.	0.250	0.167	0.333
Procedural	CF+SC	Transfer	1.000	1.000	1.000

On Isaac, CIs for Transition-Macro, Pooled Micro, and Transfer exclude zero against all five comparators. Against Logistic Regression on Procedural, those intervals include zero.

Citation

If you find MedPhyGraph useful, please cite our work.

@InProceedings{gholizadeh2026medphygraph,
  author    = {Gholizadeh HamlAbadi, Kamran and Vahdati, Monica and El Saddik, Abdulmotaleb},
  title     = {MedPhyGraph: Counterfactual Support-Graph Maintenance for Dynamic Built-Environment Digital Twins},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops (TwinWorld: Visual Intelligence for Built Environment Digital Twins)},
  year      = {2026},
  note      = {To appear}
}