Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions RELEASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@
blocks, selectable with `sampling_slices` in `sliced_wasserstein_distance`,
as described in [97] (PR #853)

#### Documentation

- Clarify how transported mass is interpreted in partial optimal transport and extend the existing 2D example with a deterministic mass sweep and matched-mass summaries (Issue #890).

#### Closed issues

- Fix ordinary JAX gradients for dense `ot.emd2` and preserve the magnitude and higher-order dependence of gradients supplied to `JaxBackend.set_gradients`, including the envelope gradients in `ot.solve` (Issue #625)
Expand Down
28 changes: 19 additions & 9 deletions docs/source/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -774,8 +774,10 @@ implemented the main function :any:`ot.barycenter_unbalanced`.
Partial optimal transport
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Partial OT is a variant of the optimal transport problem when only a fixed amount of mass m
is to be transported. The partial OT metric between two histograms a and b is defined as [28]_:
Partial OT is a variant of the optimal transport problem in which only a fixed
amount of mass :math:`m` is transported. Each source and target weight is a
capacity: some of that weight can remain unused. For nonnegative histograms
:math:`a` and :math:`b`, the partial OT problem is defined as [28]_:

.. math::
\gamma = \arg\min_\gamma <\gamma,M>_F
Expand All @@ -785,13 +787,21 @@ is to be transported. The partial OT metric between two histograms a and b is de
\gamma 1 \leq a\\
\gamma^T 1 \leq b\\
1^T \gamma^T 1 = m \leq \min\{\|a\|_1, \|b\|_1\}


Interestingly the problem can be casted into a regular OT problem by adding reservoir points
in which the surplus mass is sent [29]_. We provide a solver for partial OT
in :any:`ot.partial`. The exact resolution of the problem is computed in :any:`ot.partial.partial_wasserstein`
and :any:`ot.partial.partial_wasserstein2` that return respectively the OT matrix and the value of the
linear term. The entropic solution of the problem is computed in :any:`ot.partial.entropic_partial_wasserstein`

See the :ref:`2D partial Wasserstein example
<sphx_glr_auto_examples_unbalanced-partial_plot_partial_wass_and_gromov.py>`
for an illustration of how transported mass changes the plans and group participation.

Solvers
"""""""

The problem can be cast into a regular OT problem by adding reservoir points
to receive the surplus mass [29]_. We provide solvers in :any:`ot.partial`.
The exact problem is solved by :any:`ot.partial.partial_wasserstein` and
:any:`ot.partial.partial_wasserstein2`, which return the OT matrix and the
value of the linear term, respectively. The entropic solution is computed
by :any:`ot.partial.entropic_partial_wasserstein`
(see [3]_). Following the convention of :any:`ot.sinkhorn`, this solver takes a ``method`` parameter:
``method='sinkhorn'`` (default) runs the classical multiplicative-domain iterations, while
``method='sinkhorn_log'`` switches to a numerically stable log-domain solver
Expand Down Expand Up @@ -1224,4 +1234,4 @@ References
(pp. 2454-2465). PMLR.

.. [41] Xu, H., Luo, D., & Carin, L. (2019). `Scalable Gromov-Wasserstein learning for graph partitioning and matching
<https://arxiv.org/abs/1906.03666>`_\ , Advances in neural information processing systems, 32.
<https://arxiv.org/abs/1906.03666>`_\ , Advances in neural information processing systems, 32.
171 changes: 160 additions & 11 deletions examples/unbalanced-partial/plot_partial_wass_and_gromov.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"""

# Author: Laetitia Chapel <laetitia.chapel@irisa.fr>
# Mass-interpretation extension: Jialin Tang <jtang3898@sdsu.edu>
# License: MIT License

# sphinx_gallery_thumbnail_number = 2
Expand All @@ -38,14 +39,15 @@

n_samples = 20 # nb samples (gaussian)
n_noise = 20 # nb of samples (noise)
random_state = np.random.RandomState(42)

mu = np.array([0, 0])
cov = np.array([[1, 0], [0, 2]])

xs = ot.datasets.make_2D_samples_gauss(n_samples, mu, cov)
xs = np.append(xs, (np.random.rand(n_noise, 2) + 1) * 4).reshape((-1, 2))
xt = ot.datasets.make_2D_samples_gauss(n_samples, mu, cov)
xt = np.append(xt, (np.random.rand(n_noise, 2) + 1) * -3).reshape((-1, 2))
xs = ot.datasets.make_2D_samples_gauss(n_samples, mu, cov, random_state=random_state)
xs = np.append(xs, (random_state.rand(n_noise, 2) + 1) * 4).reshape((-1, 2))
xt = ot.datasets.make_2D_samples_gauss(n_samples, mu, cov, random_state=random_state)
xt = np.append(xt, (random_state.rand(n_noise, 2) + 1) * -3).reshape((-1, 2))

M = sp.spatial.distance.cdist(xs, xt)

Expand All @@ -72,7 +74,7 @@
print("Partial Wasserstein distance (m = 0.5): " + str(log0["partial_w_dist"]))
print("Entropic partial Wasserstein distance (m = 0.5): " + str(log["partial_w_dist"]))

pl.figure(1, (10, 5))
pl.figure(figsize=(10, 5))
pl.subplot(1, 2, 1)
pl.imshow(w0, cmap="gray_r")
pl.title("Partial Wasserstein")
Expand All @@ -82,6 +84,151 @@
pl.show()


#############################################################################
#
# Interpret the transported mass on the same 2D samples
# -----------------------------------------------------
#
# The parameter ``m`` is an absolute amount of mass, in the same units as
# the histogram weights. Both totals are one here, so ``m`` is also the
# transported fraction. With source and target totals of two and three,
# ``m=0.5`` would transport one quarter and one sixth of their masses.
# With 40 uniformly weighted samples here, each sample can supply or receive
# at most 1/40 of the mass. A sample's weight can be used partially or split
# across several pairs; ``m`` does not select a fixed number of points.
# Full mass recovers balanced OT when both totals agree. With unequal totals,
# transporting the smaller total leaves unused weight in the larger one.
#
# We vary ``m`` using the exact partial Wasserstein solver. The first 20
# samples in each distribution are Gaussian samples (called inliers below),
# and the last 20 are remote samples (called outliers). These synthetic
# labels are used only to summarize the plans; they are not solver inputs.
# A small mass can omit genuine overlap, while a large mass can require
# transporting remote samples. Partial OT does not guarantee outlier removal.

mass_study = {
"masses": np.arange(1, 9) / 8,
"plans": [],
"source_weights": p.copy(),
"target_weights": q.copy(),
"cost_matrix": M.copy(),
"inlier_count": n_samples,
"balanced_plan": ot.emd(p, q, M),
"records": [],
}
tolerance = 1e-8

for mass in mass_study["masses"]:
plan = ot.partial.partial_wasserstein(p, q, M, m=mass)
assert np.isfinite(plan).all() and (plan >= 0).all()
assert (plan.sum(axis=1) <= p + tolerance).all()
assert (plan.sum(axis=0) <= q + tolerance).all()
assert np.isclose(plan.sum(), mass, atol=tolerance, rtol=0)
mass_study["plans"].append(plan)
mass_study["records"].append(
{
"mass": float(mass),
"cost": float(np.sum(plan * M)),
"source_inlier_mass": float(plan[:n_samples, :].sum()),
"source_outlier_mass": float(plan[n_samples:, :].sum()),
"target_inlier_mass": float(plan[:, :n_samples].sum()),
"target_outlier_mass": float(plan[:, n_samples:].sum()),
# Count a pair involving either outlier group only once.
"any_outlier_mass": float(plan.sum() - plan[:n_samples, :n_samples].sum()),
}
)

# At full mass, these normalized histograms recover balanced OT. Optimal
# plans need not be unique, so compare the marginals and objective values.
full_mass_plan = mass_study["plans"][-1]
assert np.allclose(full_mass_plan.sum(axis=1), p, atol=tolerance, rtol=0)
assert np.allclose(full_mass_plan.sum(axis=0), q, atol=tolerance, rtol=0)
assert np.isclose(
np.sum(full_mass_plan * M),
np.sum(mass_study["balanced_plan"] * M),
atol=tolerance,
rtol=0,
)

print("\nExact partial Wasserstein mass study (raw transport costs):")
print(" m cost")
for record in mass_study["records"]:
print(f'{record["mass"]:5.3f} {record["cost"]:9.4f}')

#############################################################################
#
# The plan matrices show which sample groups are coupled. Dashed lines
# separate the Gaussian and remote samples on each axis. All four panels
# use the same color scale, in units of mass per source-target pair.

fig_plans, plan_axes = pl.subplots(
1, 4, figsize=(12, 3.6), sharex=True, sharey=True, constrained_layout=True
)
for ax, mass, plan in zip(
plan_axes, mass_study["masses"][1::2], mass_study["plans"][1::2]
):
image = ax.imshow(plan, cmap="Blues", vmin=0, vmax=p.max())
ax.axhline(n_samples - 0.5, color="gray", linestyle="--", linewidth=1)
ax.axvline(n_samples - 0.5, color="gray", linestyle="--", linewidth=1)
ax.set_title(f"m = {mass:g}")
ax.set_xlabel("Target sample index")
ax.set_xticks([0, n_samples, n_samples + n_noise - 1])
plan_axes[0].set_ylabel("Source sample index")
fig_plans.suptitle("Exact partial OT: indices 0-19 are Gaussian, 20-39 are remote")
fig_plans.colorbar(image, ax=plan_axes, label="Mass per pair", shrink=0.75)
pl.show()

#############################################################################
#
# Summarize the mass used from each distribution separately. Each stacked
# bar has height ``m`` and separates mass from Gaussian and remote samples.
#
# These summaries describe participation of the synthetic groups, not
# point-correspondence accuracy. Raw costs at different masses are not a
# direct comparison of matching quality, because the amount transported
# changes. The solver also does not choose ``m`` automatically.

fig_mass, mass_axes = pl.subplots(1, 2, figsize=(12, 5), sharey=True)
fig_mass.subplots_adjust(bottom=0.25, wspace=0.12)
for ax, side, title in zip(
mass_axes,
("source", "target"),
("Mass transported from source", "Mass transported to target"),
):
inlier_masses = [record[f"{side}_inlier_mass"] for record in mass_study["records"]]
outlier_masses = [
record[f"{side}_outlier_mass"] for record in mass_study["records"]
]
ax.bar(
mass_study["masses"],
inlier_masses,
width=0.08,
label="Gaussian samples (inliers)",
)
ax.bar(
mass_study["masses"],
outlier_masses,
bottom=inlier_masses,
width=0.08,
label="Remote samples (outliers)",
)
ax.set_title(title, fontsize=14)
ax.set_xlabel("Requested transported mass m", fontsize=12)
ax.set_xticks(mass_study["masses"])
ax.tick_params(labelsize=11)
ax.set_ylim(0, 1.05)
mass_axes[0].set_ylabel("Transported mass", fontsize=12)
fig_mass.legend(
*mass_axes[0].get_legend_handles_labels(),
loc="lower center",
bbox_to_anchor=(0.5, 0.01),
ncol=2,
fontsize=12,
)
mass_study["figures"] = {"plans": fig_plans, "mass_accounting": fig_mass}
pl.show()


#############################################################################
#
# Sample one 2D and 3D Gaussian distributions and plot them
Expand All @@ -104,11 +251,13 @@
cov_t = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]])


xs = ot.datasets.make_2D_samples_gauss(n_samples, mu_s, cov_s)
xs = np.concatenate((xs, ((np.random.rand(n_noise, 2) + 1) * 4)), axis=0)
xs = ot.datasets.make_2D_samples_gauss(
n_samples, mu_s, cov_s, random_state=random_state
)
xs = np.concatenate((xs, ((random_state.rand(n_noise, 2) + 1) * 4)), axis=0)
P = sp.linalg.sqrtm(cov_t)
xt = np.random.randn(n_samples, 3).dot(P) + mu_t
xt = np.concatenate((xt, ((np.random.rand(n_noise, 3) + 1) * 10)), axis=0)
xt = random_state.randn(n_samples, 3).dot(P) + mu_t
xt = np.concatenate((xt, ((random_state.rand(n_noise, 3) + 1) * 10)), axis=0)

fig = pl.figure()
ax1 = fig.add_subplot(121)
Expand Down Expand Up @@ -137,7 +286,7 @@
print("Wasserstein distance (m = 1): " + str(log0["partial_gw_dist"]))
print("Entropic Wasserstein distance (m = 1): " + str(log["partial_gw_dist"]))

pl.figure(1, (10, 5))
pl.figure(figsize=(10, 5))
pl.title("mass to be transported m = 1")
pl.axis("off")
pl.subplot(1, 2, 1)
Expand All @@ -161,7 +310,7 @@
print("Partial Wasserstein distance (m = 2/3): " + str(log0["partial_gw_dist"]))
print("Entropic partial Wasserstein distance (m = 2/3): " + str(log["partial_gw_dist"]))

pl.figure(2, (10, 5))
pl.figure(figsize=(10, 5))
pl.title("mass to be transported m = 2/3")
pl.axis("off")
pl.subplot(1, 2, 1)
Expand Down
Loading