Mask2Former, Dissected

In short. By late 2021, universal segmentation architectures existed and kept losing. MaskFormer handled panoptic, instance, and semantic segmentation with one design, and trailed the best instance specialist by over 9 AP while needing 300 training epochs and a 32 GB GPU per image. Mask2Former [Cheng et al. 2022] closed the gap without abandoning the paradigm, by rebuilding the decoder and the training recipe. Each query predicts a class and a mask. Masked cross-attention restricts the query’s next image read to its current mask, the decoder cycles through three feature scales, and mask losses are evaluated at sampled points. The strongest Swin-L variants reach 57.8 PQ and 50.1 AP on COCO and 57.7 mIoU on ADE20K. With ResNet-50, the COCO schedule falls to 50 epochs and reported training memory to 6 GB.

Contents. 0. Background · 1. The problem · 2. Two paradigms · 3. Set prediction · 4. Architecture · 5. Masked attention · 6. Multi-scale features · 7. Decoder rewiring · 8. Point-sampled losses · 9. Training · 10. Results · 11. Ablations · 12. Limitations · 13. Later work · 14. Conclusion · Appendix A · Appendix B · References · Citation

If you already know DETR and segmentation metrics, start at §4. For a shorter path, read sections 4 through 8 and section 11. Proofs and longer derivations are folded, and the main argument does not depend on them.

0. The background you need

Start here if any of this is new. Skip to §1 if words like attention, embedding, and loss already feel comfortable. The notation table below is a reference to return to, not something to memorize.

Images, pixels, and the job. An image is a grid of pixels. Segmentation assigns labels to those pixels and decides which ones belong to the same region. Section 1 separates the three tasks built from that idea.

Feature maps. A backbone converts the image into a smaller grid where each cell holds a vector that summarizes part of the image. That grid is a feature map. The stride is the shrink factor. At stride 32, a 1024 by 1024 image becomes a 32 by 32 grid. Small objects can lose useful detail at this scale. §6 explains why Mask2Former also uses finer features.

Convolutions. Most vision backbones use convolutions. A convolutional filter is a small grid of learned weights. At each image position, it multiplies those weights by the pixels beneath them and sums the products. Repeating this operation across the image produces a feature map that responds wherever the learned pattern appears. With a $3\times3$ filter $w$ and input $x$, the output at location $(i,j)$ is

$$y(i,j) = \sum_{a=-1}^{1}\sum_{b=-1}^{1} w_{ab}\, x_{i+a,\,j+b}.$$

Here $(a,b)$ indexes a position inside the filter. A layer applies many filters, adds learned biases, and passes the results through a nonlinearity such as ReLU. Stacking these layers produces a convolutional network.

Embeddings and the dot product. An embedding is a learned vector whose geometry carries information. Related inputs often point in similar directions. The dot product compares two embeddings. For nonzero vectors $u$ and $v$ with angle $\theta$ between them,

$$u^\top v = \sum_k u_k v_k = \|u\|\,\|v\|\cos\theta,$$

so it is positive when the vectors point in similar directions, zero when they are orthogonal, and negative when they point apart. Mask2Former forms each mask by taking a dot product between a query-derived vector and every pixel embedding, then applying a sigmoid.

Softmax. The scores before softmax are called logits. For logits $z_1, \dots, z_n$, softmax assigns each option a weight

$$w_i = \frac{e^{z_i}}{\sum_j e^{z_j}},$$

which are positive and sum to 1. Softmax can put almost all its weight on a small region, but it does not forbid other locations. Mask2Former adds that restriction explicitly. Sending a forbidden entry’s score to $-\infty$ removes it, and the remaining weights still sum to 1.

Attention. Attention is a differentiable lookup. A query is compared with a key at every image location. Softmax turns those comparisons into weights, and the output is the weighted average of the corresponding values. With $N$ queries and $n$ image locations, stack the queries in $Q \in \mathbb{R}^{N\times d}$ and the image keys and values in $K, V \in \mathbb{R}^{n\times d}$.

$$\operatorname{Attention}(Q, K, V) = \operatorname{softmax}\!\left(\frac{QK^\top}{\sqrt d}\right)V.$$

$QK^\top$ contains every query-key score. Softmax turns each row into weights over locations, and multiplication by $V$ returns a weighted average of their values. Dividing by $\sqrt d$ keeps the logits from growing with the key dimension and pushing softmax toward saturation. Cross-attention builds $Q$ from the queries and $K,V$ from the image. Self-attention builds all three from the same token set. A token is one vector in that set, a query or the vector at one image location. In practice the model runs several of these lookups in parallel, each on its own slice of the $d$ feature dimensions. Each slice is called a head, and the head outputs are concatenated and projected back to width $d$.

Transformers and decoders. A Transformer layer combines attention with a per-token network and residual connections. Each token carries a vector $x$. Omitting normalization, the layer applies two updates in order.

$$x \,\leftarrow\, x + \operatorname{Attention}(x,\ \text{context}), \qquad x \,\leftarrow\, x + \operatorname{FFN}(x).$$

The attention step mixes information from the image or from other queries. The feed-forward network then processes each token independently. Residual connections add both updates to the existing token state. Mask2Former stacks nine decoder layers, and each query produces one candidate segment.

Losses, gradients, training. A loss $\mathcal{L}$ is a single number measuring how wrong the current output is. Training repeats one step. Compute the gradient $\nabla_w \mathcal{L}$, the direction in weight space that increases the loss fastest, then move every weight a small step the other way, $w \leftarrow w - \eta\, \nabla_w \mathcal{L}$, where $\eta$ is the learning rate. Sections 3 and 8 define Mask2Former’s loss and explain how its predictions receive training signals.

IoU. Intersection over union compares two regions $A$ and $B$ with nonempty union.

$$\text{IoU}(A, B) = \frac{|A \cap B|}{|A \cup B|}.$$

IoU is 1 when the regions are identical and 0 when they are disjoint. It is used throughout §1 to compare predicted and ground-truth regions.

Sets versus lists. A list has an order. A set does not. Mask2Former’s predictions have no meaningful order, so the loss must keep the same value after any reordering. Section 3 shows how matching provides this invariance.

Notation

symbol	meaning
$I \in \mathbb{R}^{3\times H\times W}$	input image. $H_l \times W_l$ is the size of the feature map used at decoder layer $l$
$\Omega$	the pixel grid, the set of pixel locations of the map under discussion. $\lvert\Omega\rvert$ is its size
$C$	shared feature dimension of queries and image features (256 in the reference implementation)
$N$	number of object queries (100 by default, 200 for the largest models)
$K$, $\varnothing$	number of classes, and the “no-object” class appended to them. This $K$ is unrelated to §0’s key matrix
$(m_i, c_i)$	prediction $i$, with a soft mask $m_i \in [0,1]^{H\times W}$ and class $c_i \in \{1,\dots,K\}\cup\{\varnothing\}$
$\hat p_i(c)$	predicted probability that query $i$ has class $c$
$\mathbf{X}_l \in \mathbb{R}^{N\times C}$	query features after decoder layer $l$. $\mathbf{X}_0$ are the learnable input query features
$\mathbf{Q}_l, \mathbf{K}_l, \mathbf{V}_l$	attention projections. $\mathbf{K}_l,\mathbf{V}_l \in \mathbb{R}^{H_lW_l\times C}$ come from image features
$M_{l}$	mask predictions of layer $l$, resized as logits to the next layer’s attention resolution, then passed through the sigmoid and binarized at $0.5$
$\mathcal{M}_{l}$	the additive attention mask built from $M_{l}$. It is $0$ on foreground and $-\infty$ elsewhere
$\mathcal{E}_{\text{pixel}}$	per-pixel embeddings from the pixel decoder, at stride 4
$\sigma(\cdot)$	the logistic sigmoid $\sigma(z) = 1/(1+e^{-z})$, which squashes a real score into $[0,1]$. This $\sigma$ is unrelated to §3’s matching permutation
$\mathbb{1}[\cdot]$	the indicator, with $\mathbb{1}[P] = 1$ when condition $P$ holds and $0$ otherwise
$\lambda_{\text{ce}}, \lambda_{\text{dice}}, \lambda_{\text{cls}}$	BCE, Dice, and class-loss weights. Their values are $5$, $5$, and $2$. The no-object class receives an additional $0.1$ multiplier
$K_{\text{pt}}$	number of sampled points for mask losses, $12{,}544 = 112^2$

1. The segmentation problem

Image segmentation groups pixels and labels the groups. Datasets divide categories into things and stuff [Kirillov et al. 2019]. Things are countable objects such as people and cars. Stuff describes regions such as road and sky. Write these category sets as $\mathcal{C}_{\text{th}}$ and $\mathcal{C}_{\text{st}}$, with $\mathcal{C} = \mathcal{C}_{\text{th}} \sqcup \mathcal{C}_{\text{st}}$. The symbol $\sqcup$ means the two sets do not overlap. The three segmentation tasks differ in which groups and identities they retain.

Semantic segmentation is a map $f: \Omega \to \mathcal{C}$ on the pixel grid $\Omega$. One label per pixel means that two adjacent cars share one car region. Accumulate the predicted and ground-truth pixels over the evaluation set, then let $\mathcal{C}_{\mathrm{eval}} = \{c : |P_c \cup G_c| > 0\}$. Its metric is

$$\text{mIoU} = \frac{1}{|\mathcal{C}_{\mathrm{eval}}|}\sum_{c\in\mathcal{C}_{\mathrm{eval}}} \frac{|P_c \cap G_c|}{|P_c \cup G_c|},$$

where $P_c$ and $G_c$ are the predicted and ground-truth pixel sets of class $c$ [Everingham et al. 2015]. Restricting the average to classes with nonzero union removes the $0/0$ case.

Instance segmentation returns one scored mask per detected thing. Mask AP ranks predictions by confidence and measures precision and recall across IoU thresholds from $0.50$ to $0.95$ [Lin et al. 2014].

The core of COCO mask AP

For each class, predictions are ranked by score. A prediction is a true positive when its best unclaimed ground-truth match clears the current IoU threshold.

$$\begin{gathered} \text{IoU}(\bar m_i, g) = \frac{\sum_{x\in\Omega} \bar m_i(x)\, g(x)}{\sum_{x\in\Omega} \big(\bar m_i(x) + g(x) - \bar m_i(x)\, g(x)\big)},\\[4pt] i \in \mathit{TP} \iff \max_{g \in \mathcal{G}_i} \text{IoU}(\bar m_i, g) \ge \tau, \end{gathered}$$

where the sums visit every pixel $x$ of the grid $\Omega$, and $\bar m_i(x) = \mathbb{1}[m_i(x) > 0.5]$ is the binarized mask, $1$ on the pixels the prediction claims and $0$ elsewhere, with $g(x)$ the same for the ground truth. On $0/1$ values a product acts as an AND, so each numerator term is $1$ exactly when both masks contain the pixel, and the sum counts the intersection. The combination $\bar m_i(x) + g(x) - \bar m_i(x)\, g(x)$ acts as an OR, $1$ when either mask contains the pixel, so the denominator counts the union, and the fraction is §0’s intersection over union computed by sums. Finally $\mathcal{G}_i$ holds the ground truths of the class still unclaimed when prediction $i$ takes its turn, all of them for the top-scoring prediction. If $\mathcal{G}_i$ is empty the max is $-\infty$, an automatic false positive. A true positive claims its match, so a duplicate detection of an already-claimed object cannot score twice. Everything else is a false positive. Now fix a score cutoff and keep only the predictions above it. Precision is the fraction of kept predictions that are true positives, and recall is the fraction of ground truths recovered. Sweeping the cutoff from strict to loose traces the precision-recall curve, and AP is the area under that curve, averaged over ten IoU thresholds in the COCO style [Lin et al. 2014]:

$$P = \frac{|\mathit{TP}|}{|\mathit{TP}| + |\mathit{FP}|}, \qquad R = \frac{|\mathit{TP}|}{|G|},$$ $$\text{AP} = \frac{1}{|\mathcal{T}|}\sum_{\tau\in\mathcal{T}} \frac{1}{101}\sum_{r\in\{0,0.01,\ldots,1\}} p_\tau^{\mathrm{interp}}(r).$$

Here $|G|$ is the number of ground-truth masks, $\mathcal{T} = \{0.50, 0.55, \dots, 0.95\}$, and $p_\tau^{\mathrm{interp}}(r) = \max_{r' \ge r} p_\tau(r')$ is the interpolated precision, the best precision the curve reaches at recall $r$ or beyond. This is the per-category AP. COCO then averages over categories. When a cutoff keeps no detections, COCO records zero precision. A category with no ground-truth instances in the evaluation set is excluded from the mean. A duplicate detection cannot claim an object twice, so confidence ranking matters. This sketch omits crowd and ignore bookkeeping and the per-image detection cap.

Panoptic segmentation [Kirillov et al. 2019] unifies both. Every evaluated pixel receives a class and an instance id, with identities on things and plain categories on stuff. Evaluation first sets aside void pixels, which carry no label, and crowd regions, which group objects the annotators could not separate. Write $\mathit{TP}$ for matched pairs, $\mathit{FP}$ for unmatched predictions, and $\mathit{FN}$ for unmatched ground-truth segments. Quality is

$$\text{PQ} = \frac{\sum_{(p,g)\in \mathit{TP}}\text{IoU}(p,g)}{|\mathit{TP}| + \tfrac12|\mathit{FP}| + \tfrac12|\mathit{FN}|} = \underbrace{\frac{\sum_{(p,g)\in \mathit{TP}}\text{IoU}(p,g)}{|\mathit{TP}|}}_{\text{SQ}} \times \underbrace{\frac{|\mathit{TP}|}{|\mathit{TP}| + \tfrac12|\mathit{FP}| + \tfrac12|\mathit{FN}|}}_{\text{RQ}}.$$

A match requires $\text{IoU} > 0.5$. PQ is computed per class, then averaged over classes with at least one true-positive, false-positive, or false-negative segment. SQ is the average IoU of the matched pairs. RQ is an F1-like term that penalizes missed and extra segments. Their product follows by multiplying and dividing by $|\mathit{TP}|$ when at least one match exists. If $|\mathit{TP}| = 0$ but the denominator is nonzero, PQ is $0$ and the factorization is not used. The next lemma explains why matches above the threshold are unique.

Lemma (matches are unique). If predicted segments and ground-truth segments are each pairwise disjoint, as the panoptic format requires, then $\text{IoU} > 0.5$ pairs one-to-one. Each segment can match at most one segment from the other set.

Proof: a majority overlap can only happen once

Proof. Suppose a prediction $p$ has $\text{IoU}(p, g) > \tfrac12$. Then

$$\begin{aligned} |p \cap g| &> \tfrac12\,|p \cup g| && \text{definition of IoU, rearranged}\\[2pt] &\ge \tfrac12\,|g| && \text{since } g \subseteq p \cup g. \end{aligned}$$

So every matching prediction covers more than half of $g$. Now suppose two predictions $p_1, p_2$ both matched $g$. The panoptic format keeps predictions disjoint, so $p_1 \cap g$ and $p_2 \cap g$ are disjoint subsets of $g$, each larger than $\tfrac12|g|$. Together they would hold more than $|g|$ pixels inside $g$, which is impossible. So at most one prediction matches $g$. The same argument runs the other way. Since the panoptic format also keeps the ground-truth segments disjoint, and $p \subseteq p \cup g$ gives $|p \cap g| > \tfrac12|p|$, each prediction matches at most one ground truth. $\blacksquare$

Above the $0.5$ threshold, matching is therefore unambiguous, and greedy matching, pairing segments one at a time with the best still-unclaimed partner, is exact. Contrast this with training-time matching (§3), where predictions overlap freely, costs are soft, and a genuine assignment problem appears.

The output semantics differ, but the underlying problem is still pixel grouping. Mask2Former asks whether one architecture can serve all three tasks.

2. Two paradigms

2.1 Per-pixel classification (2015 onward)

The fully convolutional network (FCN) of [Long et al. 2015] recast semantic segmentation as dense classification. A $1{\times}1$ classifier scores every pixel feature for every class, producing $\hat y \in \mathbb{R}^{|\mathcal{C}|\times H\times W}$. The per-pixel cross-entropy loss is

$$\mathcal{L} = -\frac{1}{|\Omega|}\sum_{x\in\Omega}\log \hat p_x(g_x),$$

where $\hat p_x$ is the softmax of the class scores at pixel $x$ and $g_x$ is that pixel’s true class. Cross-entropy is the negative log probability assigned to the correct class. Later models added context through dilated convolutions, pyramid pooling, or attention [Chen et al. 2018, Zhao et al. 2017, Wang et al. 2018, Fu et al. 2019]. Segmenter and SegFormer later used Transformer-based per-pixel classifiers [Strudel et al. 2021, Xie et al. 2021].

A plain per-pixel classifier outputs one category at each pixel. It has no variable for object identity, so it cannot separate two cars that share a class without adding another mechanism. Earlier instance systems added proposals, anchors, grouping rules, or non-maximum suppression. Query-based set prediction provides identities directly and makes the loss invariant to their storage order.

2.2 Mask classification (2017 onward)

Mask R-CNN [He et al. 2017] predicts a binary mask inside each detected box. DETR [Carion et al. 2020] later replaced hand-built proposals with learned queries and one-to-one matching, and its panoptic extension decoded masks from those queries.

MaX-DeepLab [Wang et al. 2021] brought end-to-end set prediction to panoptic segmentation. MaskFormer [Cheng et al. 2021] showed that the same prediction format could outperform per-pixel models on semantic segmentation. K-Net [Zhang et al. 2021] pursued a related design with one dynamic convolution kernel per segment.

The basic idea. Each query predicts a mask and a class. The mask head answers where the segment is. The class head answers what it is. Both heads read the same query, so the predictions are related even though the outputs are separate.

Formally, mask classification predicts

$$\{(m_i, c_i)\}_{i=1}^{N}, \qquad m_i \in [0,1]^{H\times W},\quad c_i \in \{1,\dots,K\}\cup\{\varnothing\}.$$

The same prediction format supports all three tasks. Semantic targets contain one mask per class present in the image. Instance targets contain one mask per object. Panoptic targets contain one mask per thing or stuff segment. The architecture and loss keep the same form, while targets, checkpoints, and post-processing remain task-specific.

Example. For the puppy below, one query predicts class $c_i = \texttt{dog}$ and a mask $m_i$. A small binary version of that mask looks like this.

$$c_i = \texttt{dog}, \qquad m_i \;\longrightarrow\; \begin{pmatrix} 0 & 0 & 1 & 1 & 0\\ 0 & 1 & 1 & 1 & 1\\ 0 & 1 & 1 & 1 & 0\\ 0 & 0 & 1 & 0 & 0 \end{pmatrix}.$$

The matrix says where and the label says what. The model emits mask probabilities rather than hard zeros and ones.

Universal architectures still trailed specialized systems in 2021. MaskFormer reached 40.1 instance AP with a Swin-L backbone, compared with 49.5 for Swin-HTC++ [Chen et al. 2019, Liu et al. 2021]. MaskFormer also trained for 300 epochs and its dense mask losses limited training to one image per 32 GB GPU. Mask2Former’s thesis was that the paradigm was right and the decoder and the recipe were wrong.

3. Set prediction and matching

This matching structure comes from DETR and MaskFormer. Mask2Former changes where the mask costs are evaluated, as §8 explains.

3.1 Matching as an assignment problem

The model returns a fixed set of $N$ predictions, 100 by default. Before computing the loss, it finds the lowest-cost one-to-one pairing between predictions and ground truth. The Hungarian algorithm computes this matching. The loss then scores each matched pair, while unmatched predictions are trained as no-object.

Assume an image contains at most $N$ real targets. Pad the ground truth with $\varnothing$ entries up to size $N$, so predictions and targets are both $N$-element sets. Write $\text{cost}(i, j)$ for the cost of pairing prediction $i$ with target $j$, where target $j$ carries class $c^{\text{gt}}_j$ and mask $m^{\text{gt}}_j$. The padded entries have $c^{\text{gt}}_j = \varnothing$.

$$\text{cost}(i,j) = \mathbb{1}\!\left[c^{\text{gt}}_j \ne \varnothing\right]\Big(\!-\lambda_{\text{cls}}\,\hat p_{i}(c^{\text{gt}}_j) + \lambda_{\text{ce}}\,\mathcal{L}_{\text{ce}}\big(m_{i}, m^{\text{gt}}_j\big) + \lambda_{\text{dice}}\,\mathcal{L}_{\text{dice}}\big(m_{i}, m^{\text{gt}}_j\big)\Big).$$

The minus sign favors predictions that already assign high probability to the target class. The matcher uses class probability rather than log probability. Its mask terms use the same binary cross-entropy (BCE) and Dice forms used for training. The released code solves a rectangular predictions-by-targets problem over real targets. Padding with zero-cost no-object targets gives the equivalent square matching problem used in the proof below. The classification loss later trains every unmatched prediction as no-object.

In the square view, an assignment is a permutation $\sigma \in S_N$, where $S_N$ is the set of all $N!$ reorderings. Restricted to the real targets, it pairs every target with one unique prediction. Padded targets have zero cost, so their order is irrelevant. The total cost is

$$J(\sigma) = \sum_{j=1}^{N} \text{cost}(\sigma(j),\, j),$$

and the matcher chooses $\hat\sigma = \arg\min_{\sigma} J(\sigma)$. The classic Hungarian algorithm solves this assignment problem [Kuhn 1955, Munkres 1957]. A later bookkeeping refinement brings the runtime to $O(N^3)$. The released code calls scipy.optimize.linear_sum_assignment, which uses a modified Jonker-Volgenant algorithm. The code builds the cost matrix on sampled mask points, as §8 describes.

The implementation calls scipy, but the underlying algorithm is useful to understand. A companion maximization problem, the dual, supplies the correctness argument and explains the $O(N^3)$ runtime. The complete derivation is folded below.

The linear program, the duality proof, and a worked 3×3 run

As a linear program. Encode an assignment as $x \in \{0,1\}^{N\times N}$ with $x_{ij} = 1$ when prediction $i$ takes target $j$. Relaxing the integrality to $x \ge 0$ gives a linear program, a minimization of a linear objective over real variables subject only to linear equalities and inequalities,

$$\min_{x \ge 0}\ \sum_{i,j}\text{cost}(i,j)\,x_{ij} \quad\text{s.t.}\quad \textstyle\sum_j x_{ij} = 1\ (\forall i),\quad \sum_i x_{ij} = 1\ (\forall j).$$

The relaxation lets a prediction split fractionally across targets, which no real assignment can do. Total unimodularity guarantees that an optimal vertex is still a whole-number permutation. The next argument proves that property from the constraint matrix.

The matrix is the incidence matrix of a bipartite graph. Its rows are the $N$ prediction constraints and the $N$ target constraints, and each variable $x_{ij}$ appears in exactly two of them, prediction $i$‘s row and target $j$‘s row. So the rows carry a 2-coloring, predictions against targets, with every column showing a single 1 in each color. That structure makes the matrix totally unimodular, every square submatrix having determinant $-1$, $0$, or $+1$.

Lemma (total unimodularity). Every square submatrix $B$ of the constraint matrix has $\det B \in \{-1, 0, +1\}$.

Proof. Induct on the order $k$ of $B$.

Base case ($k = 1$). $B$ is a single entry, $0$ or $1$, so $\det B \in \{0, 1\}$.

Inductive step ($k > 1$). Suppose the claim holds at every order below $k$. If some column of $B$ holds fewer than two 1s, take such a column. Otherwise every column holds exactly two, the most any column can carry. That gives three cases.

• Zero 1s. The column is all zeros, so $\det B = 0$.
• One 1, at row $r$, column $c$ of $B$. Laplace-expand along that column. Only the single nonzero entry contributes, so $\det B = (-1)^{r+c}\det B'$, where $B'$ deletes row $r$ and column $c$. Now $B'$ has order $k-1$, so $\det B' \in \{-1,0,1\}$ by the hypothesis, hence $\det B = \pm\det B' \in \{-1,0,1\}$.
• Two 1s in every column. Every constraint-matrix column carries its two 1s in one prediction-row and one target-row, so each column of $B$ has one 1 among the prediction-rows $P$ and one among the target-rows $T$. Summing the rows in $P$ picks up exactly one 1 per column, giving the all-ones vector $\mathbf 1$, and summing the rows in $T$ gives $\mathbf 1$ the same way. So $\sum_{r\in P} B_r - \sum_{r\in T} B_r = \mathbf 0$, a nontrivial dependence among the rows, and $B$ is singular with $\det B = 0$.

Every case lands in $\{-1,0,1\}$. $\blacksquare$

Total unimodularity forces integrality. The feasible set is nonempty, since any permutation matrix sits in it, and bounded, since every $x_{ij} \in [0,1]$, so the minimum is attained at a vertex. Attainment at a vertex is standard. A bounded polyhedron is the convex hull of its finitely many vertices (the Minkowski-Weyl theorem), and a linear objective on a mix $x = \sum_t \theta_t v_t$ of the vertices $v_t$, with weights $\theta_t \ge 0$ summing to $1$, equals the same mix of the vertex values, so no point can beat the best vertex. At a vertex, enough of the $x_{ij} \ge 0$ constraints are tight to pin $x$ down. Those tight constraints hold their variables at $0$, and the surviving basic variables $x_B$ are fixed by the equality constraints. Dropping the one redundant equality (one is redundant because the prediction constraints and the target constraints each sum to the same equation $\sum_{i,j} x_{ij} = N$, so any single constraint follows from the other $2N-1$) leaves a square nonsingular submatrix $B$ with $B x_B = b$, where $b$ is a vector of ones. Cramer’s rule writes each entry of $x_B$ as a ratio of two determinants, $B$ with one column swapped for $b$ over $\det B$. The numerator is an integer matrix’s determinant, hence an integer, and $\det B = \pm 1$ makes every entry an integer. With $x \ge 0$ and every row and column summing to $1$, each holds a single $1$ and the rest $0$, a permutation matrix. The relaxation is therefore exact, its optimum a genuine assignment with nothing to round.

So much for the primal. Every linear program has a companion maximization, its dual, whose value never exceeds the primal minimum. Here the dual attaches a potential to each constraint, $u_i$ per row and $v_j$ per column,

$$\max_{u,v\in\mathbb{R}^N}\ \sum_i u_i + \sum_j v_j \quad\text{s.t.}\quad u_i + v_j \le \text{cost}(i,j)\ (\forall i,j),$$

and dual feasibility is exactly the reduced cost $\tilde c(i,j) = \text{cost}(i,j) - u_i - v_j$ staying nonnegative everywhere.

The optimality certificate. Take any row potentials $u_1,\dots,u_N$ and column potentials $v_1,\dots,v_N$, and sum the reduced cost $\tilde c(i,j) = \text{cost}(i,j) - u_i - v_j$ along an assignment $\sigma$:

$$\begin{aligned} \sum_{j=1}^{N} \tilde c(\sigma(j),\, j) &= \sum_{j=1}^{N}\big[\text{cost}(\sigma(j),\, j) - u_{\sigma(j)} - v_j\big] && \text{definition of } \tilde c\\[2pt] &= \sum_{j=1}^{N}\text{cost}(\sigma(j),\, j) - \sum_{j=1}^{N} u_{\sigma(j)} - \sum_{j=1}^{N} v_j && \text{split the sum}\\[2pt] &= J(\sigma) - \sum_{i=1}^{N} u_i - \sum_{j=1}^{N} v_j && \text{reindex the } u \text{ sum}. \end{aligned}$$

The reindexing is legitimate because $\sigma$ is a permutation. The map $j \mapsto \sigma(j)$ hits every row index once, so $\sum_j u_{\sigma(j)} = \sum_i u_i$. The two potential sums do not depend on $\sigma$, so the potentials shift every assignment’s total by the same constant, and $\arg\min_\sigma$ is unchanged. Call the potentials feasible when $\tilde c(i,j) \ge 0$ everywhere, and an edge tight when $\tilde c(i,j) = 0$.

Lemma (a tight assignment is optimal). If feasible potentials admit an assignment $\sigma$ that uses only tight edges, then $\sigma$ minimizes $J$.

Proof. Let $\sigma'$ be any assignment. Rearranging the certificate identity,

$$\begin{aligned} J(\sigma') &= \sum_{j=1}^{N} \tilde c(\sigma'(j),\, j) + \sum_{i=1}^{N} u_i + \sum_{j=1}^{N} v_j && \text{the identity above}\\[2pt] &\ge \sum_{i=1}^{N} u_i + \sum_{j=1}^{N} v_j && \text{since every } \tilde c \ge 0. \end{aligned}$$

So $\sum_i u_i + \sum_j v_j$ lower-bounds the cost of every assignment. The tight $\sigma$ uses only edges with $\tilde c(\sigma(j),\, j) = 0$, so its first sum vanishes and $J(\sigma) = \sum_i u_i + \sum_j v_j$. That equals the lower bound, so no assignment costs less. $\blacksquare$

A full assignment sitting on tight edges is a certificate that meets that bound. The equality is complementary slackness. An optimal $x$ puts weight only where $\tilde c = 0$.

The primal-dual algorithm. Instead of checking all $N!$ assignments, the Hungarian method searches for feasible potentials and a complete matching on tight edges. It maintains two invariants. Reduced costs stay nonnegative, and matched edges stay tight. Consider three predictions against three targets, with prediction $i$ in row $i$, target $j$ in column $j$, and the costs collected in a matrix written $C$ for this example only.

$$C = \begin{pmatrix} 4 & 1 & 3\\ 2 & 0 & 5\\ 3 & 2 & 2 \end{pmatrix}.$$

Make zeros. Start with $u = v = 0$. Row and column reductions create enough zero reduced costs to begin matching. Subtract each row’s minimum from its row, which sets $u_i = \min_j \text{cost}(i,j)$, then subtract each remaining column minimum, $v_j = \min_i \tilde c(i,j)$. The certificate identity makes this legal because the potentials shift every assignment by the same constant. Here $u = (1, 0, 2)$ and then $v = (1, 0, 0)$.

$$\tilde c = C - u - v = \begin{pmatrix} 2 & 0 & 2\\ 1 & 0 & 5\\ 0 & 0 & 0 \end{pmatrix}.$$

Read a reduced cost as regret, how much worse a pairing is than the best deal available to its row and column. Every row and every column now holds a zero, and the column pass cannot wipe out a row’s zero, because that zero sits in a column whose minimum is already $0$, so $v_j = 0$ there and the entry survives untouched.

Match on zeros. Pair rows to columns through tight edges only, never reusing a row or a column. Row 1 takes column 2 and row 3 takes column 1. Row 2’s only zero sits in column 2, already taken, so the matching stalls at two pairs.

The tool for getting unstuck is the augmenting path. Call a row or column free while it has no partner. An augmenting path starts at a free row, steps to a column through a zero, steps from that column to its partner row through the matched edge, steps to a new column through another zero, and keeps alternating until it lands on a free column. Flip every edge along it, matched becoming unmatched and back. Each interior row and column trades one partner for another while the two free endpoints gain one, so the matching grows by exactly one pair. Matching as far as possible means flipping augmenting paths until none exists.

None exists here. From the free row 2 the only zero leads to column 2, column 2’s partner is row 1, and row 1 owns no other zero to continue through. Dead end.

Create a new zero. The rows and columns just visited form the alternating tree, here rows $I = \{2, 1\}$ and columns $J = \{2\}$. These names are local to this example. Progress needs a zero on an edge from a row in $I$ to a column outside $J$. Take the smallest remaining reduced cost on such an edge.

$$\delta = \min_{i\in I,\ j\notin J}\tilde c(i,j) = \tilde c(2,1) = 1,$$

and pay for it with the potentials. Raise $u_i$ by $\delta$ on every tree row and lower $v_j$ by $\delta$ on every tree column. Three checks, one per kind of entry, show the move is safe.

• Tree row, outside column. The reduced cost drops by $\delta$. By the choice of $\delta$ nothing goes negative, and the minimizer lands exactly on zero. That is the purchased edge, $(2,1)$ here.
• Tree row, tree column. The raise and the drop cancel, so every edge inside the tree, the matched ones included, stays exactly as tight as it was.
• Outside row, tree column. The reduced cost rises by $\delta$, harmless for nonnegativity, and the only zeros this can erase are unmatched ones, because a tree column’s partner sits inside the tree.

So the invariants hold and the tree gains ground. The dual objective $\sum_i u_i + \sum_j v_j$ also rises by $\delta$, because the tree always holds one more row than column, every column it reaches being matched (a free column would already end an augmenting path) and each matched column bringing its partner row along, so only the root row lacks a column. The lower bound climbs toward the optimum with every lift. With $u = (2, 1, 2)$ and $v = (1, -1, 0)$,

$$\tilde c = \begin{pmatrix} 1 & 0 & 1\\ 0 & 0 & 4\\ 0 & 1 & 0 \end{pmatrix}.$$

Augment. The purchased zero at $(2,1)$ extends the search. Row 2 now reaches column 1, column 1’s partner row 3 joins, and row 3 owns a zero in column 3, which is free. That is an augmenting path, row 2 to column 1 to row 3 to column 3. Flip it. Row 2 takes column 1, row 3 moves to column 3, row 1 keeps column 2, and the matching is perfect.

Read off the certificate. In the original matrix the assignment costs $C_{12} + C_{21} + C_{33} = 1 + 2 + 2 = 5$, and the potentials sum to $(2 + 1 + 2) + (1 - 1 + 0) = 5$. Primal equals dual, so by the lemma no assignment can cost less, and enumerating all $3! = 6$ permutations confirms it. The permutation matrix of this matching is the optimal vertex of the linear program, the potentials are the optimal dual solution, and the equality that just closed is complementary slackness checked by hand. On a larger instance nothing new happens. Match through zeros until stuck, buy the cheapest escape zero, augment the moment a free column comes into reach, and stop when every row has a partner.

Each augmentation adds one edge, so at most $N$ augmentations finish the matching. One piece of bookkeeping makes each augmentation cost $O(N^2)$. For every column outside the tree, keep its current slack $\min_{i\in I}\tilde c(i,j)$ and update it in $O(N)$ when a row joins. Each $\delta$ is then read from the slacks rather than found by rescanning the matrix. Every lift either reaches a free column and enables an augmentation, or adds a matched column and its partner row to the tree. Thus at most $N$ lifts occur between augmentations, giving $O(N^3)$ work overall. The Jonker-Volgenant routine called by scipy is a faster-constant refinement of the same primal-dual idea.

The minimum assignment cost has one property the model needs.

Proposition (the matching objective ignores prediction order). Let $\mathcal{J} = \min_{\sigma\in S_N} J(\sigma)$, with the costs evaluated on the model’s current predictions. Relabeling the predictions by any permutation $\pi$ leaves $\mathcal{J}$ unchanged.

Proof: relabeling only reindexes the search space

Proof. Let $\pi \in S_N$ relabel the predictions, so that position $i$ now holds the prediction originally at position $\pi(i)$. Under the relabeling, assignment $\sigma$ pairs target $j$ with the prediction originally at position $\pi(\sigma(j))$, so its cost is

$$\sum_{j=1}^{N} \text{cost}\big(\pi(\sigma(j)),\, j\big) = J(\pi \circ \sigma) \qquad \text{(definition of } J\text{)}.$$

Minimize over $\sigma$. As $\sigma$ ranges over $S_N$, the composite $\pi \circ \sigma$ ranges over all of $S_N$ too, because left multiplication by the fixed $\pi$ is a bijection of the group, undone by composing with $\pi^{-1}$, so every $\tau \in S_N$ is hit exactly once, as $\tau = \pi \circ (\pi^{-1} \circ \tau)$. So

$$\min_{\sigma \in S_N} J(\pi \circ \sigma) = \min_{\tau \in S_N} J(\tau),$$

which is $\mathcal{J}$ before relabeling. $\blacksquare$

The minimum cost is invariant even when several assignments tie. When the optimum is unique, relabeling maps it to the corresponding relabeled assignment, so the downstream loss is unchanged. With exact cost ties, index-dependent tie-breaking can select different minimizers and therefore different gradients.

The model’s storage order carries no meaning, and the matching objective respects that. Once the assignment is fixed, it determines which target supplies each query’s training signal. §8 shows that changing only the matching cost improves AP by more than two points.

An independent nearest-target choice could assign two queries to one object and leave another unmatched. One-to-one training discourages these duplicates and enables inference without non-maximum suppression.

3.2 The training loss

The mask loss is BCE plus Dice,

$$\mathcal{L}_{\text{mask}} = \lambda_{\text{ce}}\,\mathcal{L}_{\text{ce}} + \lambda_{\text{dice}}\,\mathcal{L}_{\text{dice}}, \qquad \lambda_{\text{ce}} = \lambda_{\text{dice}} = 5,$$

and the total loss adds classification, $\mathcal{L} = \mathcal{L}_{\text{mask}} + \lambda_{\text{cls}}\,\mathcal{L}_{\text{cls}}$, where $\mathcal{L}_{\text{cls}} = -\log \hat p_i(c^{\text{gt}})$ is plain cross-entropy against the matched target’s class $c^{\text{gt}}$, with $\lambda_{\text{cls}} = 2$ on matched queries and $0.1$ on $\varnothing$. The gradients show why these are the right terms.

BCE. At a point $x$ with mask logit $z_x$, prediction $m_x = \sigma(z_x)$, and target $g_x \in \{0,1\}$, the loss is

$$\ell_{\text{ce}}(x) = -\big[g_x \log \sigma(z_x) + (1-g_x)\log\big(1-\sigma(z_x)\big)\big].$$

Its gradient through the logit collapses to one clean expression,

$$\frac{\partial \ell_{\text{ce}}}{\partial z_x} = \sigma(z_x) - g_x.$$

The BCE gradient, step by step

The two logarithms differentiate through the sigmoid, by the chain rule in the form $(\log u)' = u'/u$. With $\sigma'(z) = \sigma(z)\big(1-\sigma(z)\big)$ (differentiate $\sigma(z) = (1+e^{-z})^{-1}$ and factor the result),

$$\begin{aligned} \frac{d}{dz}\log\sigma(z) &= \frac{\sigma'(z)}{\sigma(z)} = 1 - \sigma(z) && \text{chain rule, then cancel } \sigma(z),\\[2pt] \frac{d}{dz}\log\big(1-\sigma(z)\big) &= \frac{-\sigma'(z)}{1-\sigma(z)} = -\sigma(z) && \text{chain rule, then cancel } 1-\sigma(z). \end{aligned}$$

Substitute both, then expand and cancel:

$$\begin{aligned} \frac{\partial \ell_{\text{ce}}}{\partial z_x} &= -\big[g_x\big(1 - \sigma(z_x)\big) - (1-g_x)\,\sigma(z_x)\big] && \text{substitute}\\[2pt] &= -\big[g_x - g_x\sigma(z_x) - \sigma(z_x) + g_x\sigma(z_x)\big] && \text{expand}\\[2pt] &= \sigma(z_x) - g_x && \text{cancel, then distribute the minus}. \end{aligned}$$

Clean, well conditioned (the gradient is bounded by 1 whatever the logit, so a confident mistake still gets a useful signal), and independent per point. The same expression holds for bilinearly sampled targets $g_x \in [0,1]$. The released loss averages BCE over sampled points and normalizes it by the number of matched masks. That per-point independence is also a weakness. Summed over a dense grid, the total gradient a segment receives scales with its area, so large segments dominate the update and small ones barely register.

Dice. Mask2Former uses a soft Dice loss that normalizes overlap by predicted and target size [Milletari et al. 2016]. Write $O = \sum_x m_xg_x$ for the overlap and $S = \sum_xm_x + \sum_xg_x$ for the size sum. The released implementation is

$$\mathcal{L}_{\text{dice}}(m,g) = 1 - \frac{2O+1}{S+1}.$$

The added $1$s smooth the empty-mask case, and Milletari’s original uses sums of squares in the denominator. Dice couples all sampled points through one region-level score. Writing the smoothing constant as $\varepsilon = 1$, the gradient at a single point is

$$\frac{\partial \mathcal{L}_{\text{dice}}}{\partial m_x} = -\,\frac{2\,g_x\,(S+\varepsilon) - (2O+\varepsilon)}{(S+\varepsilon)^2}.$$

The Dice gradient, step by step

Differentiate with respect to the single probability $m_x$. Every other term of each sum is constant, so

$$\frac{\partial O}{\partial m_x} = g_x, \qquad \frac{\partial S}{\partial m_x} = 1.$$

Then, using the quotient rule in the form $(u/v)' = (u'v - uv')/v^2$,

$$\begin{aligned} \frac{\partial \mathcal{L}_{\text{dice}}}{\partial m_x} &= -\,\frac{\partial}{\partial m_x}\!\left(\frac{2O+\varepsilon}{S+\varepsilon}\right) && \text{drop the constant } 1\\[4pt] &= -\,\frac{2\,(\partial O/\partial m_x)\,(S+\varepsilon) - (2O+\varepsilon)\,(\partial S/\partial m_x)}{(S+\varepsilon)^2} && \text{quotient rule}\\[4pt] &= -\,\frac{2\,g_x\,(S+\varepsilon) - (2O+\varepsilon)}{(S+\varepsilon)^2} && \text{insert the partials}. \end{aligned}$$

Read the fraction as a whole. The numerator is of order $S$, so each point’s gradient scales like $1/S$, and summed over the region’s roughly $S$ points the total gradient a segment receives is roughly independent of its area. The scale invariance is exact in one specific sense. Tile $k$ disjoint copies of the same prediction and target pattern, and every sum scales by $k$, leaving the unsmoothed loss unchanged. Dice weights a ten-pixel object and a sky-sized region equally. Its price lands one step further back, in logit space. When the prediction confidently misses the target, $m_x$ is near 0 exactly where $g_x = 1$, and the chain rule multiplies the healthy $\partial\mathcal{L}_{\text{dice}}/\partial m_x$ by $\sigma'(z_x) \approx 0$, so the gradient dies through the saturated sigmoid. BCE cancels that factor exactly, as its logit gradient $\sigma(z_x) - g_x$ shows. Dice does not. Summing the two losses is the standard combination. BCE supplies gradient at every point, and Dice makes that gradient scale-invariant.

What happened to focal loss? MaskFormer used focal loss [Lin et al. 2017] with weight $20$, cross-entropy scaled down on already-confident predictions so easy background points stop dominating the gradient. Mask2Former reverts to plain BCE at weight $5$. Focal loss exists to fight the extreme foreground-background imbalance of dense evaluation, and §8’s point sampling removes most of that imbalance at the source, so the simpler and better-conditioned loss suffices. The paper never measures this change on its own.

The $\varnothing$ down-weighting. With $N = 100$ queries and typically 5 to 20 real segments in an image, no-object outnumbers real matches by roughly 4 to 19 times. At full weight, predicting nothing becomes the dominant gradient. Down-weighting the class term to $0.1$ on unmatched queries is the inexpensive fix, inherited from DETR.

Deep supervision. The model predicts once from $\mathbf{X}_0$ and after each of nine decoder layers, ten supervised predictions per forward pass through the same shared heads. In the released criterion, Hungarian matching is recomputed for each of the ten outputs, so every intermediate prediction trains toward its own assignment. In most papers auxiliary losses are an optimization nicety. Here they are structural. The thresholded intermediate masks become attention masks in §5, and being a useful attention gate is otherwise never optimized.

4. The architecture

Mask2Former keeps MaskFormer’s three-part structure. A backbone produces image features. A pixel decoder turns them into a feature pyramid and a stride-4 pixel embedding map $\mathcal{E}_{\text{pixel}}$. A Transformer decoder processes $N$ queries against the image features. Each query $q_i$ produces a class distribution and a mask.

$$\hat p_i = \operatorname{softmax}(W_{\text{cls}}\, q_i + b_{\text{cls}}), \qquad m_i(x) = \sigma\big(\operatorname{MLP}(q_i)^\top\, \mathcal{E}_{\text{pixel}}(x)\big).$$

The class head is one linear layer with weights $W_{\text{cls}}$ and bias $b_{\text{cls}}$. It predicts what the query represents. The mask head is a small multi-layer perceptron (MLP). It maps the same query into an embedding and compares it with every pixel embedding to predict where the segment lies. Masks are produced at stride 4 and then upsampled. The paper uses either a ResNet [He et al. 2016] or Swin [Liu et al. 2021] backbone. MaskFormer used a feature pyramid network (FPN) pixel decoder [Lin et al. 2017b] and six standard decoder layers over one stride-32 feature map.

5. Masked attention

Global cross-attention can learn where an object is, but the authors argue that learning this localization is slow. Masked attention starts from the region each query currently predicts and refines it layer by layer.

5.1 Why localization helps

A standard decoder layer updates queries by cross-attention over the whole feature map:

$$\mathbf{X}_l = \operatorname{softmax}\!\big(\mathbf{Q}_l \mathbf{K}_l^{\top}\big)\,\mathbf{V}_l + \mathbf{X}_{l-1}. \tag{1}$$

Here $\mathbf{Q}_l$ comes from the previous query state, while $\mathbf{K}_l$ and $\mathbf{V}_l$ come from the image features. For readability, the equation omits the standard $1/\sqrt d$ scaling, the multi-head split, and the query and image position embeddings used by the implementation. It also omits the self-attention and feed-forward steps that complete the layer, folding them into $\mathbf{X}_l$. The paper writes the recurrence the same way.

Nothing restricts where a query looks. Earlier DETR studies linked slow convergence to the time needed for cross-attention to become spatially focused [Gao et al. 2021, Sun et al. 2021]. Mask2Former tests the same hypothesis. In its converged cross-attention baseline, about 20 percent of the attention mass falls inside the matched ground-truth segment on COCO val. This does not show that global attention cannot localize. It shows that this baseline remains diffuse, even after training.

One toy calculation shows the effect of having many background locations. Let $n_f$ foreground locations share logit $\mu_f$, and let $n_b$ background locations share logit $\mu_b$. Their foreground attention mass is

$$\begin{aligned} \frac{n_f\, e^{\mu_f}}{n_f\, e^{\mu_f} + n_b\, e^{\mu_b}} &= \frac{1}{1 + \dfrac{n_b\, e^{\mu_b}}{n_f\, e^{\mu_f}}} && \text{divide through by } n_f\, e^{\mu_f}\\[6pt] &= \frac{1}{1 + \dfrac{n_b}{n_f}\, e^{-(\mu_f - \mu_b)}} && \text{combine the exponentials}. \end{aligned}$$

Now suppose the object covers 2 percent of the image, so $n_f = 0.02\,|\Omega|$, $n_b = 0.98\,|\Omega|$, and $n_b/n_f = 0.98/0.02 = 49$. With foreground logit margin $\mu_f - \mu_b = 2$ and $e^{-2} \approx 0.135$,

$$\frac{1}{1 + 49\,e^{-2}} \approx \frac{1}{1 + 49 \cdot 0.135} = \frac{1}{7.615} \approx 0.13.$$

In this toy model, the background wins because it has many more locations. Real attention logits are not two spikes, and sufficiently concentrated global attention can overcome the imbalance. The calculation only explains why explicit localization can make learning easier.

5.2 The masked update

Masked attention assumes that a query can update from features inside its current segment, while self-attention carries information between queries. The mask is added before the cross-attention softmax.

$$\mathbf{X}_l = \operatorname{softmax}\!\big(\mathcal{M}_{l-1} + \mathbf{Q}_l \mathbf{K}_l^{\top}\big)\,\mathbf{V}_l + \mathbf{X}_{l-1}, \tag{2}$$ $$\mathcal{M}_{l-1}(x) = \begin{cases} 0 & \text{if } M_{l-1}(x) = 1,\\[2pt] -\infty & \text{otherwise.} \end{cases} \tag{3}$$

$M_{l-1}$ is the query’s mask from the previous layer. Its logits are bilinearly resized to the current attention resolution, passed through the sigmoid, and thresholded at $0.5$. The initial mask $M_0$ is predicted directly from the supervised learnable queries (§7).

Adding $-\infty$ before softmax gives forbidden locations zero weight and renormalizes the allowed weights to sum to 1. Zeroing weights after softmax would leave their sum below 1 unless they were normalized again. The proof is folded below.

Proposition (additive $-\infty$ masking is exact renormalization). Let $z \in \mathbb{R}^{n}$ be logits and $S \subseteq \{1,\dots,n\}$ a nonempty allowed set (an index set, not §3.2’s size sum), with $\mathcal{M}_i = 0$ for $i\in S$ and $\mathcal{M}_i = -\infty$ otherwise. Then

$$\operatorname{softmax}(z + \mathcal{M})_i = \frac{e^{z_i}\,\mathbb{1}[i\in S]}{\sum_{j\in S} e^{z_j}},$$

which is exactly the softmax of the original logits restricted to $S$.

Proof: the forbidden exponentials become exact zeros

Proof. Fix an index $i$ and write its softmax entry from the definition:

$$\operatorname{softmax}(z + \mathcal{M})_i = \frac{e^{z_i + \mathcal{M}_i}}{\sum_{j=1}^{n} e^{z_j + \mathcal{M}_j}}.$$

Evaluate the numerator by cases, using $e^{-\infty} = 0$:

$$e^{z_i + \mathcal{M}_i} = \begin{cases} e^{z_i}\,e^{0} = e^{z_i}, & i \in S \ \ (\mathcal{M}_i = 0),\\[2pt] e^{z_i}\,e^{-\infty} = 0, & i \notin S \ \ (\mathcal{M}_i = -\infty), \end{cases}$$

so in one line the numerator is $e^{z_i}\,\mathbb{1}[i \in S]$. Split the denominator at $S$:

$$\begin{aligned} \sum_{j=1}^{n} e^{z_j + \mathcal{M}_j} &= \sum_{j\in S} e^{z_j}\,e^{0} + \sum_{j\notin S} e^{z_j}\,e^{-\infty} && \text{split over } S \text{ and its complement}\\[2pt] &= \sum_{j\in S} e^{z_j} \;>\; 0 && \text{the second sum is }0,\text{ and }S\text{ nonempty makes the first positive}. \end{aligned}$$

Divide numerator by denominator:

$$\operatorname{softmax}(z + \mathcal{M})_i = \frac{e^{z_i}\,\mathbb{1}[i \in S]}{\sum_{j\in S} e^{z_j}},$$

the softmax of the original logits restricted to $S$. $\blacksquare$

The paper’s R50 ablation on COCO compares four updates. Plain cross-attention reaches 37.8 AP, spatially modulated co-attention (SMCA) [Gao et al. 2021] reaches 37.9, mask pooling reaches 43.1, and masked attention reaches 43.7. Mask pooling averages the features inside a mask. Masked attention still learns a distribution within the allowed region.

Additive attention masking already existed in Transformer decoders [Vaswani et al. 2017]. Mask2Former makes the mask spatial, query-specific, and dependent on the previous mask prediction. The prediction restricts the next read, and the next query state produces a new prediction.

5.3 Gradients and empty masks

The attention gate is Boolean, so gradients do not pass through the threshold. The learning signal for mask quality arrives only through §3.2’s per-layer auxiliary losses, and that proxy happens to be the right objective. The mask loss is minimized exactly at $m = g$. For BCE that holds pointwise. For soft Dice it follows from $2\sum_x m_x g_x \le \sum_x m_x + \sum_x g_x$, with equality only at $m = g$. One line to check: $m_x + g_x - 2m_xg_x = m_x(1-g_x) + g_x(1-m_x) \ge 0$ for $m_x \in [0,1]$ and binary $g_x$, vanishing only at $m_x = g_x$, then sum over $x$. Better, the gate is already exactly right once every point’s error $|m_x - g_x|$ falls below $0.5$, well before the mask itself converges. Imperfect intermediate masks therefore already make good gates. Residual connections preserve the previous query state alongside each new update, so one bad read cannot erase a query.

The implementation also handles an empty predicted region. The detail lives in the official code and not the paper, and without it training produces NaNs (not-a-number, the value floating-point math returns for $0/0$) within minutes. If a query’s thresholded mask is empty at some scale, every logit is $-\infty$ and the softmax is $0/0$. The code detects such rows and flips them to fully unmasked, quietly falling back to plain cross-attention for that query at that layer:

attn_mask = (mask_logits.sigmoid() < 0.5).flatten(2)   # True = forbidden
attn_mask[attn_mask.all(dim=-1)] = False               # the guard rail

One poisoned row would not stay contained. The following self-attention averages all $N$ queries together, so a single NaN spreads to every query in one step, then into the loss and every gradient behind it. The guard is also what keeps the masking proposition’s hypothesis true, since flipping the row to fully unmasked makes the allowed set $S$ the whole grid instead of empty.

5.4 Did it work?

In the supplementary analysis, average attention mass inside the matched foreground rises from about 0.20 to 0.59. One masked-attention layer also outperforms nine standard cross-attention layers on PQ in the reported layer-wise comparison. Removing masked attention costs 5.9 AP, 4.8 PQ, and 1.7 mIoU. This is the largest drop in the paper’s ablations.

6. Feeding the decoder

6.1 The cost structure

Cross-attention cost grows with the number of image tokens. Its main attention operations cost $O(NH_lW_lC)$ for $N$ queries, an $H_l$ by $W_l$ feature map, and channel width $C$. For a 1024 by 1024 input, strides 32, 16, and 8 contain 1,024, 4,096, and 16,384 tokens. The finest map therefore costs about sixteen times more to attend over than the coarsest.

6.2 The schedule

Mask2Former reads one scale per decoder layer. Layers 1, 2, and 3 use strides 32, 16, and 8, and the cycle repeats twice more. Feeding all three scales to every layer reaches 44.0 AP at 247 GFLOPs. Using only stride 8 reaches the same AP at 239 GFLOPs. The round-robin schedule reaches 43.7 AP at 226 GFLOPs, trading 0.3 AP for lower cost. Removing high-resolution features lowers AP by 2.2.

Each feature also receives DETR’s two-dimensional sinusoidal position encoding and a learned embedding that identifies its scale. The position coordinates are normalized per axis before the sine and cosine features are formed. The scale embedding follows Deformable DETR [Zhu et al. 2021].

6.3 The pixel decoder

The default pixel decoder has six multi-scale deformable-attention (MSDeformAttn) layers over strides 8, 16, and 32. It then fuses the result with stride-4 backbone features to produce pixel embeddings. Each location samples a small learned set of points from every feature level instead of attending densely to all locations [Zhu et al. 2021]. The exact operator and cost are folded below.

Multi-scale deformable attention

For feature $z_q$ at normalized reference point $\hat p_q$,

$$\text{MSDeformAttn}\big(z_q, \hat p_q, \{x^l\}\big) = \sum_{m=1}^{M} W_m \Big[ \sum_{l=1}^{L}\sum_{k=1}^{K} A_{mlqk}\; W'_m\, x^l\big(\psi_l(\hat p_q) + \Delta p_{mlqk}\big) \Big].$$

Here $m$, $l$, and $k$ index the $M$ heads, $L$ levels, and $K$ sample points per level, and $x^l$ is the feature map at level $l$. This $K$ counts points, not classes, and this $M$ counts heads, not masks. $W'_m$ projects values into head $m$, $W_m$ projects that head back to width $C$, and $\psi_l$ maps the normalized reference point into level $l$. The offsets and weights are predicted from $z_q$, with $\sum_{l,k}A_{mlqk}=1$ for each head $m$ and query $q$. Bilinear interpolation reads fractional locations. The aggregation processes $MLK$ entries of width $C/M$, so it costs $O(LKC)$ per query apart from the learned projections. Dense attention would read every location at every level.

Among conventional pyramid decoders, BiFPN has the highest AP at 43.5 and ties FaPN at 51.8 PQ. FaPN has the highest mIoU at 46.8 [Tan et al. 2020, Huang et al. 2021]. MSDeformAttn is higher on all three at 43.7 AP, 51.9 PQ, and 47.2 mIoU.

7. Rewiring the decoder layer

Mask2Former changes the decoder layer in three ways without adding FLOPs. Together they improve AP by about 1.4, PQ by 1.1, and mIoU by 0.9. Their effects overlap, so the one-at-a-time ablations in §11 do not add to those totals.

Masked attention comes first. The standard order is self-attention, then cross-attention, then the feed-forward network. Mask2Former uses masked attention, then self-attention, then the feed-forward network. Running cross-attention first gives each query image context before the queries interact. Restoring the original order costs 0.5 AP.

Query features are learnable and directly supervised. DETR zero-initializes query features and learns only positional embeddings. Mask2Former makes $\mathbf{X}_0$ learnable and supervises its masks before the decoder runs. The query positional embeddings stay learnable, as in DETR. Every decoder layer adds them to the query features when computing attention, and they appear as query_pos in Appendix A. Direct supervision also gives the first layer a useful gate. Without direct supervision, learnable queries match zero initialization on AP and PQ but improve mIoU from 45.5 to 47.0. Adding supervision reaches 43.7 AP, 51.9 PQ, and 47.2 mIoU. The paper compares the supervised initial predictions to region proposals [Ren et al. 2015] that the decoder refines.

Dropout is removed. Restoring dropout lowers AP by 0.7 and PQ by 0.6 in the ablation.

8. Matching and training on points

8.1 The memory problem and the estimator

Dense mask losses are expensive. Mask2Former evaluates them at sampled coordinates instead. Uniform points build matching costs, while prediction-dependent uncertain points train matched masks.

MaskFormer computed dense mask costs for every prediction-target pair and dense losses again for matched pairs at every supervised head. This limited training to one image per 32 GB GPU. Following PointRend [Kirillov et al. 2020], Mask2Former evaluates these terms at $K_{\text{pt}} = 12{,}544 = 112^2$ sampled points.

The implementation samples normalized continuous coordinates and reads masks with bilinear interpolation. For an additive point loss such as BCE, the mean over uniform samples is an unbiased estimate of the spatial average, and its variance falls as $1/K_{\text{pt}}$. The derivation is folded below.

Uniform point estimator

Let $u_1,\dots,u_{K_{\text{pt}}}$ be independent uniform points in the unit square, and let $\ell(u)$ be the additive loss on the interpolated masks. The Monte Carlo average is

$$\hat{\mathcal{L}} = \frac{1}{K_{\text{pt}}}\sum_{k=1}^{K_{\text{pt}}}\ell(u_k)$$

Linearity of expectation gives the spatial mean. Independence removes the covariance terms in the variance.

$$\mathbb{E}[\hat{\mathcal{L}}] = \frac{1}{K_{\text{pt}}}\sum_k \mathbb{E}[\ell(u_k)] = \int_{[0,1]^2}\ell(u)\,du, \qquad \operatorname{Var}[\hat{\mathcal{L}}] = \frac{1}{K_{\text{pt}}}\operatorname{Var}[\ell(u_1)].$$

This statement applies to additive point losses. Dice is a ratio of sampled sums, so its sampled value is not an unbiased estimate of dense Dice. The code and the paper rely on the empirical result in the section 8.3 ablation rather than an exact equivalence proof.

8.2 Two sampling rules for two jobs

The two sampling rules serve different parts of training.

Matching uses shared uniform points. One set of normalized coordinates is sampled uniformly and reused for every prediction and target mask in the image. Reuse makes the cost matrix cheaper to build. Sharing can also reduce the variance of a cost difference when the two sampled costs are positively correlated, but that is a conditional observation rather than a claim made by the paper.

The conditional covariance argument

Let $A$ and $B$ be two additive cost estimates evaluated on the same $K_{\text{pt}}$ uniform points. Then

$$\operatorname{Var}[A-B] = \operatorname{Var}[A]+\operatorname{Var}[B]-2\operatorname{Cov}(A,B).$$

Independent point sets make the covariance term zero. Shared points can make it positive because both estimates respond to the same sampled locations. When it is positive, the variance of the difference falls. When it is negative, the variance rises. The equation gives a condition under which sharing stabilizes a comparison. It does not prove that every matching comparison becomes more stable.

Training uses uncertain points. For each matched mask, the PointRend sampler draws $3K_{\text{pt}}$ uniform candidates [Kirillov et al. 2020]. It keeps the $0.75K_{\text{pt}}$ candidates whose prediction logits are closest to zero, then adds $0.25K_{\text{pt}}$ fresh uniform points. The uncertainty score depends on the prediction, not the ground truth. These points often lie near a predicted boundary, but uncertain interiors can also be selected.

This training sampler is intentionally nonuniform and applies no importance correction. It therefore changes the training objective rather than providing an unbiased estimate of the dense loss. The section 8.3 ablation shows that the changed objective preserves performance while reducing memory.

8.3 Point-sampling ablation

matching on	training loss on	AP (COCO)	PQ (COCO)	mIoU (ADE20K)	memory
masks	masks	41.0	50.3	45.9	18 GB
masks	points	41.0	50.8	45.9	6 GB
points	masks	43.1	51.4	47.3	18 GB
points	points	43.7	51.9	47.2	6 GB

Using points for the training loss reduces reported training memory from 18 GB to 6 GB in the paper’s R50 COCO ablation. AP and mIoU stay the same in the dense-matching rows, while PQ rises from 50.3 to 50.8. Using points for matching raises AP from 41.0 to 43.1 with dense training, then to 43.7 with point training. The table measures the effect. Why point-based matching improves the assignment is left open.

9. Training recipe

The table compares Mask2Former’s training recipe with MaskFormer’s. In the decoder row, SA, CA, and MA abbreviate self-attention, cross-attention, and masked attention.

	MaskFormer	Mask2Former
optimizer	AdamW [Loshchilov & Hutter 2019], lr $10^{-4}$	AdamW, lr $10^{-4}$
weight decay	$10^{-4}$	0.05
backbone lr multiplier	0.1 (CNN backbones)	0.1 (CNN and Transformer backbones)
schedule (COCO)	300 epochs at batch 64	50 epochs at batch 16, lr ×0.1 at 90% and 95% of steps
augmentation	standard scale and crop	LSJ (large-scale jittering) [Ghiasi et al. 2021], scale 0.1 to 2.0 with a fixed $1024^2$ crop
mask loss	focal ($\lambda{=}20$) + dice ($\lambda{=}1$), dense	BCE ($\lambda{=}5$) + dice ($\lambda{=}5$) on 12,544 points
$\lambda_{\text{cls}}$	1.0	2.0, with a 0.1 no-object multiplier
decoder	6 layers, SA→CA→FFN, dropout 0.1, stride 32 only, zero-init queries	9 layers, MA→SA→FFN, no dropout, strides {32,16,8}×3, learnable supervised queries

COCO inference resizes the shorter image side to 800 pixels and caps the longer side at 1333. Most models use 100 queries. The largest panoptic and instance models use 200. In the R50 ablation, 100 queries gives the best AP and mIoU, while 200 raises PQ from 51.9 to 52.2. The best setting depends on how many segments an image contains.

Training settings for the other datasets

Cityscapes [Cordts et al. 2016] uses 90,000 iterations with 512 by 1024 crops. ADE20K [Zhou et al. 2017] uses 640 by 640 crops for panoptic and instance training. Semantic crop size varies by backbone. Mapillary Vistas [Neuhold et al. 2017] uses 300,000 iterations with 1024 by 1024 crops.

Post-processing follows MaskFormer. Semantic output sums the masks weighted by each class probability, then selects the highest-scoring class at every pixel.

Panoptic output first keeps queries whose best class is not no-object and whose class probability exceeds $0.8$. Each pixel chooses the kept query with the largest class-probability times mask-probability score. The pixel is written only if the winning query’s mask probability is at least $0.5$. For each query, the code divides the number of pixels it wins by the number of pixels in its own thresholded mask and discards the segment when this ratio is below $0.8$. The final segment is the intersection of the winning pixels and that thresholded mask. Stuff segments with the same class are merged. Rejected pixels remain void.

Instance output flattens the $N\times K$ query-class probabilities and selects the highest-scoring pairs. The released code keeps the top 100 pairs per image. The same query mask can therefore appear with more than one class before the top-pair cutoff. For a selected pair $(i,c)$, the final score multiplies its class probability by the average mask probability over the predicted foreground.

$$s_{i,c} = \hat p_i(c)\, \frac{\sum_x m_i(x)\,\mathbb{1}[m_i(x)>0.5]} {\sum_x \mathbb{1}[m_i(x)>0.5] + \varepsilon}.$$

The implementation uses $\varepsilon = 10^{-6}$, so an empty foreground scores zero.

10. Results

The largest models reported in the paper achieve the following results.

task / dataset	Mask2Former (Swin-L)	previous best	margin
Panoptic, COCO val	57.8 PQ	MaskFormer 52.7, K-Net 54.6	+5.1 / +3.2
Instance, COCO val	50.1 AP (36.2 boundary AP)	Swin-HTC++ 49.5 (34.1)	+0.6 (+2.1)
Semantic, ADE20K val	57.7 mIoU (Swin-L, FaPN, multi-scale inference)	BEiT 57.0	+0.7 at less than half the parameters

Boundary AP uses the minimum of mask IoU and Boundary IoU [Cheng et al. 2021c]. Multi-scale in the semantic row means test-time inference over several input sizes. It is unrelated to the decoder schedule. The 57.7 mIoU model also uses FaPN instead of the default MSDeformAttn pixel decoder.

With ResNet-50, Mask2Former reaches 51.9 PQ after 50 epochs. MaskFormer reports 46.5 after 300 epochs. The comparison includes architecture, loss, augmentation, batch size, and optimizer-setting changes, so it should not be read as a controlled convergence experiment. The paper’s appendix isolates one of these variables. Trained with standard augmentation instead of LSJ, R50 Mask2Former converges in 25 epochs, so the short schedule is not an artifact of the augmentation.

The detailed results show three contrasts. Large-object AP reaches 71.2 on COCO test-dev, while small-object AP reaches 29.1. COCO buckets objects by mask area, small under 32 by 32 pixels and large over 96 by 96. Boundary AP improves by 2.1 over HTC++, compared with 0.6 overall. The larger Boundary AP gain shows better boundary accuracy, but the comparison does not isolate its cause. Swin-T Mask2Former reaches 53.2 PQ at 232 GFLOPs, while Swin-L MaskFormer reaches 52.7 at 792 GFLOPs. Throughput moves the other way. The R50 panoptic model runs at 8.6 frames per second, compared with MaskFormer’s 17.6.

The architecture is also evaluated on Cityscapes, ADE20K, and Mapillary Vistas. A Swin-L panoptic checkpoint reaches 66.6 PQ and 43.6 $\text{AP}^{\mathrm{Th}}_{\mathrm{pan}}$ on Cityscapes, the instance AP on thing classes computed from the panoptic checkpoint. The separate Cityscapes instance checkpoint reaches 43.7 AP. Swin-L panoptic checkpoints reach 48.1 PQ on ADE20K and 45.5 PQ on Mapillary Vistas. Each checkpoint is trained for its dataset and task.

11. Ablations and interactions

The table reports controlled R50 ablations across all three tasks.

change (removed or varied)	ΔAP	ΔPQ	ΔmIoU	takeaway
remove masked attention	−5.9	−4.8	−1.7	largest measured drop
remove multi-scale high-res features	−2.2	−1.7	−1.1	finer features matter
replace point matching with dense masks	−2.7	−1.1	−1.3*	point matching improves all three tasks
replace supervised learnable features with zero-init	−0.8	−0.7	−1.8	direct supervision of initial queries matters
restore dropout	−0.7	−0.6	0.0	dropout hurts AP and PQ in this setup
vanilla layer order	−0.5	−0.3	−0.9	masked attention first performs better

*Measured with point-sampled training loss fixed. The full two-by-two comparison appears in §8.3.

The sequential substitutions show how the complete system was assembled. Retraining MaskFormer with Mask2Former’s recipe raises AP from 34.0 to 37.8. Replacing the decoder raises it to 41.5. Replacing the FPN pixel decoder with MSDeformAttn reaches 43.7. These increments are not independent causal shares. The components interact, and a different substitution order could produce different gains.

12. Limitations

Universal means one architecture, not one checkpoint. The paper trains separate models for each dataset and task. A panoptic COCO checkpoint reaches 41.7 instance AP, compared with 43.7 for instance-specific training. Its semantic score is 61.7 mIoU, compared with 61.5 for semantic-specific training. Results differ across ADE20K and Cityscapes, so panoptic training does not consistently replace task-specific training.

Small objects remain weaker. Small-object AP is 29.1, compared with 71.2 for large objects. The paper lists fuller use of the feature pyramid as future work. Mask2Former is also slower than MaskFormer, and its query count must be chosen for the expected number of segments.

13. Later work

The same group extended Mask2Former to video instance segmentation [Cheng et al. 2021b]. OneFormer [Jain et al. 2023] trains one task-conditioned model for all three image segmentation tasks. Mask DINO [Li et al. 2023] combines mask prediction with DETR-style detection. These models keep the query-based mask-classification structure while changing the training objective or supported outputs.

14. Conclusion

Mask2Former’s changes support one another rather than stack. Matching gives each query one target. Deep supervision makes every intermediate mask worth trusting. Masked attention turns those masks into the next layer’s reading list, and point sampling makes the loop affordable enough to train. That is why §11’s one-at-a-time deltas never add up to the whole. The parts were built as a loop, not a list.

The headline numbers were passed within a year, as headline numbers are. What the successors in §13 kept is the loop. A query states a hypothesis about a segment, the image grades it, and the next layer reads only where the hypothesis points. That design outlived its own results table.

Appendix A. Implementation notes

The reference implementation is facebookresearch/Mask2Former, built on Detectron2 [Wu et al. 2019]. It was archived on January 1, 2025. Hugging Face provides Mask2FormerForUniversalSegmentation with Mask2FormerImageProcessor or AutoImageProcessor. The image processor exposes semantic, instance, and panoptic post-processing methods. The decoder core is shown below in simplified PyTorch-style pseudocode.

def decoder_layer(
    x, feats_l, size_l, pos_l, lvl_emb, query_pos,
    prev_mask_logits, pixel_embeddings, num_heads,
):
    # x: (N,B,C) · feats_l: (H_l*W_l,B,C) · size_l: (H_l,W_l)
    m = F.interpolate(
        prev_mask_logits,
        size=size_l,
        mode="bilinear",
        align_corners=False,
    )
    attn_mask = (m.sigmoid() < 0.5).flatten(2)  # (B,N,H_l*W_l)
    attn_mask = attn_mask[:, None].repeat(
        1, num_heads, 1, 1
    ).flatten(0, 1).detach()                     # (B*num_heads,N,H_l*W_l)
    attn_mask[attn_mask.all(dim=-1)] = False    # Prevent all-masked softmax.

    # The scale embedding is part of both the key and value memory.
    feats_l = feats_l + lvl_emb
    x = norm1(x + cross_attn(
        q=with_pos(x, query_pos),
        k=with_pos(feats_l, pos_l),
        v=feats_l,
        attn_mask=attn_mask,
    ))
    x = norm2(x + self_attn(
        q=with_pos(x, query_pos),
        k=with_pos(x, query_pos),
        v=x,
    ))
    x = norm3(x + ffn(x))

    h = decoder_norm(x).transpose(0, 1)
    cls_logits = class_head(h)
    mask_logits = einsum(
        "bnc,bchw->bnhw", mask_head(h), pixel_embeddings
    )
    return x, cls_logits, mask_logits

The heads run once on $\mathbf{X}_0$ before the loop and after each of nine decoder layers. The initial prediction supplies $M_0$. The final output receives the main loss, and the preceding nine outputs receive auxiliary losses.

Appendix B. Questions and answers

Prove that adding $-\infty$ before the softmax renormalizes over the allowed set, and say precisely what post-softmax zeroing breaks.

Answer

The §5.2 proposition shows that $e^{z+\mathcal{M}}$ vanishes outside $S$. The remaining weights form a restricted softmax and still sum to 1. Zeroing after softmax leaves coefficients whose sum is below 1 unless they are normalized again. This does not imply that the resulting vector norm always shrinks.

Derive the foreground attention share for an object covering fraction $\rho$ of the image with logit margin $\Delta$, and evaluate it at $\rho = 0.02$, $\Delta = 2$.

Answer

Share $= 1/\big(1 + \frac{1-\rho}{\rho}e^{-\Delta}\big) = 1/(1+49e^{-2}) \approx 0.13$. This is the result for the two-logit toy model. It is not a prediction for real attention distributions.

Show that the minimum matching cost is permutation-invariant.

Answer

Relabeling by $\pi$ turns the cost of $\sigma$ into the cost of $\pi\circ\sigma$. Composition with a fixed permutation is a bijection of $S_N$, so the minimum is unchanged. When the optimum is unique, relabeling maps it to the corresponding relabeled assignment and the downstream loss is unchanged. With exact ties, index-dependent tie-breaking can select different minimizers and therefore different gradients.

Gradients do not flow through a thresholded attention mask. How do masks learn to be good gates?

Answer

They do not learn through the Boolean gate. Per-layer deep supervision trains every intermediate mask directly. The auxiliary losses are load-bearing.

Why does matching use shared uniform points while training uses per-mask uncertain points?

Answer

Sharing one uniform coordinate set makes the cost matrix efficient to build. For additive losses, each entry estimates the spatial average of the interpolated point loss. Training instead favors prediction logits near zero and keeps a random fraction for coverage. The uncertain sampler does not use the target mask.

Masked attention improves instance and panoptic metrics more than semantic mIoU in the ablation. What can and cannot be concluded from that?

Answer

The ablation shows a larger gain for instance and panoptic segmentation than for semantic segmentation. Identity separation is a plausible explanation, but the experiment cannot single it out.

Mask2Former produces overlapping soft masks, while panoptic output must be a partition. How are they reconciled?

Answer

Post-processing (§9) imposes exclusivity. Panoptic inference keeps confident non-no-object queries and finds each pixel’s strongest query. It writes the pixel only when the winning mask probability is at least $0.5$, applies the $0.8$ overlap filter, and merges same-class stuff. Rejected pixels remain void. The surviving segments do not overlap.

Why can PQ evaluation use greedy matching while training cannot?

Answer

The §1 lemma proves uniqueness for non-overlapping segments above 0.5 IoU. Training predictions can overlap and use soft costs, so they still require an assignment algorithm.

References

1. Cheng, Misra, Schwing, Kirillov, Girdhar. “Masked-attention Mask Transformer for Universal Image Segmentation.” CVPR 2022. Paper · Supplement · Code
2. Cheng, Schwing, Kirillov. “Per-Pixel Classification is Not All You Need for Semantic Segmentation.” NeurIPS 2021. arXiv:2107.06278
3. Carion, Massa, Synnaeve, Usunier, Kirillov, Zagoruyko. “End-to-End Object Detection with Transformers.” ECCV 2020. arXiv:2005.12872
4. Kirillov, He, Girshick, Rother, Dollár. “Panoptic Segmentation.” CVPR 2019. arXiv:1801.00868
5. Kirillov, Wu, He, Girshick. “PointRend: Image Segmentation as Rendering.” CVPR 2020. arXiv:1912.08193
6. Zhu, Su, Lu, Li, Wang, Dai. “Deformable DETR.” ICLR 2021. arXiv:2010.04159
7. Zhang, Pang, Chen, Loy. “K-Net: Towards Unified Image Segmentation.” NeurIPS 2021. arXiv:2106.14855
8. Wang, Zhu, Adam, Yuille, Chen. “MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers.” CVPR 2021. arXiv:2012.00759
9. He, Gkioxari, Dollár, Girshick. “Mask R-CNN.” ICCV 2017. arXiv:1703.06870
10. Long, Shelhamer, Darrell. “Fully Convolutional Networks for Semantic Segmentation.” CVPR 2015. arXiv:1411.4038
11. Vaswani et al. “Attention Is All You Need.” NeurIPS 2017. arXiv:1706.03762
12. Kuhn. “The Hungarian Method for the Assignment Problem.” Naval Research Logistics Quarterly, 1955.
13. Munkres. “Algorithms for the Assignment and Transportation Problems.” Journal of the SIAM, 1957.
14. Milletari, Navab, Ahmadi. “V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation.” 3DV 2016. arXiv:1606.04797
15. Lin, Goyal, Girshick, He, Dollár. “Focal Loss for Dense Object Detection.” ICCV 2017. arXiv:1708.02002
16. Lin, Dollár, Girshick, He, Hariharan, Belongie. “Feature Pyramid Networks for Object Detection.” CVPR 2017. arXiv:1612.03144
17. Gao, Zheng, Wang, Dai, Li. “Fast Convergence of DETR with Spatially Modulated Co-Attention.” ICCV 2021. arXiv:2101.07448
18. Sun, Cao, Yang, Kitani. “Rethinking Transformer-based Set Prediction for Object Detection.” ICCV 2021. arXiv:2011.10881
19. Liu et al. “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows.” ICCV 2021. arXiv:2103.14030
20. He, Zhang, Ren, Sun. “Deep Residual Learning for Image Recognition.” CVPR 2016. arXiv:1512.03385
21. Chen et al. “Hybrid Task Cascade for Instance Segmentation.” CVPR 2019. arXiv:1901.07518
22. Ren, He, Girshick, Sun. “Faster R-CNN.” NeurIPS 2015. arXiv:1506.01497
23. Tan, Pang, Le. “EfficientDet: Scalable and Efficient Object Detection.” CVPR 2020. arXiv:1911.09070
24. Huang, Lu, Cheng, He. “FaPN: Feature-Aligned Pyramid Network for Dense Image Prediction.” ICCV 2021. arXiv:2108.07058
25. Loshchilov, Hutter. “Decoupled Weight Decay Regularization.” ICLR 2019. arXiv:1711.05101
26. Ghiasi et al. “Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation.” CVPR 2021. arXiv:2012.07177
27. Chen, Papandreou, Kokkinos, Murphy, Yuille. “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs.” TPAMI 2018. arXiv:1606.00915
28. Zhao, Shi, Qi, Wang, Jia. “Pyramid Scene Parsing Network.” CVPR 2017. arXiv:1612.01105
29. Wang, Girshick, Gupta, He. “Non-local Neural Networks.” CVPR 2018. arXiv:1711.07971
30. Fu et al. “Dual Attention Network for Scene Segmentation.” CVPR 2019. arXiv:1809.02983
31. Strudel, Garcia, Laptev, Schmid. “Segmenter: Transformer for Semantic Segmentation.” ICCV 2021. arXiv:2105.05633
32. Xie et al. “SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers.” NeurIPS 2021. arXiv:2105.15203
33. Bao, Dong, Wei. “BEiT: BERT Pre-Training of Image Transformers.” ICLR 2022. arXiv:2106.08254
34. Everingham et al. “The PASCAL Visual Object Classes Challenge: A Retrospective.” IJCV 2015.
35. Lin et al. “Microsoft COCO: Common Objects in Context.” ECCV 2014. arXiv:1405.0312
36. Cordts et al. “The Cityscapes Dataset for Semantic Urban Scene Understanding.” CVPR 2016.
37. Zhou et al. “Scene Parsing through ADE20K.” CVPR 2017.
38. Neuhold, Ollmann, Rota Bulò, Kontschieder. “The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes.” ICCV 2017.
39. Cheng et al. “Mask2Former for Video Instance Segmentation.” 2021. arXiv:2112.10764
40. Jain et al. “OneFormer: One Transformer to Rule Universal Image Segmentation.” CVPR 2023. arXiv:2211.06220
41. Li et al. “Mask DINO: Towards a Unified Transformer-based Framework for Object Detection and Segmentation.” CVPR 2023. arXiv:2206.02777
42. Wu, Kirillov, Massa, Lo, Girshick. “Detectron2.” 2019. github.com/facebookresearch/detectron2
43. Cheng, Girshick, Dollár, Berg, Kirillov. “Boundary IoU: Improving Object-Centric Image Segmentation Evaluation.” CVPR 2021. arXiv:2103.16562

Citation

Suggested citation

Massih, Peter. “Mask2Former, Dissected.” Peter’s Patches, no. 1, Jul 2026. https://peteramassih.com/writing/mask2former.

BibTeX

@article{massih2026mask2former,
  title   = {Mask2Former, Dissected},
  author  = {Massih, Peter},
  journal = {peteramassih.com},
  series  = {Peter's Patches},
  number  = {1},
  year    = {2026},
  month   = {July},
  url     = {https://peteramassih.com/writing/mask2former}
}