PKINet-v2: Towards Powerful and Efficient Poly-Kernel Remote Sensing Object Detection¶

Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/NUST-Machine-Intelligence-Laboratory/PKINet
Area: Object Detection / Remote Sensing
Keywords: oriented object detection, poly-kernel scope attention, axial-strip convolution, multi-scale receptive fields, heterogeneous kernel re-parameterization

TL;DR¶

PKINet-v2 combines heterogeneous kernels to address object geometry and scale in remote sensing, then equivalently merges its training-time aggregation branches for deployment, achieving 80.46% mAP and 54.60 FPS on single-scale DOTA-v1.0 versus 78.39% and 14.05 FPS for PKINet-v1-S.

Background & Motivation¶

Remote sensing detection must identify objects and localize them with oriented boxes regardless of their direction. Slender bridges, compact storage tanks, small vehicles, and large sports fields can coexist in one aerial image, so geometry and scale cannot be treated as unrelated challenges. Improving angle regression or rotated RoI extraction helps represent boxes but does not automatically make backbone features appropriate for these objects. Strip RCNN uses strip kernels to address geometry, whereas LSKNet uses large square kernels to capture context; the paper argues that the former can weaken spatial coherence and tiny textures, while the latter can introduce background around slender targets. The issue is not that either kernel type is universally ineffective, but that a single geometric bias has difficulty accommodating these coexisting objects.

PKINet-v1 already expands coverage with parallel multi-scale square kernels, yet remains constrained by their shared geometric bias. It also has a practical deployment bottleneck: additional branches entail separate operator launches, repeated reads and writes, and intermediate feature materialization that can slow GPU execution. Fewer parameters or fewer theoretical operations therefore do not guarantee greater throughput, and a new architecture should also consider whether its training graph can become an efficient inference graph. In Table 1, v1-S uses 184G FLOPs at 14.05 FPS, whereas v2-S uses 173G FLOPs at 54.60 FPS; the much smaller change in arithmetic cost cannot by itself explain the throughput difference.

The paper treats representation design and deployment conversion together: differently shaped kernels with different sampling densities learn during training, then algebraically mergeable branches are eliminated for inference. Wide coverage does not require abandoning local detail; the central region instead receives additional dense responses to protect small objects. Core Idea: use Poly-Kernel Scope to combine strip-shaped biases, cross-scale context, and dense local cues, then use Heterogeneous Kernel Re-parameterization to turn the mergeable training branches into one deployment operator.

Method¶

Overall Architecture¶

PKINet-v2 is a feature extraction backbone, not a new detection head replacing the entire detection pipeline. An input remote sensing image passes through four stages, each beginning with overlapping patch embedding for downsampling and channel expansion, followed by repeated PKINet-v2 blocks. Each block contains a PKS residual sub-block incorporating Poly-Kernel Scope and an FFN residual sub-block for channel mixing and feature refinement. The PKS sub-block also contains two surrounding fully connected layers; these surrounding structures and residual paths are outside the five-branch merge performed by HKR. The hierarchical backbone features feed an existing oriented detection framework such as Oriented RCNN, which produces classes and oriented bounding boxes.

For a \(1024\times1024\) input, the four stage resolutions are \(256\times256\), \(128\times128\), \(64\times64\), and \(32\times32\). The Small variant uses 64, 128, 320, and 512 channels, with 2, 2, 4, and 2 blocks respectively. The Tiny variant uses 32, 64, 160, and 256 channels and 3, 3, 5, and 2 blocks; it is not simply a model with fewer blocks at every stage. These configurations come from Table 2 on page 9; its 13.6M parameters and 54.0G FLOPs describe the Small backbone, not the complete detector.

The diagram separates the training-time data path from post-training deployment conversion; dashed edges indicate parameter conversion, not additional supervision or a step performed on every image.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Remote sensing image / stage features"] --> Embed["Overlapping patch embedding"]
    Embed --> PKS["Poly-Kernel Scope (PKS)<br/>Local extraction, heterogeneous aggregation, gating"]
    PKS --> Refine["Residual paths and FFN<br/>Repeated blocks form four-stage features"]
    Refine --> Detector["Oriented detection framework<br/>Classes and oriented boxes"]
    PKS -. "Post-training branch parameter conversion" .-> HKR["Heterogeneous Kernel Re-parameterization (HKR)<br/>Five branches become one deployment convolution"]
    HKR -. "Replace aggregation; retain other structures" .-> Refine

Key Designs¶

1. Poly-Kernel Scope (PKS): shape attention through both geometric bias and sampling density

PKS first applies a \(5\times5\) depth-wise convolution to obtain local features, which are then shared by five parallel branches. Depth-wise convolution processes spatial information separately in each channel without mixing channels in that operation, allowing relatively large spatial kernels at modest cost. The first branch applies a \(1\times19\) axial-strip convolution followed by a \(19\times1\) convolution; the two directions are sequential operations, not two independent parallel branches. This factorization introduces a directional parameterization, but the combined strips cover a two-dimensional region rather than only a horizontal and a vertical line. The second through fourth branches use \(7\times7\), \(5\times5\), and \(3\times3\) square depth-wise convolutions, all with dilation 3. Their spatial spans are 19, 13, and 7, respectively, supplying context at different distances through sparse sampling. The fifth branch uses a dense \(3\times3\) kernel with dilation 1 to retain responses from neighboring pixels. These spans describe each parallel branch relative to its input, not the receptive field of the entire PKS including the preceding \(5\times5\) convolution.

Each branch has its own Batch Normalization (BN), after which responses are added element-wise and fused across channels by a \(1\times1\) convolution to produce a spatial attention map matching the module input shape. The five features are not concatenated along channels, and no softmax selects a single kernel. The resulting attention map multiplies the original PKS input element-wise; it modulates the input features rather than only the locally convolved features. The text describes multiplicative gating without specifying an additional sigmoid at this point, so the map should not be assumed to contain probabilities bounded between 0 and 1. The following notation summarizes the textual relationships accompanying Equations (4)-(5) on page 7 rather than reproducing the corrupted equation formatting in the cache:

\[ A=\operatorname{Conv}_{1\times1}\!\left(\sum_{m=1}^{5}\operatorname{BN}^{(m)}(Z^{(m)})\right),\qquad Y=X\odot A. \]

Here \(X\) is the PKS input, \(Z^{(m)}\) is the response extracted from local features by branch \(m\), \(A\) is the attention map, and \(\odot\) denotes element-wise multiplication. This separates collecting information at different ranges from using it to modulate the original features, allowing distant context to inform a decision without directly replacing local representations. Figure 3 describes kernel coverage as progressively denser toward the center: wide-span branches provide broad coverage, while smaller and dense branches add central contributions. Long geometric structures useful for bridges, contextual layouts useful for fields, and local textures useful for vehicles can therefore influence the output through one aggregation module. Overlapping coverage does not mean the learned weights must be larger, nor does it guarantee strict rotation equivariance; it is an architectural bias. Likewise, full-span coverage refers to a finite kernel extent, not image-wide access in the sense of global self-attention.

2. Heterogeneous Kernel Re-parameterization (HKR): convert linear branches into a contiguous deployment operator

Multiple branches provide useful inductive biases during training, but executing all five directly still requires multiple operator launches and intermediate tensors. HKR does not prune low-contributing branches; it evaluates the same linear aggregation in a different computational form. First, in evaluation mode, the running mean, running variance, scale, and shift of each BN are folded into the corresponding convolution weights and bias. This relies on fixed BN statistics rather than recomputing normalization from the current inference batch. Second, square kernels are center-aligned on a shared \(19\times19\) grid, with dilated weights placed at their sampling offsets and zeros elsewhere. For the sequential axial-strip kernels, a per-channel outer product forms the equivalent two-dimensional kernel before insertion into that grid. Adding aligned weights and corresponding biases yields a single \(19\times19\) depth-wise convolution. These steps correspond to Section 3.4 on pages 8-9 and Figure 3; the essential requirement is that the paths being merged have composable linear or affine structure.

HKR merges the sum of the five branches and their BN layers; the preceding local convolution, following \(1\times1\) fusion, multiplicative gate, residual paths, and FFN remain. In particular, the gate depends on image content, so the entire PKS cannot be treated as one fixed linear convolution. Convolution padding, bias handling, and BN mode must remain consistent with the equivalence transformation; this note does not verify those implementation details by executing code. The paper claims identical outputs after conversion, while the direct experimental evidence in Table 9d is unchanged reported mAP for both T and S. Reducing operator fragmentation primarily benefits hardware execution and memory traffic; it should not be paraphrased as a guarantee that branch merging reduces all FLOPs. The speed difference relative to v1 also includes changes to the overall backbone, and cannot be attributed entirely to HKR.

A Worked Example¶

Consider a \(1024\times1024\) harbor image containing slender ships, compact storage tanks, and small vehicles; this is a mechanism illustration, not an additional experiment. The first stage converts it to a \(256\times256\) feature map with 64 channels in the Small variant. Within a PKS module, the initial \(5\times5\) convolution extracts local structures, and the five branches then read different neighborhoods around the same location. The strip branch supplies long-span directional factorization, sparse square branches add surrounding layout, and the dense branch preserves details near vehicles. BN, summation, and channel fusion turn these responses into attention, which multiplies the input before residual and FFN updates. Later stages further reduce spatial resolution and expand representation capacity, and the oriented detector uses the hierarchical features to produce boxes. Deployment does not first recognize a ship and then select a strip kernel; HKR has already merged the five sets of branch parameters, while content-dependent gating still operates. This example illustrates the feature-processing path, not an experimentally established exclusive assignment of categories to branches.

Loss & Training¶

The contributions concern the backbone and deployment conversion; Section 4.1 introduces no new detection loss, and supervision comes from the connected detection framework. Training begins with ImageNet-1K backbone pretraining, followed by remote sensing detection training; main experiments use 300 pretraining epochs, whereas ablations use 100. Pretraining uses 4 GPUs and a total batch size of 1024; this batch size should not be assumed for downstream detection. Downstream models train in MMRotate for 36 epochs; the text specifies AdamW, an initial learning rate of 0.0001, weight decay of 0.05, and random flipping. DOTA-v1.0 and DOTA-v1.5 images are cropped into \(1024\times1024\) patches with 200-pixel overlap; HRSC2016 and DIOR-R use \(800\times800\) inputs. Main experiments train on trainval and evaluate on test, keeping the same resolution at training and testing without test-time data augmentation. The reported results for the proposed model on HRSC2016 and DIOR-R average 5 runs; this statistical protocol should not automatically be assigned to the other tables. HKR is performed after training and requires neither additional supervision nor a separately retrained detector.

Key Experimental Results¶

Main Results¶

The following main results use 300 epochs of ImageNet-1K pretraining; DOTA uses single-scale training and testing, and the proposed backbone is connected to Oriented RCNN. mAP is reported as a percentage and gains are percentage points; the VOC 2007 and VOC 2012 metrics on HRSC2016 must remain separate.

Dataset and metric	PKINet-v1-S	PKINet-v2-S	Gain	Source
DOTA-v1.0 mAP	78.39	80.46	+2.07	Table 3, page 11
DOTA-v1.5 mAP	71.47	73.57	+2.10	Table 4, page 11
HRSC2016 mAP (VOC 2007)	90.70	90.75	+0.05	Table 5, page 12
HRSC2016 mAP (VOC 2012)	98.54	98.84	+0.30	Table 5, page 12
DIOR-R mAP	67.03	69.40	+2.37	Table 6, page 12

The standard PKINet-v2-S configuration in Table 3 has 30.7M parameters and 173G FLOPs; the separate Strip Head configuration reports 80.68% mAP, 31.4M parameters, and 206G FLOPs, and must not be conflated with it. Table 1 on page 3 reports 14.05 FPS for v1-S and 54.60 FPS for v2-S on a single NVIDIA A100-40G, approximately 3.9 times the throughput; efficiency statistics use \(1024\times1024\) inputs. This is neither a separate speed measurement for HRSC2016 at \(800\times800\) nor a deployment guarantee across all hardware.

Ablation Study¶

Table 9a on page 14 uses DOTA-v1.0, Oriented RCNN, and 100 pretraining epochs; parameters describe the backbone, and FPS is measured after HKR.

Kernel design	Backbone parameters	FPS	mAP (%)
Dense-only (3)	13.2M	54.6	78.57
Strip-only (19)	13.3M	54.6	79.62
Sparse-only (3,5,7)	13.5M	54.6	79.36
Hybrid	13.6M	54.6	80.11

Hybrid kernels improve over strip-only by 0.49 percentage points and dense-only by 1.54 percentage points, supporting complementarity between kernel shapes and receptive scopes. This table does not establish identical execution times before conversion, because the authors explicitly measure speed after HKR. Table 9d on page 14 isolates HKR under the same ablation pretraining setting and single-A100-40G efficiency protocol.

Variant	HKR	FPS	mAP (%)
T	Without	46.2	79.10
T	With	58.0	79.10
S	Without	43.4	80.11
S	With	54.6	80.11

The isolated HKR gain for S is 11.2 FPS, approximately 1.26 times the original speed; it is not the 3.9-times cross-model gain relative to v1. The table shows unchanged mAP, not per-pixel output errors or numerical error bounds under different precision modes.

Key Findings¶

In Table 9b, square-branch dilations (1,3,3,3) perform best at 80.11 mAP, while (1,4,4,4) yields 80.01; a larger span does not guarantee better accuracy.
In Table 9c, 1, 3, 4, and 5 branches produce 79.62, 79.90, 79.99, and 80.11 mAP, respectively, showing incremental benefits beyond a single largest kernel.
In Table 3, bridge AP rises from 55.81 with v1-S to 57.72, but roundabout AP falls from 70.23 to 67.76; aggregate gains do not imply improvement in every category.

Highlights & Insights¶

Training parameterization and the deployment graph can be designed separately. Heterogeneous branches assist learning, while equivalent conversion removes execution fragmentation instead of requiring a single-path training graph.
Receptive fields have more dimensions than size alone. Kernel shape, sampling intervals, and central overlap jointly determine how local detail and context enter a representation.
Backbone innovation complements oriented detection heads. Table 7 tests backbone replacement in multiple detection frameworks, extending the value beyond one specialized head.

Limitations & Future Work¶

Reader assessment: efficiency is primarily measured on A100-40G, without validation on mobile GPUs, CPUs, different batch sizes, or deployment backends, so edge-device gains cannot be inferred directly.
All four datasets are remote sensing detection benchmarks; they do not establish equal benefits for segmentation, natural-image detection, or cross-sensor domain transfer.
Mixing strip kernels does not prove rotation equivariance; controlled analyses of object orientation and background proportion would test the geometric explanation more directly.
The VOC 2007 improvement on HRSC2016 is only 0.05 percentage points; although the result averages 5 runs, the table provides no variance, preventing a statistical-significance conclusion.
Several cached equations have damaged formatting, so this note summarizes only operations supported by the text. The first relative-area bin in Table 8 reads 0.01–0.01, leaving its boundaries uncertain; the area-bin results are therefore not transcribed here.
The paper has no dedicated limitations section; examining mixed-precision HKR errors and cross-hardware throughput would be useful before simply enlarging kernels or adding branches.

Compared with PKINet-v1: the method retains parallel multi-scale kernels while adding heterogeneous geometric biases and changing deployment; the improvement concerns representation and execution, not just a larger model.
Compared with LSKNet: LSKNet emphasizes large kernels and spatial selection, whereas this work combines strips, sparse squares, and dense small kernels; both address context but use different parameterizations.
Compared with Strip RCNN: the method retains strip kernels and supplements them with square and local branches; the 80.68% result with Strip Head also shows that backbone and detection-head designs can be combined.
Research question: with a fixed deployment kernel size, testing whether different training branch sets consistently improve cross-domain generalization is more targeted than merely increasing deployment computation; this is a reader proposal, not an established result of the paper.

Rating¶

Novelty: 4/5. Heterogeneous kernels and re-parameterization have precedents, but the paper organizes them into a targeted remote sensing backbone improvement.
Experimental Thoroughness: 4/5. Four benchmarks, cross-framework tests, and isolated HKR ablations provide substantial coverage, with hardware diversity and error statistics still missing.
Writing Quality: 4/5. The problems and architecture align clearly, but corrupted cached equations and some table labels reduce verifiability.
Value: 4/5. The work offers practical guidance for research requiring both oriented detection accuracy and measured throughput.