Vessel classifier reliance on hull parts measured with new swap method
HullProbe: Measuring Interventional Part Reliance in Maritime Vessel Classifiers
Computer Vision and Pattern RecognitionMachine Learning
Summary
Figuring out what kind of ship is in a picture can be tricky because computers might just look at the plain hull instead of the ship’s more unique upper parts. The authors created HullProbe, a method that swaps parts of ships in images to see which parts really affect computer guesses. This lets them measure how much the hull influences the decision directly, instead of guessing from overall errors. They found that many ship classifiers still rely heavily on the generic hull, even after attempts to reduce this bias.
What this means in practice
- •For maritime security teams: Assess ship type classifiers to identify if decisions rely improperly on common hull features prone to spoofing or alteration.
- •For vehicle recognition developers: Improve evaluation of fine-grained vehicle classifiers by measuring specific part reliance using HullProbe’s swap and transform method.
Authors
Faisal Mohammad, Mohamed Younis
Abstract
Recognizing a vessel's type from an image supports maritime surveillance and sanctions monitoring, but a classifier can reach high accuracy by leaning on the generic hull instead of the more distinctive superstructure, and is then fooled when a vessel is re-hulled or occluded. De-biasing methods claim to remove such reliance, yet the field certifies them with worst-group accuracy, a proxy that needs group labels and only infers reliance from accuracy. We measure the reliance directly. HullProbe swaps a registered anatomical part of a vessel for another instance's and reads how much that part drives the decision, using a same-class swap to control for compositing artifacts. Because a surface vessel is a hull and a superstructure arranged along one axis, its parts form a 2x3 grid whose 2^6 = 64 swaps we evaluate exhaustively, and a single Walsh transform returns each part's interventional influence and its interaction spectrum. On planted shortcuts, where the true reliance is known, the measure tracks worst-group accuracy (r = -0.94 over 45 configurations), and replacing its learned segmenter with human annotations changes the hull-share by less than the audit's resolution. On the 26 vessel types of MARVEL, the classifier places about 46% of its corrected sensitivity on the generic hull yet follows the superstructure when the two conflict, and the group-label-free de-biasers available there do not detectably reduce the hull's share. The findings hold across three architectures and a second maritime dataset, and the method extends to fine-grained vehicle recognition on Stanford Cars.