Hardware Trojan detection accuracy depends on test data choices
Protocol effects on feature-based hardware-Trojan detection across Trust-Hub families
Machine LearningArtificial IntelligenceComputational Engineering, Finance, and ScienceDatabasesComputer Science and Game Theory
Summary
Detecting hidden malicious components called hardware Trojans in computer chips is important for security. This paper shows that how researchers split their test data can greatly affect the reported accuracy of Trojan detectors. The authors find that if test data includes parts very similar to training data, detection seems much better than if whole families of similar chip designs are excluded from testing. They suggest that evaluations should carefully separate data from different chip families to get a true measure of how well detectors work.
Hardware TrojanDetectionMachine learningRandom forestXGBoostF1 scoreAverage precisionNetlistBenchmarkTrust-Hub
Authors
Hang Xiao, Chuhong Xu, Kainan Zhou, Gangzhen Qian, Lu Yi
Abstract
Trust-Hub reuses host circuits: several files differ mainly in the inserted Trojan. When gates from sibling variants enter both training and test folds, a detector can benefit from host logic it has already seen. We measure that effect instead of proposing another classifier. The corpus contains 49,124 gates from 16 netlists grouped into five host families. We left the parser, 36 gate features, class weighting, model settings, threshold, and family-level aggregation unchanged and altered one choice: the test boundary. The three settings draw test gates from the pooled corpus, withhold a complete netlist, or withhold every variant of one host. The choice matters. Random forest records F1/AP of 0.914/0.978 with pooled gates, 0.636/0.851 with one netlist held out, and 0.460/0.577 with a host family held out. XGBoost falls from 0.946/0.976 to 0.464/0.544 across the same comparison. Logistic regression loses AP, although its fixed-threshold F1 is not monotonic. Each family shows the same pooled-to-family direction. Feature removal, repeated model and simulator seeds, score normalization, parser-related exclusions, and a smaller sample change the size of the gap without reversing it. Aggregation also matters: a gate-weighted average is dominated by the larger ISCAS files, so the headline values give each host family one vote. Bootstrap and jackknife summaries keep the gap positive, but their folds reuse training families. We treat the five family rows as descriptive evidence rather than independent trials. Five host families are too few for a population claim, and the experiment says nothing about transfer to a new cell library or an industrial design. It supports a narrower conclusion: sibling benchmark variants can inflate apparent transfer. Benchmarks with several variants of one host circuit should report family-aware holdouts and all five family results beside pooled scores.