Neural network models have become a popular machine-learning technique for the toxicity prediction of chemicals. However, due to their complex structure, it is difficult to understand predictions made by these models which limits confidence. Current techniques to tackle this problem such as SHAP or integrated gradients provide insights by attributing importance to the input features of individual compounds. While these methods have produced promising results in some cases, they do not shed light on how representations of compounds are transformed in hidden layers, which constitute how neural networks learn. We present a novel technique to interpret neural networks which identifies chemical substructures in training data found to be responsible for the activation of hidden neurons. For individual test compounds, the importance of hidden neurons is determined, and the associated substructures are leveraged to explain the model prediction. Using structural alerts for mutagenicity from the Derek Nexus expert system as ground truth, we demonstrate the validity of the approach and show that model explanations are competitive with and complementary to explanations obtained from an established feature attribution method.

中文翻译：

---

通过提取学习的化学特征来解释用于毒性预测的神经网络模型

神经网络模型已成为化学品毒性预测的流行机器学习技术。然而，由于其复杂的结构，很难理解这些模型做出的预测，这限制了可信度。目前解决这一问题的技术（例如 SHAP 或积分梯度）通过重视单个化合物的输入特征来提供见解。虽然这些方法在某些情况下产生了有希望的结果，但它们并没有揭示化合物的表示如何在隐藏层中转换，而隐藏层构成了神经网络的学习方式。我们提出了一种解释神经网络的新技术，该技术可以识别训练数据中负责隐藏神经元激活的化学子结构。对于单个测试化合物，隐藏神经元的重要性被确定，并利用相关的子结构来解释模型预测。使用 Derek Nexus 专家系统的致突变性结构警报作为基本事实，我们证明了该方法的有效性，并表明模型解释与从已建立的特征归因方法获得的解释具有竞争性和补充性。