Differentiable neural architecture search delivers point estimation to the optimal architecture, which yields arbitrarily high confidence to the learned architecture. This approach thus suffers in calibration and robustness, in contrast with the maximum a posteriori estimation scheme. In this paper, we propose a novel Variational Neural Architecture Search (VNAS) method that estimates and exploits the weight variability in the following three steps. VNAS first learns the weight distribution through variational inference which minimizes the expected lower bound on the marginal likelihood of architecture using unbiased Monte Carlo gradient estimation. A group of optimal architecture candidates is then drawn according to the learned weight distribution with the complexity constraint. The optimal architecture is further inferred under a novel training-free architecture-performance estimator, designed to score the network architectures at initialization without training, which significantly reduces the computational cost of the optimal architecture estimator. Extensive experiments show that VNAS significantly outperforms the state-of-the-art methods in classification performance and adversarial robustness.

可微神经架构搜索为最佳架构提供点估计，从而为学习的架构提供任意高的置信度。因此，与最大后验估计方案相比，该方法在校准和鲁棒性方面受到影响。在本文中，我们提出了一种新颖的变分神经架构搜索（VNAS）方法，该方法通过以下三个步骤估计和利用权重变异性。 VNAS 首先通过变分推理来学习权重分布，该变分推理使用无偏蒙特卡罗梯度估计来最小化架构边际似然的预期下限。然后根据学习到的权重分布和复杂度约束来绘制一组最佳架构候选者。在一种新颖的免训练架构性能估计器下进一步推断出最佳架构，该估计器旨在在无需训练的情况下在初始化时对网络架构进行评分，这显着降低了最佳架构估计器的计算成本。大量实验表明，VNAS 在分类性能和对抗鲁棒性方面显着优于最先进的方法。