02 / Master’s thesis · Queen Mary University of London
Continuously progressive
neural architecture search
ECoProNAS combines P-DARTS’s progressive method with XNAS’s continuous elimination and EG-based selection mechanism.
Enable JavaScript to inspect recorded CIFAR-10 architecture search.
A later, smaller CIFAR-10 run using my original model illustrates how the search grows. The thesis results are described below.
Differentiable architecture search
Most research on improved NAS reduces the task to discovering one “cell” architecture that is replicated several times to form a deep network. DARTS is a technique that uses automatic differentiation directly on architecture weights rather than Evolutionary Algorithms or Reinforcement Learning-based approaches.
ECoProNAS
In P-DARTS, Chen et al. indicate that searching on 7–8 cells but assessing on 14–20 cells results in a depth gap. ECoProNAS combines P-DARTS’s progressive method with XNAS’s continuous elimination and EG-based selection mechanism to accelerate search and allow for more target-specific search. As candidate operations are pruned, the network grows deeper.
An error rate of 2.62% was achieved on CIFAR-10, approximately matching P-DARTS accuracy. The direct search on ImageNet32 delivers an error rate of 39.57%, 0.62% better than P-DARTS’ 40.19%. The architectures were also tested inside a show-and-tell-like image captioning model with an LSTM and cross-attention, where the new technique matched or outperformed the P-DARTS model.
The recorded search
The replay reads architecture weights, losses, accuracies and growth events from recorded CIFAR-10 search. Line thickness follows each operation’s learned mixing weight.
This later demonstration is separate from my master’s thesis evaluation. It uses 2,500 CIFAR-10 images, split equally between fitting and held-out architecture updates. Each cell learns its own operation mixture. Each architecture epoch can prune up to three weak operations per cell. Freed parameter capacity funds up to two new cells, followed by four recovery epochs. This eight-channel model reached 20 cells after 56 epochs, then completed recovery, architecture refinement and final training. Loss and accuracy share a time axis and use separate vertical scales; every point is measured.
Up next / Doctoral research