Contributed equally with the first author. We proposed several speedup strategies for RL based NAS. Quantitatively, in 8 GPU-days our approach discovers a set of architectures performing on-par with stateof-the-art among compact models on the semantic segmentation, pose estimation and depth prediction tasks