The millimeter-wave radar has the ability to sense the subtle movement of hand. However, the traditional hand gesture recognition methods are not robust in the scenario with dynamic interference. To address this issue, a robust hand gesture recognition method is proposed based on the self-attention time-series neural networks. Firstly, the original radar echo is constructed in terms of frame, sequence and channel at the input terminal of network.