This paper proposes a novel low-bitrate animation codec leveraging pose-guided human video generation with on-the-fly training. On the encoder side, the whole sequence is divided into key and non-key frames. Instead of compressing the whole sequences, only the keyframes and pose information are compressed. On the decoder side, the non-key frames are generated using a novel pose-guided human video generation model. The model is trained on-the-fly using keyframes to learn the mapping from pose to full frames.