Abstract The goal of this work is to propose a real-time accompaniment system to assist singers in complete a simple demo by themselves. By current audio signal technology, we have seen some achievements on accompani-ment generation and some on vocal transcription. Basing on these great works, we propose a novel “Real-time accompany generation system” to combine current state-of-arts and further improve the efficiency to reach real-time human interactive mode. To reach a ac-ceptable computing efficiency, we do a lot pruning on original model and apply DenseNet concept to enhance its gradient propagate. In this system, we integrate efficiency improved vocal transcription model and sim-plified HMM-base accompaniment generation model which can better fit small training set situation to output musical accompaniment in limited time. We believe that this work will benefit many solo singers to deliver their live shows or demos by themselves whenever they need. As a result, we reach real-time under 180 BPM which covers most of pop music and propose a highly improved vocal transcription model with 1/1000 parameters and 1/50 FLOPs.