Efficient WaveGlow: An Improved WaveGlow Vocoder with Enhanced Speed

Authors: Wei Song, Guanghui Xu, Zhengchen Zhang, Chao Zhang, Xiaodong He, Bowen Zhou

Abstract: Neural vocoder, such as WaveGlow, has become an important component in recent high-quality text-to-speech (TTS) systems. In this paper, we propose Efficient WaveGlow (EWG), which is a flow-based generative model serving as an efficient neural vocoder. Similar to WaveGlow, Efficient WaveGlow has a normalizing flow backbone where each flow step consists of an affine coupling layer and an invertible 1×1 convolution. To reduce the model parameters and enhance the speed without sacrificing the quality of the synthesized speech, Efficient WaveGlow improves WaveGlow in the following aspects. First, the WaveNet-style transformation network in WaveGlow is replaced with an FFTNet-style dilated convolution network. Next, to reduce the computation cost, group convolution is applied to both audio and local condition features. At last, the local condition is shared among the transformation network layers in each coupling layer. As a result, Efficient WaveGlow can reduce the number of floating-point operations (FLOPs) required to generate one-second audio and the number of model parameters both by more than 12 times. Experimental results show that Efficient WaveGlow can reduce real-world inference time cost by more than twice, without any obvious reduction in the quality of the synthesized speech.

Paper link:

Wave Demo

EWG means Efficient WaveGlow SLC means the local condition is between layers in one flow.
EWG G8 means group convolution with 8 group is used both for audio and local condition features.

Encode mel-spectrum by BLSTM

1. "The poorer prisoners were not in abject want, as in other prisons,"

2. "At another time there were fifty, one of whom had been under sentence a couple of years."

3. "No prison dress was allowed; no reception-room was provided, no soap, towels, or baths."

4. "Painters, plasterers, and carpenters were allowed to follow their handicrafts, with the reward of sixpence per diem and a double allowance of food."

5. "but I indulge a hope that many of them will contract with the counties,"

Encode mel-spectrum by Conv1d

1. "The poorer prisoners were not in abject want, as in other prisons,"

2. "At another time there were fifty, one of whom had been under sentence a couple of years."

3. "No prison dress was allowed; no reception-room was provided, no soap, towels, or baths."

4. "Painters, plasterers, and carpenters were allowed to follow their handicrafts, with the reward of sixpence per diem and a double allowance of food."

5. "but I indulge a hope that many of them will contract with the counties,"