Skip to content

chat.infer 并行生成异常 #907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TwilightShd opened this issue Feb 28, 2025 · 3 comments
Open

chat.infer 并行生成异常 #907

TwilightShd opened this issue Feb 28, 2025 · 3 comments
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed

Comments

@TwilightShd
Copy link

问题:
更新到最新版本后,当textx是个list时chat.infer(texts)的并行推理似乎有问题:运行以下代码时,只会生成一条音频,内容是两句话拼起来,显存也是生成一条音频的水平。
以前textx是个list时会生成多条语音,显存占用也是跟texts长度成正比的

代码:
import ChatTTS
import torch
import torchaudio

chat = ChatTTS.Chat()
chat.load(compile=False) # Set to True for better performance

texts = ["PUT YOUR 1st TEXT HERE", "PUT YOUR 2nd TEXT HERE"]

wavs = chat.infer(texts)

for i in range(len(wavs)):
"""
In some versions of torchaudio, the first line works but in other versions, so does the second line.
"""
try:
torchaudio.save(f"basic_output{i}.wav", torch.from_numpy(wavs[i]).unsqueeze(0), 24000)
except:
torchaudio.save(f"basic_output{i}.wav", torch.from_numpy(wavs[i]), 24000)

@TwilightShd
Copy link
Author

将chat.infer(texts)改为chat.infer(texts,split_text=False)能够解决问题,也许split_text不应该被默认为True?

@TwilightShd
Copy link
Author

还有一点不太懂,我理解的433-451行作用是指定一种音色,保持全程一致。
但句子短的时候(比如少于5个字),指定音色似乎会显著影响生成质量,比如吞字等(短句子后已经加了[uv_break][uv_break]),而如果在451行后手动把spk_smp和txt_smp又置成none,生成出来的句子会明显好很多,也不吞字(虽然音色显然不一致),句子较长时不会产生上述现象。
按理说指定的音色也是随机出来的,不知道具体机理是什么

@fumiama
Copy link
Member

fumiama commented Mar 12, 2025

给infer传入一个list时,位于不同index的说话音色是有区别的,所以使用zero shot的办法进行抑制,同时增大并行度。短句子应该是模型自身问题,目前只有尽量避免了。

@fumiama fumiama added documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed labels Mar 12, 2025
@fumiama fumiama added this to ChatTTS Mar 12, 2025
@fumiama fumiama moved this to Todo in ChatTTS Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed
Projects
Status: Todo
Development

No branches or pull requests

2 participants