Can pyramid-flow utilize kv-cache to reduce computation?

[Edwardmark]

It seems that the computation of the history condition latent is calculated in each frame prediction, but I think it can use kv-cache to reduce this redundant computation, is that true?

[feifeiobama]

yes, but due to the complex temporal compression designs, im afraid that kv cache wouldnt save much computation here

[Edwardmark]

yes, but due to the complex temporal compression designs, im afraid that kv cache wouldnt save much computation here

thanks, so the model will run slower and slower for the latter frame for more and more condition frames is used, is that true?

[feifeiobama]

yes unless you apply some methods to truncate history condition, such as sliding window.

[Edwardmark]

yes, but due to the complex temporal compression designs, im afraid that kv cache wouldnt save much computation here

could you please explain in detail why kv cache wouldn't save much computation?I am confused about that.

[zhichengsun]

if you compress history context aggressively, then most of the compute is spent on self-attention among new frame tokens, instead of cross-attention between new tokens and history tokens. kv cache can only reduce the latter part of compute.