hyperzzw's comments

On: I unified convolution and attention into a single framework

[hyperzzw]

Hi, I have read your interesting paper. I recommend you our previous HyperZZW paper (https://arxiv.org/pdf/2401.17948). I think there are a lot of similar concepts here.

1. Context-dependent convolution

2. Global & Local branches

3. Replace large-filter Conv with matrix multiplication

4. Information bottleneck -> Information loss

I also want to share that Mamba is based on the concept of Hyena. And the simplicity is the best (HyperZZW), and Hyena is a failure.

[umjunsik132 → hyperzzw]

Thank you for your comment and for sharing your interesting work. I'll take a look.