Abstract: Attention mechanism has become the backbone of machine learning applications, expanding beyond natural language processing into domains, such as computer vision and recommendation systems.