Self-attention architectures can be seen as graph architectures (imagine the token (node) interactions as adjacency matrix) and are applied to graph problems (Veličković et al., 2018;Li et al., 2019).
in very recent work Koncel-Kedziorski et al.