For textual networks, when the embedding objective is network topology reconstruction, a natural choice is to treat the text as context (Tu et al., 2017).
for (i), if the sequence embeddings used do not explicitly address the syntactic structure of the text, a relatively dense attention score matrix can be expected.