Because of small amount of causal pairs, we simply build an independent final linear layer apart from the original linear layer in Figure 2.
there are two final linear layers: only one of them is active when training temporal or causal pairs.