If the length of d is 1, all the weights in W d have value 1, and γ is a linear activation, then this formula is equivalent to a regular cosine similarity.
we use a larger length for d to capture more features, use tanh as the activation function, and optimise the weights of W d during training, giving the framework more flexibility to customise the model for the task of metaphor detection.