Then the algorithm iteratively applies the following three steps for a specified number of epochs: (1) Learning custom embeddings: The algorithm learns custom embeddings for all entities and patterns observed in the dataset, using the current entPool c s. We train our embeddings for both entities and patterns by maximizing the objective function J: where the individual components of the objective function designed to model both the unsupervised, language model part of the task as well as the light supervision coming from the seed examples.
it is important for unlabeled data to be used for regularization to be able to learn good abstractions in the different layers.