In this project, we propose a general framework to incorporate semantic knowledge into the popular data-driven learning process of word embeddings to improve the quality of them. Under this framework, we represent semantic knowledge as many ordinal ranking inequalities and formulate the learning of semantic word embeddings (SWE) as a constrained optimization problem, where the data-derived objective function is optimized subject to all ordinal knowledge inequality constraints extracted from available knowledge resources such as Thesaurus and Word-Net. We have demonstrated that this constrained optimization problem can be efficiently solved by the stochastic gradient descent (SGD) algorithm, even for a large number of inequality constraints.

Project website:

• Code and resource download at USTC or GitHub site

Reference:

[1] Quan Liu, Hui Jiang, Si Wei, Zhen-Hua Ling and Yu Hu, “Learning SemanticWord Embeddings based on Ordinal Knowledge Constraints,” Proceedings of The 53th Annual Meeting of the Association for Computational Linguistics (ACL 2015), July, 2015. (here)