vane
/
word2vec
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Automatically exported from code.google.com/p/word2vec
28 Commits 2 Branches 0 Tags 159 KiB
C 84.4%
Shell 14.4%
Makefile 1.2%
Go to file
tmikolov ada9ca3db6 removed some specific information from README, as I just changed the scritps 2013-08-01 20:12:56 +00:00
LICENSE aa 2013-07-30 05:35:50 +00:00
README.txt removed some specific information from README, as I just changed the scritps 2013-08-01 20:12:56 +00:00
compute-accuracy.c aa 2013-08-01 19:36:25 +00:00
demo-analogy.sh aa 2013-08-01 19:28:23 +00:00
demo-classes.sh aa 2013-08-01 19:28:42 +00:00
demo-phrase-accuracy.sh aa 2013-08-01 19:28:55 +00:00
demo-phrases.sh aa 2013-08-01 19:29:23 +00:00
demo-word-accuracy.sh aa 2013-08-01 19:29:38 +00:00
demo-word.sh aa 2013-08-01 19:30:05 +00:00
distance.c removed gets() function 2013-07-30 20:05:43 +00:00
makefile aa 2013-08-01 19:31:03 +00:00
questions-phrases.txt aa 2013-08-01 19:31:17 +00:00
questions-words.txt aa 2013-08-01 19:31:32 +00:00
word-analogy.c demo code for comupting analogies with the word vectors 2013-07-31 23:00:30 +00:00
word2phrase.c aa 2013-07-30 05:36:45 +00:00
word2vec.c aa 2013-08-01 19:33:54 +00:00

README.txt 

Tools for computing distributed representtion of words
------------------------------------------------------

We provide an implementation of the Continuous Bag-of-Words (CBOW) and the Skip-gram model (SG), as well as several demo scripts.

Given a text corpus, the word2vec tool learns a vector for every word in the vocabulary using the Continuous
Bag-of-Words or the Skip-Gram neural network architectures. The user should to specify the following:
 - desired vector dimensionality
 - the size of the context window for either the Skip-Gram or the Continuous Bag-of-Words model
 - training algorithm: hierarchical softmax and / or negative sampling
 - threshold for downsampling the frequent words 
 - number of threads to use
 - the format of the output word vector file (text or binary)

Usually, the other hyper-parameters such as the learning rate do not need to be tuned for different training sets. 

The script demo-word.sh downloads a small (100MB) text corpus from the web, and trains a small word vector model. After the training
is finished, the user can interactively explore the similarity of the words.

More information about the scripts is provided at https://code.google.com/p/word2vec/