W2C - Web To Corpus

Changes RSS Code RSS Complete


Download

Author

Code

Lines of Code

Examples

for i in `langList.sh -w 1000000 | cut -f1`; do 
	wikiMiniCorpora.sh -c 100 $i; 
done

From each wikipedia with at least 100000 articles download 100 pages and creates corpus.


for i in `langList.sh -w 1000000 | cut -f1`; do 
	wikiCorpora.sh $i; 
done

From each wikipedia with at least 100000 creates corpus.