whitespace_tokenizer.h File Reference

#include "analyzers/token_stream.h"
#include "util/clonable.h"

Go to the source code of this file.

Classes
class	meta::analyzers::tokenizers::whitespace_tokenizer
	Converts documents into streams of whitespace delimited tokens. More...

Namespaces
	meta
	The ModErn Text Analysis toolkit is a suite of natural language processing, classification, information retreival, data mining, and other applications of text processing.
	meta::corpus
	Various ways to convert corpus formats into META-readable documents.
	meta::analyzers
	Contains various ways to segment text and deal with preprocessed files (POS tags, parse trees, etc).
	meta::analyzers::tokenizers
	Contains tokenizers that start off a filter chain.

Detailed Description

Author

Chase Geigle

All files in META are released under the MIT license. For more details, consult the file LICENSE in the root of the project.