ModErn Text Analysis
META Enumerates Textual Applications
|
The multi_analyzer class contains more than one analyzer. More...
#include <multi_analyzer.h>
Public Member Functions | |
multi_analyzer (std::vector< std::unique_ptr< analyzer >> &&toks) | |
Constructs a multi_analyzer from a vector of other analyzers. More... | |
multi_analyzer (const multi_analyzer &other) | |
Copy constructor. More... | |
virtual void | tokenize (corpus::document &doc) override |
Tokenizes a file into a document. More... | |
Public Member Functions inherited from meta::util::multilevel_clonable< Root, Base, Derived > | |
virtual std::unique_ptr< Root > | clone () const |
Clones the given object. More... | |
Private Attributes | |
std::vector< std::unique_ptr< analyzer > > | analyzers_ |
Holds all the analyzers in this multi_analyzer. | |
The multi_analyzer class contains more than one analyzer.
This is useful for trying combined feature methods.
For example, you could tokenize based on ngrams of words and parse tree rewrite rules. The multi_analyzer keeps track of all the features in one set for however many internal analyzers it contains.
meta::analyzers::multi_analyzer::multi_analyzer | ( | std::vector< std::unique_ptr< analyzer >> && | toks | ) |
Constructs a multi_analyzer from a vector of other analyzers.
toks | A vector of analyzers to combine features from |
meta::analyzers::multi_analyzer::multi_analyzer | ( | const multi_analyzer & | other | ) |
Copy constructor.
other | The other multi_analyzer to copy from |
|
overridevirtual |
Tokenizes a file into a document.
doc | The document to store the tokenized information in |