|
ModErn Text Analysis
META Enumerates Textual Applications
|
Tokenizes parse trees by only tokenizing the tree structure itself. More...
#include <skeleton_featurizer.h>
Public Member Functions | |
| void | tree_tokenize (corpus::document &doc, const parser::parse_tree &tree) const override |
| Ignores node labels and only tokenizes the tree structure. More... | |
Public Member Functions inherited from meta::util::multilevel_clonable< Root, Base, Derived > | |
| virtual std::unique_ptr< Root > | clone () const |
| Clones the given object. More... | |
Static Public Attributes | |
| static const std::string | id = "skel" |
| Identifier for this featurizer. | |
Tokenizes parse trees by only tokenizing the tree structure itself.
|
override |
Ignores node labels and only tokenizes the tree structure.
| doc | The document to parse |
| tree | The current parse_tree in the document |
1.8.9.1