ModErn Text Analysis
META Enumerates Textual Applications
|
Filter that applies an ICU transliteration to each token in the sequence. More...
#include <icu_filter.h>
Public Member Functions | |
icu_filter (std::unique_ptr< token_stream > source, const std::string &id) | |
Constructs an icu_filter which reads tokens from the given source, using a utf::transformer constructed with the specified id. More... | |
icu_filter (const icu_filter &other) | |
Copies an icu_filter. More... | |
void | set_content (const std::string &content) override |
Sets the content for the beginning of the filter chain. More... | |
std::string | next () override |
operator bool () const override | |
Determines whether there are more tokens available in the stream. | |
Public Member Functions inherited from meta::util::multilevel_clonable< Root, Base, Derived > | |
virtual std::unique_ptr< Root > | clone () const |
Clones the given object. More... | |
Static Public Attributes | |
static const std::string | id = "icu" |
Identifier for this filter. | |
Private Member Functions | |
void | next_token () |
Finds the next valid token for this filter. | |
Private Attributes | |
std::unique_ptr< token_stream > | source_ |
The source to read tokens from. | |
utf::transformer | trans_ |
The transformer to use. | |
util::optional< std::string > | token_ |
Current token (if available) | |
Filter that applies an ICU transliteration to each token in the sequence.
meta::analyzers::filters::icu_filter::icu_filter | ( | std::unique_ptr< token_stream > | source, |
const std::string & | id | ||
) |
Constructs an icu_filter which reads tokens from the given source, using a utf::transformer constructed with the specified id.
source | Where to read tokens from |
id | To specify which utf::transformer to use |
meta::analyzers::filters::icu_filter::icu_filter | ( | const icu_filter & | other | ) |
Copies an icu_filter.
other | The other filter to copy from |
|
override |
Sets the content for the beginning of the filter chain.
content | The string content to set |
|
override |