ModErn Text Analysis
META Enumerates Textual Applications
empty_sentence_filter.h
Go to the documentation of this file.
1 
9 #ifndef META_EMPTY_SENTENCE_FILTER_H_
10 #define META_EMPTY_SENTENCE_FILTER_H_
11 
12 #include "analyzers/token_stream.h"
13 #include "util/clonable.h"
14 #include "util/optional.h"
15 
16 namespace meta
17 {
18 namespace analyzers
19 {
20 namespace filters
21 {
28  : public util::clonable<token_stream, empty_sentence_filter>
29 {
30  public:
36  empty_sentence_filter(std::unique_ptr<token_stream> source);
37 
43 
48  void set_content(const std::string& content) override;
49 
53  std::string next() override;
54 
58  operator bool() const override;
59 
61  const static std::string id;
62 
63  private:
67  void next_token();
68 
70  std::unique_ptr<token_stream> source_;
71 
74 
77 };
78 }
79 }
80 }
81 #endif
static const std::string id
Identifier for this filter.
Definition: empty_sentence_filter.h:61
util::optional< std::string > first_
Keeps track of the left hand side of a potentially empty sentence.
Definition: empty_sentence_filter.h:73
empty_sentence_filter(std::unique_ptr< token_stream > source)
Constructs an empty_sentence_filter which reads tokens from the given source.
Definition: empty_sentence_filter.cpp:17
Template class to facilitate polymorphic cloning.
Definition: clonable.h:28
std::string next() override
Obtains the next token in the sequence.
Definition: empty_sentence_filter.cpp:60
The ModErn Text Analysis toolkit is a suite of natural language processing, classification, information retreival, data mining, and other applications of text processing.
Definition: analyzer.h:24
Filter that removes any empty sentences from the token stream.
Definition: empty_sentence_filter.h:27
util::optional< std::string > second_
Keeps track of the right hand side of a potentially empty sentence.
Definition: empty_sentence_filter.h:76
void set_content(const std::string &content) override
Sets the content for the beginning of the filter chain.
Definition: empty_sentence_filter.cpp:32
std::unique_ptr< token_stream > source_
The source to read tokens from.
Definition: empty_sentence_filter.h:70
void next_token()
Finds the next valid token for this filter.
Definition: empty_sentence_filter.cpp:39