ModErn Text Analysis
META Enumerates Textual Applications
token_stream.h
Go to the documentation of this file.
1 
10 #ifndef META_TOKEN_STREAM_H_
11 #define META_TOKEN_STREAM_H_
12 
13 #include <memory>
14 #include <string>
15 #include <stdexcept>
16 
17 namespace meta
18 {
19 namespace analyzers
20 {
21 
28 {
29  public:
33  virtual std::string next() = 0;
34 
39  virtual operator bool() const = 0;
40 
45  virtual void set_content(const std::string& content) = 0;
46 
50  virtual ~token_stream() = default;
51 
56  virtual std::unique_ptr<token_stream> clone() const = 0;
57 
61  class token_stream_exception : public std::runtime_error
62  {
63  using std::runtime_error::runtime_error;
64  };
65 };
66 
67 }
68 }
69 #endif
Basic exception class for token stream interactions.
Definition: token_stream.h:61
virtual std::unique_ptr< token_stream > clone() const =0
Clones the given token stream.
The ModErn Text Analysis toolkit is a suite of natural language processing, classification, information retreival, data mining, and other applications of text processing.
Definition: analyzer.h:24
virtual void set_content(const std::string &content)=0
Sets the content for the stream.
virtual ~token_stream()=default
Destructor.
virtual std::string next()=0
Obtains the next token in the sequence.
Base class that represents a stream of tokens that have been extracted from a document.
Definition: token_stream.h:27