ModErn Text Analysis
META Enumerates Textual Applications
compressed_file_writer.h
Go to the documentation of this file.
1 
10 #ifndef META_COMPRESSED_FILE_WRITER_H_
11 #define META_COMPRESSED_FILE_WRITER_H_
12 
13 #include <functional>
14 #include <stdexcept>
15 #include <string>
16 
17 namespace meta
18 {
19 namespace io
20 {
21 
26 {
27  public:
36  compressed_file_writer(const std::string& filename,
37  std::function<uint64_t(uint64_t)> mapping);
38 
43 
48  uint64_t bit_location() const;
49 
54  void write(uint64_t value);
55 
60  void write(const std::string& str);
61 
65  void close();
66 
67  private:
73  void write_bit(bool bit);
74 
78  void write_buffer() const;
79 
81  FILE* outfile_;
82 
84  uint64_t char_cursor_;
85 
87  uint64_t bit_cursor_;
88 
90  uint64_t buffer_size_;
91 
93  unsigned char* buffer_;
94 
96  std::function<uint64_t(uint64_t)> mapping_;
97 
99  uint64_t bit_location_;
100 
102  bool closed_;
103 
104  public:
108  class compressed_file_writer_exception : public std::runtime_error
109  {
110  public:
111  using std::runtime_error::runtime_error;
112  };
113 };
114 
120 uint64_t default_compression_writer_func(uint64_t key);
121 }
122 }
123 
124 #endif
void write(uint64_t value)
Writes a value to the end of the compressed file.
Definition: compressed_file_writer.cpp:72
uint64_t bit_location_
The number of total bits that have been written (for seeking)
Definition: compressed_file_writer.h:99
uint64_t char_cursor_
The current byte this reader is on.
Definition: compressed_file_writer.h:84
void write_bit(bool bit)
Writes a bit to the file and advances the current location in the file.
Definition: compressed_file_writer.cpp:86
uint64_t default_compression_writer_func(uint64_t key)
Shows how to convert a number into its compressed form.
Definition: compressed_file_writer.cpp:111
compressed_file_writer(const std::string &filename, std::function< uint64_t(uint64_t)> mapping)
Constructor; Opens a compressed file for writing or creates a new file if it doesn't exist...
Definition: compressed_file_writer.cpp:16
void close()
Closes this compressed file.
Definition: compressed_file_writer.cpp:59
std::function< uint64_t(uint64_t)> mapping_
The mapping to use (actual -> compressed id)
Definition: compressed_file_writer.h:96
uint64_t bit_location() const
Definition: compressed_file_writer.cpp:48
uint64_t buffer_size_
How large to make the internal writer buffer.
Definition: compressed_file_writer.h:90
unsigned char * buffer_
Saved data that is not yet written to disk.
Definition: compressed_file_writer.h:93
The ModErn Text Analysis toolkit is a suite of natural language processing, classification, information retreival, data mining, and other applications of text processing.
Definition: analyzer.h:24
FILE * outfile_
Where to write the compressed data.
Definition: compressed_file_writer.h:81
~compressed_file_writer()
Destructor; closes the compressed file.
Definition: compressed_file_writer.cpp:53
bool closed_
Ensures the file isn't closed more than once.
Definition: compressed_file_writer.h:102
uint64_t bit_cursor_
The current bit of the current byte this reader is on.
Definition: compressed_file_writer.h:87
Writes to a file of unsigned integers using gamma compression.
Definition: compressed_file_writer.h:25
Basic exception for compressed_file_writer interactions.
Definition: compressed_file_writer.h:108
void write_buffer() const
Writes the buffer to the file.
Definition: compressed_file_writer.cpp:104
std::string filename(const std::string &path)
Definition: unit_test.h:114