An interface for writing and merging inverted chunks of postings_data for a disk_index. More...

Classes
class	chunk_handler_exception
	Simple exception class for chunk_handler interactions. More...

class	producer
	The object that is fed postings_data by the index. More...

Public Types
using	index_pdata_type = typename Index::index_pdata_type

using	primary_key_type = typename index_pdata_type::primary_key_type

using	secondary_key_type = typename index_pdata_type::secondary_key_type

using	chunk_t = chunk< primary_key_type, secondary_key_type >

Public Member Functions
	chunk_handler (const std::string &prefix)
	Constructs a chunk_handler that writes to the given prefix. More...

producer	make_producer ()
	Creates a producer for this chunk_handler. More...

uint32_t	size () const

uint64_t	final_size () const

void	merge_chunks ()
	Merge the remaining on-disk chunks.

uint64_t	unique_primary_keys () const

Private Member Functions
void	write_chunk (std::vector< index_pdata_type > &pdata)

Private Attributes
std::string	prefix_
	The prefix for all chunks to be written.

std::atomic< uint32_t >	chunk_num_ {0}
	The current chunk number.

std::priority_queue< chunk_t >	chunks_
	Queue of chunks on disk that need to be merged */.

std::mutex	mutables_
	Mutex used for protecting the chunk queue.

util::optional< uint64_t >	unique_primary_keys_
	Number of unique primary keys encountered while merging.

Detailed Description

An interface for writing and merging inverted chunks of postings_data for a disk_index.

Constructor & Destructor Documentation

template<class Index >

meta::index::chunk_handler< Index >::chunk_handler ( const std::string & prefix )

Constructs a chunk_handler that writes to the given prefix.

Parameters

prefix The prefix for all chunks to be written

template<class Index >

auto meta::index::chunk_handler< Index >::make_producer ( )

Creates a producer for this chunk_handler.

Producers are designed to be thread-local buffers of chunks that write to disk when their buffer is full.

template<class Index >

uint32_t meta::index::chunk_handler< Index >::size ( ) const

template<class Index >

uint64_t meta::index::chunk_handler< Index >::final_size ( ) const

Returns: the size, in bytes, of the last chunk written to disk after merging.

template<class Index >

uint64_t meta::index::chunk_handler< Index >::unique_primary_keys ( ) const

template<class Index >

void meta::index::chunk_handler< Index >::write_chunk ( std::vector< index_pdata_type > & pdata )

private

Parameters

pdata The collection of postings_data objects to combine into a chunk

The documentation for this class was generated from the following files: