Module parquet_writer 

Structs

ParquetDataRow

Represents a row of data in the Parquet file format. Fields used depend on the measurement type and configuration (unused fields stay None).

Constants

MAX_ROW_GROUP_ROW_COUNT 🔒

ROW_BUFFER_CAPACITY 🔒

Functions

build_parquet_schema

Creates a parquet data schema from the headers based on the measurement type and configuration.

get_parquet_header

Returns the fixed superset of columns for a measurement type. The session column is only included for feed measurements with sessions enabled.

get_parquet_metadata

Returns a vector of key-value pairs containing the metadata of the measurement.

measurement_reply_to_parquet_row 🔒

Converts a MeasurementReply into a ParquetDataRow for writing to a Parquet file.

trace_reply_to_parquet_row 🔒

Converts a TraceReply into a ParquetDataRow for writing to a Parquet file.

write_batch_to_parquet

Writes a batch of ParquetDataRow to the Parquet file using the provided writer. The batch is sorted by reply source address for better compression.

write_results_parquet

Write results to a Parquet file as they are received from the channel. This function processes the results in batches to optimize writing performance.