Input and output

Reading data

The paraatm.io module provides functions for reading from common data formats. Currently, this includes the Integrated Flight Format (IFF) and GNATS simulation output files.

paraatm.io.iff.read_iff_file(filename, record_types=3, callsigns=None, chunksize=50000, encoding='latin-1')

Read IFF file and return data frames for requested record types

From IFF 2.15 specification, record types include:

  1. header
  2. track point
  3. flight plan
  4. data source program
  5. sectorization
  6. minimum safe altitude
  7. flight progress
  8. aircraft state
Parameters:
  • filename (str) – File to read
  • record_types (int, sequence of ints, or 'all') – Record types to return
  • callsigns (None, string, or list of strings) – If None, return records for all aircraft callsigns. Otherwise, only return records that match the given callsign (in the case of a single string) or match one of the specified callsigns (in the case of a list of strings).
  • chunksize (int) – Number of rows that are read at a time by pd.read_csv. This limits memory usage when working with large files, as we can extract out the desired rows from each chunk, isntead of reading everything into one large DataFrame and then taking a subset.
  • encoding (str) – Encoding argument passed on to open and pd.read_csv. Using ‘latin-1’ instead of the default will suppress errors that might otherwise occur with minor data corruption. See http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html
Returns:

If record_types is a scalar, return a DataFrame containing the data for that record type only. Otherwise, return a dictionary mapping each requested record type to a corresponding DataFrame.
Return type:

DataFrame or dict of DataFrames
paraatm.io.gnats.read_gnats_output_file(filename)

Read the specified GNATS output file

Parameters:filename (str) – Input file to read
Returns:
Return type:Single formatted data frame

Utilities

General-purpose utility functions are provided in the paraatm.io.utils module. read_data_file() will try and automatically detect the file type. write_csv_file() writes a DataFrame back to a CSV format file that can be subsequently read back in.

paraatm.io.utils.read_data_file(filename, *args, **kwargs)

Detect file format and read scenario data

Supports GNATS and IFF format

Parameters:filename (str) –
paraatm.io.utils.write_csv_file(df, filename)

Write a DataFrame to csv

Parameters:
  • df (DataFrame) –
  • filename (str) –
paraatm.io.utils.read_csv_file(filename)

Read a CSV file previously created by PARA_ATM

Example: parse large IFF file

The following example illustrates how read_iff_file() can be used to parse select values from a very large IFF file and store the results to a new file. This way, the extracted results can be re-used without the need to re-read the large file, which may be time consuming.

from paraatm import io

# Read track point data (record 3) for callsign 'ABC123':
df = io.iff.read_iff_file('huge_data_file.iff', record_types=3, callsigns='ABC123')

# Write extracted data to new file in CSV format:
io.utils.write_csv_file(df, 'extracted_data.csv')

# Read extracted data back:
io.utils.read_data_file('extracted_data.csv')