cogrecon.core.data_flexing.spatial_navigation package

Submodules

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics module

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.convert_to_iposition(input_path, output_path, expected_number_of_trials=4)

This function takes a path to a test intermediate file and generates a directory of iPosition formatted files.

Note: The correct coordinates are from log_parser.study_realX, log_parser.study_realY, thus the assumption is made that this will only be run on the VR form of the task (not the 2D form), or that the 2D form data will be converted to the coordinate system of the VR task.

Parameters:

• expected_number_of_trials – the expected number of trials to output (error will throw if number exceeds expectation, but no error will occur if the data is incomplete)
• output_path – the output directory into which to save the iposition files (will be created if doesn’t exist)
• input_path – the path to the test intermediate file

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_basic_exploration_metrics_from_intermediate_files(input_path, output_path)

This function will generate basic exploration metrics given the intermediate files from generate_intermediate_files.

Parameters:

• input_path – the path to the intermediate file
• output_path – the output path to which to save the metrics

Returns:

Nothing

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_basic_exploration_metrics_from_intermediate_files_command_line()

This function can be called from a python script to wrap the generate_basic_exploration_metrics_from_intermediate_files function in command line arguments which will be parsed and passed to the command.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_buffered_points_and_clicks(data_dict, subject_id, trial_num)

This function generates positions, times, clicks, room_by_color and lates items from a data dictionary.

Parameters:

• data_dict – the data dictionary from which to extract data
• subject_id – the subject id to extract
• trial_num – the trial number to extract

Returns:

a tuple containing the x, y, t, click, room_by_color, and latest_item lists for all iterations in the data dictionary

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_intermediate_files(search_path, full_study_path=False, full_study_look=False, full_test_path=False, full_test_look=False, full_practice_path=False, full_practice_look=False, full_test_2d=False, full_test_vr=False, min_num_trials=4)

This script will take a folder containing subject data for the Holodeck Navigation Task and generate CSV files containing the meta-data of interest as requested by the script options. The default is for all data to be generated in a folder adjacent to this script tagged with the current data and time. To speed processing, this can be restricted to particular subsets of the meta-data. If any one option is given to specify a subset of processing, the script will by default, exclude subsets of data not included as options.

Parameters:

• search_path – the path in which to search for files recursively
• full_study_path – if True, study paths will be processed
• full_study_look – if True, study looking will be processed
• full_test_path – if True, test paths will be processed
• full_test_look – if True, test looking will be processed
• full_practice_path – if True, practice paths will be processed
• full_practice_look – if True, practice looking will be processed
• full_test_2d – if True, test 2D will be processed
• full_test_vr – if True, test VR will be processed
• log_level – the logging.level to output
• min_num_trials – the minimum number of trials required for included data

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_intermediate_files_command_line()

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_item_specific_efficiency(data_dict, subject_id, trial_num)

This function generates an error list by item showing the efficiency in getting to particular items.

Parameters:

• data_dict – the data dictionary from which to extract data
• subject_id – the subject id to extract
• trial_num – the trial number to extract

Returns:

a list of dictionaries containing ‘item’ and ‘error’ keys

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_segmentation_analysis(input_path, output_path)

This function will generate a segmentation analysis given the intermediate files from generate_intermediate_files.

Parameters:

• input_path – the path to the intermediate file
• output_path – the output path to which to save the metrics

Returns:

Nothing

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_segmentation_analysis_command_line()

This function can be called from a python script to wrap the generate_segmentation_analysis function in command line arguments which will be parsed and passed to the command.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_study_path_efficiency(input_file)

This function will generate an efficiency_by_item.csv output file which lists the item-by-item efficiency for a log file. The input file is assumed to be an intermediate file from generate_intermediate_files.

Parameters:input_file – an input file to be processed

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_analytics.generate_study_path_efficiency_command_line()

This function can be called from a python script to wrap the generate_study_path_efficiency function in command line arguments which will be parsed and passed to the command.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser module

class cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.FileType

Bases: enum.Enum

This Enum represents the four possible file types.

look_file = 2

path_file = 1

test_file_2d = 3

test_file_vr = 4

class cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.Individual

The Individual object stores and assists with processing of information pertaining to an individual subject.

get_full_file_list()

This helper function is to get the list of any files reference in this individual.

Returns:a list of all files across all trials in this individual (with None files excluded)

meets_trial_number_requirement(required_number_of_trials)

This helper function is to confirm the trial requirements are met.

Parameters:required_number_of_trials – an integer number of trials which are required Returns:if True, all trials meet the requirements

exception cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.LogParseError(*args)

Bases: exceptions.Exception

This Exception object is a specialized exception for use internally.

class cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.SummaryType

Bases: enum.Enum

The summary type enum is for readability and convenience. It differentiates between when a test vs study/practice summary is present to know which schema to use (unfortunately, they use different syntax).

study_practice = 2

test = 1

unknown = 3

class cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.Trial

This class contains information about a Trial, as well as Trial specific helper functions.

all_trial_dates_match()

This function determines if the datetimes across the study, test and practice match.

Returns:True if file datetimes match, false otherwise.

get_full_file_list()

This function returns a full list of the Trial files. None files are automatically removed.

Returns:a list of files in the following order: study_path, study_look, study_summary, test_path, test_look, test_summary, practice_path, practice_look, practice_summary, test_2d, test_vr

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.calculate_euler_vector_from_quaternion(q0, q1, q2, q3)

This function takes in a quaternion and returns an x,y,z euler vector.

Parameters:

• q0 – the first quaternion parameter
• q1 – the second quaternion parameter
• q2 – the third quaternion parameter
• q3 – the fourth quaternion parameter

Returns:

a tuple containing the x,y,z euler angles

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.catalog_files(files, min_num_trials)

This function should produce an Individual list filled according to input restrictions.

Parameters:

• files – a list of files
• min_num_trials – a minimum criteria for number of trials

Returns:

a tuple containing a list of Individual objects, a list of excluded files, and a list of non matching files

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.close_writer(file_pointer)

Helper function for closing a csv writer (this, in combination with the open and parse functions, prevents the external application from needing an import csv reference.

Parameters:file_pointer – a file pointer to be closer Returns:Nothing

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.compare_summary_and_raw_events()

This function will eventually return a bool value which is True if the summary events from read_summary_file and raw events from read_raw_file match.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.compress(pos, orient)

This function compresses an input of positions and orientations, removing repeats to make the data size smaller.

Parameters:

• pos – a list of positions
• orient – a list of orientation angles

Returns:

a tuple containing a list of positions and list of orientations with consecutive duplicates removed

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.context_item_indicies = [[4, 7, 11, 12], [8, 9, 13, 15], [0, 1, 2, 10], [3, 5, 6, 14]]

The following configuration variables have to do with file naming formats and internal saving representations. These values should never be changed unless changes are being made to saving/naming formats. They’re held as constants at the top of this file for convenience in case such changes need to be made, but seriously, don’t change them unless you know when you’re doing.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.datetime_from_dot_net_binary(data)

This function returns a datetime from a .NET binary datetime representation.

From http://stackoverflow.com/questions/15919598/serialize-datetime-as-binary

Parameters:data – the datetime data in .NET binary format Returns:the Python datetime object corresponding to the input data

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.extract_date_time_from_filename_custom(filename)

Helper function which will extract a datetime object given two possible formats of datetime strings (the formats which are used by the Unity program and the 2D test program.

Parameters:filename – the filename (basename) from which to extract the datetime Returns:the Python datetime object associated with the filename contents

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.get_filename_meta_data(fn, path)

This function makes a meta-data object given a file path from the spatial navigation task.

Example input filename: RawLog_Sub999_Trial1_19_22_02_10-04-2017.csv

Parameters:

• fn – the filename (basename)
• path – the full file path

Returns:

a dictionary with ‘fileType’, ‘subID’, ‘trial’, ‘phase’, and ‘datetime’ where ‘fileType’ is either RawLog or SummaryLog, ‘phase’ is either ‘practice’, ‘study’, or ‘test’ and datetime is a datetime object.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.get_final_state_from_summary_events()

This function will eventually return a final item state given the summary events from read_summary_file.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.get_location_by_name(name, test2d=True)

This helper function will get the expected location in either test (2d test) or navigation space.

Parameters:

• name – the name of the item
• test2d – if True, the 2D test items are used

Returns:

the correct location of the item

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.get_object_info_from_string(info_string)

This function returns position, rotation and scaling given an object’s info string.

Parameters:info_string – the object info string Returns:three tuples of floats; position (3), rotation (4), and scaling (3)

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.get_object_info_from_summary_string(summary_info_string)

This function gets name and position given an object’s summary string.

Parameters:summary_info_string – the summary string line for an object Returns:the tuple containing a string name and position (3)

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.get_simple_orientation_path_from_raw_iterations(raw_iterations)

This function returns a list of angles given the iterations from read_raw_file.

Parameters:raw_iterations – the iterations from read_raw_file Returns:a list of angles

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.get_simple_path_from_raw_iterations(raw_iterations, make_2d=True)

This function gets a simple list of location points given raw data iterations.

Parameters:

• raw_iterations – the iterations list from read_raw_file
• make_2d – if True, only X and Z are returned (Y is omitted) as these are the only coordinates which change in the task

Returns:

a list of location points

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.make_output_file(directory, filename, header)

Helper function which will make a csv writer reference to be passed around for file writing.

Parameters:

• directory – the directory to which the output file should be saved
• filename – the filename (basename) of the file to output
• header – the header to write before returning

Returns:

a tuple containing the csv writer and file pointer to the output file

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.nav_get_room_by_location(location)

This helper function will get the room in the vr navigation environment in which an (x,y) point is contained.

Accepts (x,y) coordinates (aka (x,z) in Unity space).

Parameters:location – a tuple containing the (x, y) coordinate whose room should be identified Returns:a tuple containing a room index (in study_context_boundaries) and a context label (from context_labels)

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.nav_time_format_string = '%H_%M_%S_%d-%m-%Y'

These values represent the regular expressions being used to find the three types of files of interest

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.parse_file_and_write(path, subject_id, trial_num, file_type, output_file_writer, summary_file_path)

Helper function which can be used externally to parse any given file type into the appropriate output format then write those rows to the appropriate output file.

Parameters:

• path – the path to the log file
• subject_id – the subject identifier
• trial_num – the trial number
• file_type – the type of file to be parsed
• output_file_writer – the output file writer pointer
• summary_file_path – the path to the summary file

Returns:

Nothing or [] if path is empty

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.parse_look_file(path, subject_id, trial_number, summary_file_path)

A special parser for look files (Raw Unity). It should produce lines with following format: subject_id,trial_number,time,x,y,z,w,euler_x,euler_y,euler_z,room_by_order,room_by_color,items_clicked, distance_from_last_point,time_since_last_point

Parameters:

• path – the file path to the raw data file
• subject_id – the subject identifier
• trial_number – the trial number
• summary_file_path – the file path to the summary data file

Returns:

a list of rows with the following contents: subject_id,trial_number,time,x,y,z,w,euler_x,euler_y,euler_z, room_by_order,room_by_color,items_clicked,distance_from_last_point,time_since_last_point

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.parse_nav_file(path, subject_id, trial_number, summary_file_path, process_as_look=False)

Internal parser for navigation files which can handle either path (process_as_look=False) or look (process_as_look=True) elements. Formerly, this code was broken into two functions (parse_path_file and parse_look_file), but their code were almost identical copies, so they were combined for maintainability.

Parameters:

• path – the file path to the raw data file
• subject_id – the subject identifier
• trial_number – the trial number
• summary_file_path – the file path to the summary data file
• process_as_look – if True, the navigation file will be parsed via looking behavior rather than position

Returns:

a list of rows with the navigation data contents

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.parse_path_file(path, subject_id, trial_number, summary_file_path)

A special parser for path files (Raw Unity). It should produce lines with following format: subject_id,trial_number,time,x,y,z,room_by_order,room_by_color,items_clicked,distance_from_last_point, time_since_last_point

Parameters:

• path – the file path to the raw data file
• subject_id – the subject identifier
• trial_number – the trial number
• summary_file_path – the file path to the summary data file

Returns:

a list of rows with the following contents: subject_id,trial_number,time,x,y,z,room_by_order,room_by_color, items_clicked,distance_from_last_point,time_since_last_point

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.parse_summary_file(path)

This helper function will parse a summary file of either type and return summary type, times, event types, object names (in the format of the summary type), and location (if test type, the placed location, if study/practice type, the location the object was when clicked).

Parameters:path – a string path to a summary file Returns:a tuple containing summary_type, times, event_types, object_types, and locations

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.parse_test_2d_file(path, subject_id, trial_number, summary_file_path)

A special parser for 2d test files (2D test files). It should produce lines with following format: subject_id,trial_number,item_id,x_placed,y_placed,x_expected,y_expected,order_clicked_study,expected_room_by_order, expected_room_by_color,actual_room_by_order,actual_room_by_color

Parameters:

• path – the file path to the raw data file
• subject_id – the subject identifier
• trial_number – the trial number
• summary_file_path – the file path to the summary data file

Returns:

a list of rows with the following contents: subject_id,trial_number,item_id,x_placed,y_placed,x_expected, y_expected,order_clicked_study,expected_room_by_order,expected_room_by_color,actual_room_by_order, actual_room_by_color

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.parse_test_vr_file(path, subject_id, trial_number, summary_file_path)

This is a special parser for vr test files (Summary Unity Test). It should produce lines with following format:

subject_id,trial_number,item_id,x_placed,y_placed,x_expected,y_expected,order_clicked_study,expected_room_by_order, expected_room_by_color,actual_room_by_order,actual_room_by_color,number_of_replacements,time_placed

Parameters:

• path – the file path to the raw data file
• subject_id – the subject identifier
• trial_number – the trial number
• summary_file_path – the file path to the summary data file

Returns:

a list of rows with the following contents: subject_id,trial_number,item_id,x_placed,y_placed,x_expected, y_expected,order_clicked_study,expected_room_by_order,expected_room_by_color,actual_room_by_order, actual_room_by_color,number_of_replacements,time_placed

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.point_is_in_rectangle(point, rectangle)

This helper function will return true if the point given is within the rectangle given (point is (x,y), rectangle is (dictionary ‘x’, ‘y’, ‘w’, ‘h’).

Parameters:

• point – a tuple or list containing an (x,y) point
• rectangle – a dictionary containing ‘x’, ‘y’, ‘w’, and ‘h’ keys associated with the x,y top left corner of a rectangle and the width and height of the rectangle

Returns:

True if the point is within the rectangle, False otherwise

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.quat2euler(q)

This function converts a quaternion into roll, pitch and yaw values

Parameters:q – a quaternion Returns:a tuple containing roll, pitch and yaw

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.read_raw_file(path)

This function reads a raw log file and returns a tuple containing a list of iterations and a list of events. The iterations are dictionaries containing ‘time’ and ‘state’ keys while the events contain ‘time’, ‘eventType’, and ‘objectName’ keys.

Parameters:path – the raw input data file path Returns:a tuple with a list of iterations and list of events

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.read_summary_file(path)

This function takes in a summary file path and returns a list of the events in the summary file (in the form of a dictionary whose members are ‘time’, ‘eventType’, ‘objectName’, and ‘location’.

Parameters:path – the input file path to a summary file Returns:a list of event dictionaries

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.study_context_boundries = [{'y': 37.5, 'x': -12.5, 'w': 40, 'h': 40}, {'y': 37.5, 'x': 27.5, 'w': 40, 'h': 40}, {'y': -2.5, 'x': -12.5, 'w': 40, 'h': 40}, {'y': -2.5, 'x': 27.5, 'w': 40, 'h': 40}]

These values represent the context labels and the indicies (from the other variable lists) of the associated items in the given contexts.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.test2d_get_room_by_location(location)

This helper function will get the room in the 2d test image space in which an (x,y) point is contained.

Parameters:location – a tuple containing the x and y coordinates from the 2d test Returns:a tuple containing a room index (in test_context_boundaries) and a context label (from context_labels)

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.test2d_raw_regex_search_key = '.*\\\\GMDA.*_Raw\\.csv'

These values represent the search string (_search_key) to be used to determine what phase a file is while the _store_key_ and _store_key values represent the internal dictionary representations which are used to store the various file types in the parser dictionary. The search key may need to be adjusted if file naming changes one day, but the store keys should almost never have a reason to be changed.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.test_context_boundries = [{'y': 0, 'x': -375, 'w': 375, 'h': 375}, {'y': 0, 'x': 0, 'w': 375, 'h': 375}, {'y': -375, 'x': -375, 'w': 375, 'h': 375}, {'y': -375, 'x': 0, 'w': 375, 'h': 375}]

These variables represent the configuration of the study items and their contexts. study_labels is the list of all of the labels (in the Unity log files) for each item. study_realX and study_realY comprise the Unity space locations of the objects from study_labels (in order). study_context_boundaries is a list containing the x, y, width, and height of each context (in order defined by context_labels) in the Unity space.

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.test_skip_lines = 124

These values represent the test2d and navigation input file date/time tag format and associated parsing format string (for finding and for reading file date/times)

cogrecon.core.data_flexing.spatial_navigation.spatial_navigation_parser.validate_summary_events_are_complete()

This function will eventually return a bool value which is True if the summary events from read_summary_file are complete.

Module contents