Extract Value from Array
Extract only selected values of interest from a column of type Array, consisting of multiple entries and possibly nested Arrays, into new columns.
Arrays in Visual Notebooks can contain the following combinations of data types:
- Strings, numeric (i.e., double, integer, long), timestamp and Boolean (Note: all data is converted to string)
- Multiple arrays
If an Array consists entirely of numeric values and has only one level (i.e., it is one-dimensional), it is labeled as type Vector.
Data analysis often involves a large number of concurrent mathematical operations on structured data (i.e., stored in a database and indexable by row and column). Matrix algebra is most commonly used to perform these operations efficiently. Arrays are equivalent to row vectors, and they complement the efficient column operations built into Visual Notebooks. Several transformations are provided for manipulating Arrays, including creating Arrays, extracting values from Arrays and disassembling Arrays.
Configuration
| Field | Description |
|---|---|
Name default=none | Name of the node: A user-specified node name, displayed in the canvas and in the dataframe as a tab. |
| Select Array Column to Extract Required | Select single column of type Array: Select the input Array column from which to extract a specific field. |
| Position to Extract Required | Element to extract from the Array column: Enter a non-zero integer corresponding to the index of the entry to extract. The index of the first entry is "1". Indexing can also be done starting with the last entry, which has position "-1". |
Output Column Name default=Default | Desired name for the output column: Select "Default" if you want the column name to be of the format "_" where n ≥ 1 is an integer that increments by one for each column. Select "Custom" to enter a custom column name. |
Drop Original Column(s) default=On | Toggle indicating whether the selected input column(s) should be dropped from the output: Leave the switch on to delete the Array column(s) from which output columns are extracted. Toggle the switch off to keep the selected column(s). |
Node Inputs/Outputs
| Input | A Visual Notebooks dataframe |
|---|---|
| Output | A dataframe with extracted values |
Figure 1: Example dataframe output
Examples
The data shown in Figure 2 is used in the following examples. It contains operating parameters for a number of gas turbine powered generators, including temperatures, pressures, speed, and output power components. We would like to extract the Combustion Temperature and Output Current from a single nested array containing all of the parameters.
Figure 2: Example input data
To create the nested array shown in Figure 2 from the linked file, complete Example 2 - Nested Arrays in the documentation for the Assemble Array node. Then, follow the steps below to extract the Combustion Temperature and Output Current:
- Connect an Extract Value from Array node to an existing node. In this case, it should be connected to an Assemble Array node called "Parameters".
- Enter "Extract Temps and Power" for Name.
- Select "Parameters" for Select Array Column to Extract and "1" for Position to Extract. Toggle "Custom" for Output Column Name and enter "Temps" for the column name.
- Click + Additional Value to specify another value to extract.
- Select "Parameters" for Select Array Column to Extract and "4" for Position to Extract. Toggle "Custom" for Output Column Name and enter "Power_Comp" for the column name.
- Toggle Drop Original Column off and click Run.
After running the node, the temperatures and power components are extracted as separate columns, which are shown in Figure 3. Note that only a single level in the nested Array is extracted. Given the input is a nested Array of numerical values, the output columns are Vectors.
Figure 3: Extracting temperature and power component Vectors from a nested Array
We now extract the Combustion Temperature and Output Current directly from these Vectors:
- Connect another Extract Value from Array node to the previous one.
- Enter "Extract CT and OC" for Name.
- Select "Temps" for Select Array Column to Extract and "2" for Position to Extract. Toggle "Custom" for Output Column Name and enter "Comb_Temp" for the column name.
- Click + Additional Value to specify another value to extract.
- Select "Power_Comp" for Select Array Column to Extract and "2" for Position to Extract. Toggle "Custom" for Output Column Name and enter "I_out_A" for the column name.
- Toggle Drop Original Column off and click Run.
After running the node, the Combustion Temperature and Output Current are extracted as type Double in separate columns, which are shown in Figure 4.
Figure 4: Extracting Combustion Temperature and Output Current parameters