The purpose of the spectrogram processor in Specview is to apply a sequence of operations on one or more spectrograms, resulting in the end in a combined and eventually low-pass filtered version of the input data.
Processing is controlled in a pipeline fashion, in which each step is executed in sequence, the result of one step being fed as input to the next. Spectrograms can be first scaled, offset and trimmed at the end points. Next, they can be resampled to a uniform spacing in wavelength, coadded, detrended and low-pass filtered. Each step in this sequence can be fine tuned, or even completely skipped over. The result is immediately plotted, and also stored for later use.
The steps are:
By default, the Detrend
and Filter modules are inactive when the processor is first invoked. See
in Sec. 5.7 how to activate them.
When the Process button is clicked in the memory buffer selector window (Sec. 1.3), the Pipeline Manager is invoked, initialized with the spectrograms currently selected in the memory buffer selector window, and the pipeline runs once with default parameters for all steps. The result is automatically displayed. A Pipeline Manager control window is also displayed.
By default, when the Pipeline manager is invoked, the
pipeline will just resample and coadd together the input spectrograms.
To re-run the pipeline, just click on the Execute button at the bottom row in the Pipeline Manager window.
The first time the pipeline runs, the resulting spectrogram is stored in the memory buffer (Sec. 1.3) with a “Processed” identifier. From there, it can be used for subsequent plotting and data analysis operations.
Subsequent runs of the pipeline by the Execute button do not result in a new spectrogram being added to the buffer. The results are overwritten on the same buffer slot instead.
However, for the purposes of enabling a history mechanism (Sec. 5.6), the Pipeline Manager also keeps an internal storage area in which every run of the pipeline is kept.
The default format for the display of processed spectrograms uses two plot areas on a single window. The top area displays the processed spectrogram(s), that is, the result at the very end of the pipeline. If either the detrending or filtering steps, or both, are turned on, the top display will also depict (in lighter color) the spectrograms(s) in the pipeline right before they are submitted to the detrending and/or filtering steps.
The bottom area
displays the input spectrogram(s) in co plot mode (as if displayed by the Plot
button in the memory buffer selector window – Sec. 1.3). The top and bottom displays have
their WCS view ports synchronized with each other in the sense that any zoom or
pan operation executed in one of them will be repeated on the other.
To display just the processed result, select the Single button at the bottom row in the Pipeline Manager window (Sec. 5.3). To go back to dual display mode, select the Dual button.
Every time the Execute button runs the pipeline, the previous run (results and parameters) is pushed into a stack-like storage. The Undo button at the bottom row in the Pipeline Manager window (Sec. 5.3) pops the stack back one level (thus discarding the most recent run). The selector at its side pulls the requested run from the inside of the stack back to the top of the stack. This mechanism is meant to facilitate a trial-and-error approach to the problem of finding the best set of parameters for a given data set and science goal.
Steps are implemented as separate modules. Each module has associated to it a set of parameters, which can be accessed by selecting the corresponding tabbed pane with the tabs at the top of the Pipeline Manager window (Sec. 5.3).
Each module can be in an active or inactive state. The state of each module can be quickly glanced at via the Active steps row of indicators in the Pipeline Manager window (Sec. 5.3). When a steps runs, its indicator number briefly changes to a red color. If the step stops with an error, the red color remains until the pipeline is run again.
Modules can be individually set to active or inactive mode by checking / unchecking the Execute this step box in the corresponding module control panel.
If active, a given module will get the list of input spectrograms from the previous module in the pipeline, process it, and pass it to the next module. If inactive, the product of the previous processor module is passed along, unchanged, to the next.
The only exception to this rule is the Splice / Co-add step. Even when inactive, it operates upon the set of input spectrograms. The result of that operation, when in inactive mode, is identical with the result obtained when clicking the Plot button in the memory buffer selector window (Sec. 1.3). The reason behind this behavior is that at some point down in the pipeline, processor modules will accept only a single input spectrogram to operate upon. Thus, at some point in the pipeline stream, a many-to-one operation must necessarily take place. In the case where the initial list of spectrograms contains a single element, the Splice / Co-add step will act as a no-operation.
At any time, the entire parameter set of a given processor module can be reverted back to its default state by clicking the Default button.
All computations in the processor modules are carried out with the data expressed in photlam (photons/s/cm^2/Angstrom) and Angstrom units. Also, input parameters for all modules must be expressed in these same units, when applicable.
Note that the display may be set to a different set of units. Units for the processor display output are always taken to be whatever units are selected as Preferred Units (Sec. 2.12). These can be selected with the Preferences menu but can also be set by a command line parameter (Sec. 8), or by a user preferences setting (Sec. 9).
In principle, processor modules should be activated one at a time, from first to last. Due to the many possible combinations of input data, and also the specific science to be achieved with the data, there is no general recipe on how to run the pipeline. It is up to the user to decide which steps to activate, and what parameter values to use in each. Individual processor modules may impose requirements over the data that is submitted to them. See the individual descriptions for details.
At this point, the pipeline can only process the flux and error arrays on input spectrograms. Even the error array is discarded after a coadd operation, thus the resulting spectrogram will have often only one single flux array.
Spectrograms resulting from a processor run can be saved to a display file (Sec. 2.16). These display files do not store the spectrograms themselves, but just a list of instructions on how to access the input spectrograms and process them. Once such a display file is ingested, the original input spectrograms are read, and the processor is invoked and run with parameters supplied in the display file.
Bug: ISO spectrograms won’t save correctly to a display file. Reading such files can cause an exception to be thrown.
The spectrogram created by the processing pipeline can be saved in IVOA SED Data Model compliant format using Specview’s main Save as menu (see Sec. 11).
The spectrogram can also be saved to file in the form of a basic ASCII table, if one intends to import it into other environments that do not support the SED standard, such as IDL or IRAF. The table is suitable for ingestion again in Specview using the ASCII text table format (Sec. 1.6.17). This feature is accessed by the File menu in the main processor window (Sec. 5.3). The
The processor described in this chapter may require significant amounts of memory, in particular when running on large spectrograms. When the processor runs out of memory, an “OutOfMemoryError” message may be displayed, and the processor stops executing. In this case it is advisable to quit the processor and discard whatever spectrogram(s) were being submitted to it (use the Delete button in the memory buffer selector window - Sec. 1.3).
Another situation that may lead to either memory overflow or an apparent CPU freeze is caused by mixing together spectrograms with extremely different characteristics. As an example, if one simultaneously inputs in the processor a finely grained high resolution echelle UV spectrogram together with an infrared data set, the processor default settings will cause the generation of extremely large arrays (for holding the broad band data with a very fine resolution). Either this may cause an “OutOfMemoryError” condition, or the system may spend minutes in processing and plotting the extremely large arrays.
A temporary fix for the above case is in place for now. It generates an error message when the number of data points in the final processed product exceeds 100,000.
See the “Memory requirements” section in the INSTALL_NOTES.txt file distributed in the Specview package for instructions on how to increase the amount of available memory for the Java Virtual Machine.