Version History

9.4.0

  • Added the Danish wavefront estimation algorithm.

9.3.1

  • Added conditional sigma clipping for averaging Zernike coefficients.

9.3.0

  • Added a separate instrument for full-array mode

  • Updated the ComCam mask model to match the bug fixes in Batoid

9.2.1

  • Added unit test directly comparing ImageMapper optical models to Batoid raytracing.

9.2.0

  • Add LSSTComCamSim as allowed camera type.

9.1.1

  • Fix latiss tests by using getpass, and updating Zk values

9.1.0

  • Added jmin arguments to Zernike utility functions.

  • Added jmin and jmax value checks to the Zernike utility functions.

9.0.0

This is a big backwards-incompatible refactor of WEP. The major changes are:

  • Split the cwfs modules into centroid, and estimation.

  • Donut Images are now held by the Image class. This class is meant to hold information in the global camera coordinate system (CCS).

  • A new Instrument class with new configurations in the policy/instruments directory. This class holds geometric information about the different telescopes and cameras, as well as interfaces with the Batoid models.

  • The ImageMapper class maps Image objects between the image and pupil planes, and creates pupil and image masks. The “offAxis” model now uses a real-time band-dependent fit with Batoid. The “onAxis” and “paraxial” models work the same as before.

  • The Zernike estimation classes have been generalized to allow different wavefront algorithm classes to plug into WfEstimator.

  • The TIE algorithm is implemented in estimation.TieAlgorithm.

  • There are new utilities in utils for fitting mask models and plotting mask models and the ImageMapper methods.

  • Instrument configuration in tasks is now pulled from the default parameter files for each camera type. Overrides can be provided via the instConfigFile parameter. With the default instrument configurations, defocal offsets are pulled from the exposure metadata. If defocalOffset is explicitly set in the instConfigFile override, that defocal offset is used instead of the values from the exposure metadata.

  • The donutTemplateSize config parameter has been removed from all the relevant tasks, as the new ImageMapper can predict the required template size. initialCutoutPadding provides padding beyond this predicted value.

  • The multiplyMask and maskGrowthIter parameters have been removed from CutOutDonutsBase. To mask blends during TIE fitting, instead use the maskKwargs parameter of the EstimateZernikesTieTask.

  • When estimating Zernikes, the maximum Noll index (jmax) is now a configurable parameter (maxNollIndex in EstimateZernikesBaseConfig). You can also toggle whether estimation starts from zero or from the telescope’s instrinsic Zernikes. You can toggle whether the task returns the full optical path difference (OPD) or just the wavefront deviation (OPD - intrinsic Zernikes). You can toggle whether the returned Zernikes start with Noll index 4 (the previous standard), or with index 0 (matching the Galsim convention). You can also set the units of the returned Zernikes.

  • The algorithm history can now be saved at the Task level using the saveHistory option in EstimateZernikesBaseConfig. The history is saved in the task metadata in a json-compatible format. To convert the history back to the native format, use utils.convertMetadataToHistory.

  • Changing from the native butler coordinate system (data visualization coordinate system with rotated wavefront sensors) to the WEP coordinate system (camera coordinate system with de-rotated wavefront sensors) now happens entirely in task.DonutStamp._setWepImage. Furthermore, the defocal_distance saved in the stamp is now the detector offset (or equivalent detector offset) rather than the raw focusZ info.

  • The AuxTel/LATISS unit tests have been fixed, and the LATISS Zernike calculation test has been explicitly switched to a regression test (rather than an accuracy test).

  • Enum’s now map to strings instead of integers. This natural Enum-string connection replaces the various utils that previously existed to map between Enums and strings.

8.3.1

  • Update tests to be more robust to DM changes and fix failures after DM stack update to w_2024_08.

  • Run black v24.2.

8.3.0

  • Remove mask_comp and mask_pupil from DonutStamp since they don’t persist and mask is already contained in MaskedImage stamp.

8.2.0

  • Add background subtraction to cutOutDonutsBase.

8.1.1

  • Replace calls to removed pipeBase.ButlerQuantumContext with pipeBase.QuantumContext.

8.1.0

  • Remove Zemax Coordinate System (ZCS) conversions now that ts_ofc works exclusively in Camera Coordinate System (CCS).

8.0.4

  • Update default config on GenerateDonutFromRefitWcsTask after updates in meas_astrom.

8.0.3

  • Attach locally linear WCSs to DonutStamps.

8.0.2

  • Adds support for MacOS.

8.0.1

  • Add convertZernikesToPsfWidth to zernikeUtils.

8.0.0

  • Save all DonutStamps with images aligned with focal plane science sensors.

  • This version will break compatibility in the closed loop with Phosim and ts_phosim going forward.

7.0.1

  • Fix generateDonutDirectDetect when doDonutSelection is not run.

7.0.0

  • Organize all utility functions inside the utils module.

6.4.12

  • Update ts_pre_commit_config with ruff.

6.4.11

  • Fix GenerateDonutFromRefitWcsTask adding coord_raErr, coord_decErr fields.

6.4.10

  • Update calcZernikesLatissPipeline yaml with instrument-specific setup for generateDonutDirectDetectTask.

6.4.9

  • Replacing lookUpCalibrations function to use the one in lsst.fgcmcal.utilities

6.4.8

  • Add github actions to check version history was updated and linting.

  • Fix black and flake8 violations.

  • Fix Jenkinfile.

6.4.7

  • Set default optical model for comCam to onAxis.

6.4.6

  • Fix tests that failed due to changes in numpy testing methods and WCS output.

6.4.5

  • Update setup files with pre-commit hooks, run black and isort.

6.4.4

  • In utility, update getFilterTypeFromBandLabel to return FilterType.REF if the bandLabel is not recognized.

6.4.3

  • Fix error in Jenkinsfile that caused git-lfs to fail when running on develop branch.

6.4.2

  • Move fits files to git-lfs.

6.4.1

  • Add documentation explaining how to run the WEP pipeline on the USDF batch system.

6.4.0

  • Create generateDonutCatalogUtils to store common methods.

  • Update generateDonutCatalogOnlineTask to match output of other generateDonutCatalog…Tasks.

6.3.5

  • Make sure output from empty catalogs match that expected from catalogs with sources in donutSourceSelectorTask.

  • Add tests for run method in donutSourceSelectorTask.

6.3.4

  • Patch refCatalogInterface to eliminate warnings from latest version of daf_butler.

6.3.3

  • Change filter name in testData/gen3TestRepo camera fits files to comply with new obs_lsst convention.

6.3.2

  • Change CWFS pipeline configuration files to have 1.5mm offset included and to handle this properly in CWFS version of tasks.

6.3.1

  • Directly calculate dI/dz in Algorithm, without the intermediate dI.

  • Save dI/dz and I0 in Algorithm history when debugLevel>=1.

6.3.0

  • Add filterLabel property to CompensableImage.

6.2.0

  • Add optional pipeline task to fit WCS from direct detect donut catalogs and generate new donut catalog from reference catalogs with fit WCS.

6.1.3

  • Add license information to test_generateDonutDirectDetectTask.

6.1.2

  • Import MaskedImage directly from afw.image to fix errors from change in w_2023_16.

6.1.1

  • Fix blend_centroid_x and blend_centroid_y to only return donuts bright enough to count as blended when maxBlended is greater than the total number of donuts.

6.1.0

  • Add optional ability to specify filter in GenerateDonutCatalogWcsTask.

6.0.2

  • Fix assignment of blend_centroid_x and blend_centroid_y in donut catalogs.

6.0.1

  • Redesign and enhance documentation to match style and detail of other TS repositories.

6.0.0

  • Rename all modules to start with lowercase in order to align with DM convention.

  • Add information into documentation about how this new change breaks repositories with data processed with older versions of ts_wep and how to fix it.

5.1.0

  • Add bandpass information into DonutStamp objects.

5.0.1

  • Run black v23.1.0.

5.0.0

  • Remove deprecated keywords in cwfs/Tool.py and deprecated function in cwfs/CompensableImage.py.

  • Remove deprecated EstimateZernikes Tasks.

  • Remove deprecated pipelineConfig test files.

4.2.3

  • Add transposeImages as optional config to CalcZernikesTask.

4.2.2

  • Add flux sort into GenerateDonutDirectDetectTask to make it consistent with other catalog generation tasks.

4.2.1

  • Use nan string instead of None so we can convert to float and use writeFits method in DonutStamps successfully and save in butler.

4.2.0

  • Add DonutQuickMeasurementTask.py to incorporate quick donut detection and measurement using LSST Science Pipelines.

  • Integrate DonutQuickMeasurementTask into GenerateDonutDirectDetectTask to speed up direct detection catalog generation.

4.1.0

  • GenerateDonutCatalogWcsTask takes filter information from exposures automatically.

  • DonutSourceSelectorTask uses policy/task/magLimitStar.yaml for default magnitude limits.

4.0.4

  • Update calls to deprecated LoadIndexedReferenceObjectsTask to use LoadReferenceObjectsTask.

4.0.3

  • Add blend_centroid_x and blend_centroid_y to GenerateDonutDirectDetectTask catalogs.

4.0.2

  • Fix test_estimateZernikesCwfsTask call to ButlerQuantumContext.

4.0.1

  • Remove Gen2 daf_persistence from UPS table.

4.0.0

  • Add masked deblending to CompensableImage and pipeline tasks.

  • Change how DonutSourceSelectorTask works by adding minBlendedSeparation parameter and changing DonutRadius to unblendedSeparation parameter.

3.2.0

  • Port Latiss functionality from EstimateZernikesLatissTask into CutOutDonutsScienceSensorTask + CalcZernikesTask pipeline.

  • Deprecate EstimateZernikes family of tasks. These tasks will no longer be updated and will be removed after January 2023.

3.1.5

  • Throw exception when auxTel is trying to use offAxis model.

3.1.4

  • Remove imageCoCenter step from Algorithm.

  • Add DeprecationWarning that imageCoCenter function in CompensableImage will be removed after January 2023.

3.1.3

  • Added default value to DonutStamp for DFC_DIST to allow the butler to read DonutStamp from repositories created with older versions of ts_wep.

3.1.2

  • Update phosimOutput corner sensors test files.

3.1.1

  • Fix tests pipeline yaml files updating the ISR setting to use ‘MEDIAN’ for overscan fit type.

  • Remove obsolete _generateTestExposures.

  • Fix test_generateDonutDirectDetectTask.py

3.1.0

  • Added a history to the Algorithm class that stores intermediate products of the algorithm (see Algorithm.getHistory()).

  • Fixed the algorithm so that it is once again symmetric with respect to I1 and I2. This involved simplifying the way that mask and image orientation are handled for the extrafocal image (see below).

  • Added the option to create masks in the orientation of the original images by setting compensated=False in CompensableImage.makeMask().

3.0.1

  • Fix test_generateDonutCatalogWcsTask.py to work with more recent versions of the DM stack.

3.0.0

  • Refactor tasks to directly accept instrument parameters in their configuration.

2.7.0

  • Remove dictionary defining allowable offsets in Instrument.py and replace with settable parameter.

  • Allow Instrument.py to be configured directly from dictionary of instrument parameters in addition to policy file.

2.6.0

  • Replace getters and setters in Instrument.py with properties to make more pythonic.

  • Update Algorithm, CompensableImage and DonutTemplateModel with new Instrument.py design.

2.5.8

  • Change focusZ in headers of repackaged phosim data to be in mm instead of microns after phosim_utils update.

2.5.7

  • Add defocal distance into DonutStamp.

2.5.6

  • Fix task input order in test_estimateZernikes… tests.

2.5.5

  • Change default maxFieldDistance in DonutSourceSelectorTask.py to 1.813 degrees based upon results from DM-33180.

  • Fix test in test_calcZernikesTaskScienceSensor to use correct intraFocal dataId.

2.5.4

  • Update science sensor and LATISS tasks to get focusZ from exposure visitInfo instead of metadata after update in DM-35186.

2.5.3

  • Update tests and gen3TestRepo to work with latest version of the stack (w_2022_28).

2.5.2

  • Add ComCam to donutTemplateModel.

  • Add error message to donutTemplateModel for AuxTel if not run with ‘onAxis’ optical model.

2.5.1

  • Correct orientation of masks in pipeline tasks.

2.5.0

  • Update names of cMask to mask_comp (padded), pMask to mask_pupil (non-padded)

  • Correct output of getPaddedMask to mask_comp, getNonPaddedMask to mask_pupil

2.4.4

  • Added documentation link to the README.

2.4.3

  • Fix online documentation build errors.

2.4.2

  • Remove matplotlib backend switching in PlotUtil.py

2.4.1

  • Add information on Jupyter Notebooks in ts_analysis_notebooks to README.

2.4.0

  • Add CutOutDonuts tasks and CalcZernikesTask to separate cutting out donut stamps and calculating Zernikes from donut stamps as separate tasks.

2.3.8

  • Remove phosim_utils dependency.

2.3.7

  • Optimize CWFS algorithms.

2.3.6

  • Fix rotation of sensors in EstimateZernikesBase.

2.3.5

  • Update scipy.ndimage namespace to fix deprecation warnings.

  • Run black v22.3.

2.3.4

  • Fix test for EstimateZernikesLatissTask, to run for any user with /repo/main/ access.

2.3.3

  • Add donut location configuration setting to DonutSourceSelectorTask.

2.3.2

  • Change CombineZernikesSigmaClip to use the more robust mad_std standard deviation algorithm.

  • Add maxZernClip configuration parameter to CombineZernikesSigmaClip.

  • Change CombineZernikes metadata to use integer flags.

2.3.1

  • Rely on GalSim for Zernike and Cartesian polynomial evaluation.

2.3.0

  • Add EstimateZernikesLatissTask to process auxTel data

  • Add GenerateDonutDirectDetectTask to find donuts with template fitting

  • Add choices for binary image creation in DonutDetector

  • Add getCamType and getDefocalDisInMm to Utility

  • Add donut template for auxTel in DonutTemplateModel

2.2.4

  • Update Jenkinsfile to always pull the image before new builds and improve cleanup stages to make build more robust.

2.2.3

  • Change EstimateZernikesCwfsTask to be able to accept only a single pair of wavefront sensors.

  • Remove runQuantum function from EstimateZernikesScienceSensorTask since it does not add any functionality now that the task gets the camera from the butler.

2.2.2

  • Update functions marked deprecated as of stack version w_2022_06.

2.2.1

  • Distinguish AuxTel ZWO camera from LATISS

2.2.0

  • Add CombineZernikes…Tasks that combine the Zernike coefficients from multiple donut pairs into a single set of coefficients.

2.1.4

  • Remove timeMethod deprecation warnings and use static calibration camera.

2.1.3

  • Fix maxBlended parameter in DonutSourceSelectorTask and improve tests to check this configuration setting.

2.1.2

  • Make sure catalogs from GenerateDonutCatalog…Tasks have same columns.

2.1.1

  • Get camera from the butler when running pipeline tasks.

2.1.0

  • Refactor GenerateDonutCatalog*.py tasks.

  • Update EstimateZernikes…Tasks after DonutCatalog refactor.

2.0.4

  • Add DonutSourceSelectorTask to task module.

2.0.3

  • Add RefCatalogInterface to task module.

2.0.2

  • Patch to work with weekly w_2022_2:

    • loadSkyCircle no longer returns centroid column, use loadPixelBox instead.

2.0.1

  • Patch to work with latest weekly.

  • Update Jenkinsfile for CI job:

    • git command is no longer working after the latest update on our Jenkins server.

    • update path to plantuml.

2.0.0

  • Removed code not used in Gen3 Pipelines.

1.8.2

  • Removed CreatePhosimDonutTemplates.py and moved to ts_phosim.

1.8.1

  • Get sensor orientation and field position directly from camera through new DonutStamp objects instead of using SourceProcessor.

  • Fix rotation of postage stamps sent to WFEsti.

1.8.0

  • Refactored DonutStamp.py and added ability to recreate masks as afwImage.Mask objects.

1.7.10

  • Save outputZernikes for pairs of wavefront detectors not just a single output for all detectors.

1.7.9

  • Remove _shiftCenterWfs from Source Processor.

1.7.8

  • Update stamp rotations to work with CWFS.

1.7.7

  • Update focalplanelayout.txt with new Euler angle for SW0 sensors.

1.7.6

  • Update donutStamp with archive property.

  • Add LSSTCam/calib to collections path in test Gen3 pipelines.

1.7.5

  • Break generic pieces of GenerateDonutCatalogOnlineTask.py into GenerateDonutCatalogOnlineBase.py

  • Add GenerateDonutCatalogWcsTask.py to calculate donut catalogs when WCS is available

1.7.4

  • Remove old e-image corner wavefront sensor files.

  • Add updated corner wavefront sensor test data.

  • Add CWFS Zernikes code and tests.

1.7.3

  • Break generic pieces of EstimateZernikesFamTask.py into EstimateZernikesBase.py

1.7.2

  • Fix append and extend methods in DonutStamps.py.

  • Update tests in test_donutStamps.py to properly check append and extend methods.

1.7.1

  • Update FOCUSZ parameter in test data.

1.7.0

  • Replace WcsSol by DM’s wcs code in GenerateDonutCatalogOnlineTask.

  • Fix intra/extra zernike selection.

1.6.9

  • Add focusz as an argument to repackagePhosimImages in CreatePhosimDonutTemplates.py

1.6.8

  • Return both raw and averaged Zernikes to Butler repository in EstimateZernikesFamTask.py.

1.6.7

  • Fix flake error and update Jenkinsfile

1.6.6

  • Remove 90 degree offset from WcsSol.py now that phosim headers are updated.

1.6.5

  • Use FOCUSZ header information in EstimateZernikesFamTask.py.

1.6.4

  • Add EstimateZernikesFamTask.py to calculate Zernike coefficients in full-array mode through a Gen 3 pipeline.

1.6.3

  • Add DonutStamp and DonutStamps storage classes to hold postage stamps of donuts.

1.6.2

  • Update ROTANG header in realComcam test files

1.6.1

  • Update GenerateDonutCatalogOnlineTask.py to get instrument directly from pipeline configuration.

  • Setup ctrl_mpexec package in Jenkinsfile so tests can run pipetask command.

1.6.0

  • Create new task module

  • Add GenerateDonutCatalogOnlineTask.py in task module

  • Add tests/testData/gen3TestRepo as sample Gen 3 repo for testing

1.5.9

  • Build and upload documentation as part of the CI job.

  • Use develop-env image for the CI job, due to the need of java to build the documentation.

  • Disable concurrent builds.

  • Fix docstring in SourceSelector.connect method.

1.5.8

  • Reformat the code by black v20.8b1.

1.5.7

  • Update import of DetectorType.

1.5.6

  • Reformat code with black.

1.5.5

1.5.4

  • Update to using LsstCamMapper and new geometry, including focalplanelayout.txt

1.5.3

  • Add DonutTemplatePhosim class.

  • Add CreatePhosimDonutTemplates class and add bin.src/runCreatePhosimDonutTemplates.py

1.5.2

  • Fix the ZernikeMaskedFit() when passing masked data

1.5.1

  • Add donut template classes to make templates for CentroidConvolveTemplate.

  • Add DonutTemplateFactory, DonutTemplateDefault, and DonutTemplateModel.

1.5.0

  • Add CentroidConvolveTemplate as a new centroid finding method.

1.4.9

  • Unify the line ending to LF.

1.4.8

  • Remove the abbrevDectectorName() and expandDetectorName().

  • Remove the unused arguments of epoch, includeDistortion, and mjd in WCS related functions.

  • Fix the calcWfErr() for the LsstCamMapper.

1.4.7

  • Remove sims and obs_lsstSim dependencies.

  • Update WCS code to use obs_lsst.

1.4.6

  • Use the sims_w_2020_38.

1.4.5

  • Use the sims_w_2020_36.

  • Support the LSST full-array mode (FAM). Add the classes of BaseCwfsTestCase and BaseBscTestCase.

  • Put the limits of star’s magnitude into a configuration file.

  • Remove the serialization functions in FilterType enum.

1.4.4

  • Use the pybind11 instead of cython.

  • Add the clang-format check to .githooks.

1.4.3

  • Reformat the code by black.

  • Add the black check to .githooks.

  • Ignore flake8 check of E203 ans W503 for the black.

  • Use the sims_w_2020_21.

1.4.2

  • Improved handling of IO errors - catch more OS Errors instead of only file not exists.

1.4.1

  • Add the function to recenter the donut image with the template.

  • Add the instrument and test data of auxilirary telescope.

1.4.0

  • Use the sims_w_2020_15.

  • Use the factory pattern for deblend module.

1.3.9

  • Use the sims_w_2020_14.

1.3.8

  • Use the sims_w_2020_07.

1.3.7

  • Use the sims_w_2020_06.

  • Skip two tests in test_butlerWrapper.py and test_camIsrWrapper.py for the bugs in upstream.

  • Feedback to DM team.

1.3.6

  • Use the sims_w_2020_04.

1.3.5

  • Use the sims_w_2019_50.

1.3.4

  • Use the sims_w_2019_38.

1.3.3

  • Use the sims_w_2019_31.

  • Remove the conda package installation in Jenkinsfile.

  • Update the permission of workspace after the unit test.

1.3.2

  • Use the sims_w_2019_29.

  • Add the unit tests of cwfs module to check the outputs of cython related code.

  • Move the plotImage() from Tool.py to PlotUtil.py.

  • Install the ipython in Jenkinsfile to make the test environment to be consistent with the development.

1.3.1

  • Use the factory pattern for centroid find algorithms.

  • Move the SensorWavefrontError class of ts_ofc to here.

1.3.0

  • Use sims_w_2019_24.

  • Support the eimage.

  • Enable to update and save the setting file.

1.2.9

  • Use sims_w_2019_22.

  • Adapt the new version of ip_isr that fixes the bug that can not do the ISR continuously.

1.2.8

  • Use sims_w_2019_20.

1.2.7

  • Put the default BSC path and sky file path in default yaml file.

  • Concrete WEPCalculation class will connect and disconnect the database at each query.

  • Use sims_w_2019_18.

1.2.6

  • Utilize the interface classes to main telescope active optics system (MTAOS).

  • Use sims_w_2019_17.

1.2.5

  • Support the documenteer.

1.2.4

  • Use the yaml format for configuration files of cwfs module.

  • Use sims_w_2019_15.

1.2.3

  • Add the eups as the package manager.

  • Use sims_w_2019_12.

1.2.2

  • Add the RawExpData class and update the related functions.

1.2.1

  • Add the interface to MTAOS in ctrlIntf module.

1.1.1

  • Updated to use the scientific pipeline of sims_w_2019_02.

  • Add the referece filter type.

1.1.0

  • Updated the WEP to use the obs_lsst and scientific pipeline of sims_w_2018_47.

  • The phosim_utils is used to repackage the PhoSim output amplifer images to the format of multi-extention FITS.

1.0.1

  • Updated the WEP to use the obs_lsst and scientific pipeline of sims_w_2018_47.

  • The phosim_utils is used to repackage the PhoSim output amplifer images to the format of multi-extention FITS.

1.0.0

  • Finished the WEP in totally ideal condition with the scientific pipeline v.14.