pyKLIP¶
pyKLIP is a python library for direct imaging of exoplanets and disks. It uses an implmentation of KLIP and KLIPFM to perform point spread function (PSF) subtraction. KLIP is based off of principal component analysis to model and subtract off the stellar PSF to look for faint exoplanets and disks and are around it.
pyKLIP is open source, BSDlicened, and available at this Bitbucket repo. You can use the issue tracker there to submit issues and all contributions are welcome!
Features¶
 Capable of running ADI, SDI, ADI+SDI with spectral templates to optimize the PSF subtraction
 Library of KLIPFM capabilties including forwardmodelling a PSF, detection algorithms, and spectral extraction.
 A Forward Model Matched Filter of KLIP is available for GPI as well as postprocessing planet detection algorithms.
 Parallelized with both a quick memoryintensive mode and a slower memorylite mode
 Modularized to support data from multiple instruments. Currently there are interfaces to P1640, GPI, SPHERE, CHARIS, MagAO/VisAO, and Keck/NIRC2.
 If confused about what a function is doing, read the docstring for it. We have tried our best to document everything
 See Release Notes for update notes
Bugs/Feature Requests¶
Please use the Issue Tracker on Bitbucket to submit bugs and new feature requests. Anyone is able to open issues.
Attribution¶
The development of pyKLIP is led by Jason Wang with contributions made by Jonathan Aguilar, JB Ruffio, Rob de Rosa, Schuyler Wolff, Abhijith Rajan, Zack Briesemeister, Kate Follette, Maxwell MillarBlanchaer, Alexandra Greenbaum, Simon Ko, Tom Esposito, Elijah Spiro, Pauline Arriaga, Bin Ren, Alan Rainot, Arthur Vigan, Johan Mazoyer, Graça Rocha, Jacob Golomb, Jea Adams, Aarynn Carter, Minghan Chen, Robert Thompson, and Laurent Pueyo. If you use this code, please cite the Astrophysical Source Code Library record of it (ASCL or ADS)
Wang, J. J., Ruffio, J.B., De Rosa, R. J., et al. 2015, Astrophysics Source Code Library, ascl:1506.001
Contents¶
Installation¶
Dependencies¶
Before you install pyKLIP, you will need to install the following packages, which are useful for most astronomical data analysis situations anyways. The main pyKLIP code is crosscompatible with both python2.7 and python3.5.
 numpy
 scipy
 astropy
 Optional: matplotlib, mklservice
For the optional packages, matplotlib is useful to actually plot the images. For mklservice, pyKLIP autmoatically toggles off MKL parallelism during parallelized KLIP if the mklservice package is installed. Otherwise, you will need to toggle them off yourself for optimal performance. See notes on parallelized performance below.
For FowardModel Astrometry and Photometry specifically, you’ll also want to install the following packages:
 emcee
 corner
As pyKLIP is computationally expensive, we recommend a powerful computer to optimize the computation. As direct imaging data comes in many different forms, we cannot say right here what the hardware requirements are for your data reduction needs. For data from the Gemini Planet Imager (GPI), a computer with 20+ GB of memory is optimal for an 1 hour sequence taken with the integral field spectrograph and reduced using ADI+SDI. For broadband polarimetry data from GPI, any laptop can reduce the data.
Install¶
Due to the continually developing nature of pyKLIP, we recommend you use the current version of the code on Bitbucket and keep it updated. To install the most up to date developer version, clone this repository if you haven’t already:
$ git clone https://bitbucket.org/pyKLIP/pyklip.git
This clones the repoistory using HTTPS authentication. Once the repository is cloned onto your computer, cd
into it and run the setup file:
$ python setup.py develop
If you use multiple versions of python, you will need to run setup.py
with each version of python
(this should not apply to most people).
Note on parallelized performance¶
Due to the fact that numpy compiled with BLAS and MKL also parallelizes linear algebra routines across multiple cores, performance can actually sharply decrease when multiprocessing and BLAS/MKL both try to parallelize the KLIP math. If you are noticing your load averages greatly exceeding the number of threads/CPUs, try disabling the BLAS/MKL optimization when running pyKLIP.
To disable OpenBLAS, just set the following environment variable before running pyKLIP:
$ export OPENBLAS_NUM_THREADS=1
A recent update to anaconda included some MKL optimizations which may cause load averages to greatly exceed the number of threads specified in pyKLIP. As with the OpenBLAS optimizations, this can be avoided by setting the maximum number of threads the MKLenabled processes can use:
$ export MKL_NUM_THREADS=1
As these optimizations may be useful for other python tasks, you may only want MKL_NUM_THREADS=1 only when pyKLIP is called,
rather than on a systemwide level. By defaulf in parallelized.py
, if mklservice
is installed, the original
maximum number of threads for MKL is saved, and restored to its original value after pyKLIP has finished. You can also
modify the number of threads MKL uses on a percode basis by running the following piece of code (assuming mklservice
is installed):
import mkl
mkl.set_num_threads(1)
Parallelization on windows¶
On windows, you might run into the following error:
RuntimeError:
Attempt to start a new process before the current process
has finished its bootstrapping phase.
This probably means that you are on Windows and you have
forgotten to use the proper idiom in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce a Windows executable.
If so, simply encapsulate the main code of your script within the following if statement if __name__ == '__main__':
as shown below.
Note that definitions of new functions or include statements can be placed before this statement. Example:
import glob
import pyklip.instruments.GPI as GPI
import pyklip.parallelized as parallelized
if __name__ == '__main__':
filelist = glob.glob("path/to/dataset/*.fits")
dataset = GPI.GPIData(filelist, highpass=True)
parallelized.klip_dataset(dataset, outputdir="path/to/save/dir/", fileprefix="myobject",
annuli=9, subsections=4, movement=1, numbasis=[1,20,50,100],
calibrate_flux=True, mode="ADI+SDI")
Release Notes¶
 Version 2.4.1
 Use pyKLIP version number rather than git commit to track versioning in headers (Jason Wang)
 Version 2.4
 Forward modeling can handle time dependent PSFs now (Jason Wang)
 Added STIS.py interface and demo notebook (Robert Thompson)
 Removed an extra 2x scaling in klip.nan_gaussian_filter() (Jason Wang)
 Fixed RDI bug where the reference library only has 1 image (Aarynn Carter)
 Fixed bug in background subtraction in GPIData.generate_psf_cube() (JB Ruffio)
 Version 2.3
 GPI interface improvements: coronagrpahic throughput, updated astrometric calibration, edge cases (Jason Wang, Rob De Rosa)
 GPI interface: Removed wind butterfly PCA subtraction has it was not effective (JB Ruffio)
 For PFS library, fixed diagonal elements of correlation matrix (JB Ruffio)
 Improvements to DiskFM implementation and python > 3.7 compatability (Johan Mazoyer)
 Fixed bug where pyKLIP crashes if you only have one science frame (Aarynn Carter)
 Added warning for debug mode, and supressing print statements if not in verbose mode (Jea Adams)
 Reorganized navigation bar for docs (Jason Wang)
 Version 2.2
 Field dependent throughput to account for changes in the offaxis PSF due to e.g., coronagraphic throughput (Jea Adams)
 Added verbose flag that can be used to turn off print statements within pyklip (Jea Adams)
 Various bug fixes (Jason Wang, Johan Mazoyer)
 Added for explanatory material to docs so that they are more accessible (Jea Adams)
 Version 2.1
 RDI support in forward modeling framework (currently works for DiskFM, support for other FM modules coming) (Johan Mazoyer)
 GenericData is more feature rich (better saving, automatic wcs generation) (Jason Wang)
 Minor bug fixes and documentation updates
 Version 2.0.1
 Update Python 3 version to Python 3.6
 Version 2.0
 Forgot to update for a long while. Lots of new changes. A few key summaries below.
 Forward modeling for planet detection, astrometry, photometry, spectral extraction, and disk forward modeling
 Support for Keck/NIRC2, Keck/OSIRIS, Subaru/CHARIS, VLT/SPHERE, MagAO/VisAO, and a generic instrument interface for all else
 Alternative algorithms to KLIP: emperically weighted PCA, nonnegative matrix factorization
 RDI library support
 Automated tests to ensure correctness of main features
 Now released on PyPI/pip
 Version 1.1
 Updated installation to be much easier
 Reorganized repo structure to match standard python repos
 Improvements to automatic planet detection code
 Version 1.0
 Initial Release
 Fullyfunctional KLIP implementation for ADI and SDI
 Interface for GPI data in both spectral and polarimetry mode
 Utility functions like fake injection and contrast calculation
Basic KLIP Tutorial with GPI¶
Here, we will explain how to run a simple PSF subtraction using the KLIP algorithm in pyKLIP. If you are not familiar with KLIP, we suggest you first read the KLIP paper which describes the algorithm in detail. In this tutorial, we assume you are familiar with the terminology in KLIP. We will use GPI data as an example to explain the process. If you are planning to reduce data from another instrument, see For All Else: Generic Data Tutorial to learn how to ingest it. We recommend you go through the quick GPI example first. Other than reading in the data, all the PSF subtraction steps are the same.
Reading in GPI Data¶
First, you’ll need some reduced GPI datacubes to run KLIP on since pyKLIP does not reduce raw data. If you have raw GPI data you need to reduce, see the GPI Data Reduction Pipeline Documentation page, which has all of the instructions and tutorials to reduce GPI data. After reducing the data, you should have a series of 3D datacubes where the third dimension is either wavelength or polarization depending on if you are working with spectral or polarimetric data. Regardless, the data should have the satellite spot fluxes and locations measured and stored in the header as we will need these to register and calibrate the datacubes. If you don’t have any GPI data or are simply too lazy to reduce some yourself, you can use the reduced Beta Pic datacubes from the GPI Public Data Release.
Once you have reduced your data, we can identify and convert the GPI data from GPI specific information to standardized information for pyKLIP. These initial steps of reading in and parsing the data are demonstrated below:
import glob
import pyklip.instruments.GPI as GPI
filelist = glob.glob("path/to/dataset/*.fits")
dataset = GPI.GPIData(filelist, highpass=True)
pyklip.instruments.GPI.GPIData
returns dataset
, an implementation of the abstract class pyklip.instruments.Instrument.Data
with standardized fields
that are needed to perform the KLIP subtraction, none of which are instrument specific.
Please read the docstring for pyklip.instruments.GPI.GPIData
for more information on the the fields for GPI data.
Note
If you get an error here, you likely did not reduce the raw GPI data correctly, so please check that the satellite spots were measured and stored in the header.
Note
When reading in the GPI data, the data are no longer automatically highpass filtered.
You should explictly high pass filter the data if desired (we find it is typically good for planet SNR
using the optional keyword highpass=True
. You can also apply the highpass filter as preprocessing
step before KLIP in pyklip.parallelized.klip_dataset if you don’t want to do it here as it is slower.
Running KLIP¶
Next, we will perform the actual KLIP subtraction using Angular Differential Imaging and Spectral Differential Imaging (ADI + SDI). To take advantage of the easily parallelizable computation, we will use the
pyklip.parallelized
module to perform the KLIP subtraction, which uses the python multiprocessing
library to parallelize the code (i.e. split the work up across multiple processes on your device).
import pyklip.parallelized as parallelized
parallelized.klip_dataset(dataset, outputdir="path/to/save/dir/", fileprefix="myobject",
annuli=9, subsections=4, movement=1, numbasis=[1,20,50,100],
calibrate_flux=True, mode="ADI+SDI")
pyklip.parallelized.klip_dataset will save the processed KLIP images in the field dataset.output
and as FITS files saved using the directory and fileprefix
specified. The FITS files contain two different kinds of outputs. The first is a “KLmode cube”, a single 3D datacube where the zaxis is all the
different KL mode cutoffs used to model the stellar PSF. Here is an example KLmode cube using GPI public data on beta
Pic, where the planet is quite visible.
The second kind of output is a series of spectral datacubes where the zaxis is wavelength and each datacube uses a different KL mode cutoff as specified by its filename. Here is an example of a 20 KLmode cutoff cube using the same GPI data on Beta Pic.
Picking KLIP Parameters for Point Sources¶
There are a lot of ways to tune the reduction, so check out the pyklip.parallelized.klip_dataset()
docstring for
all the keywords you can use.
Here, we have provided the keywords which we use the most and should be sufficient for most
cases.
Geometry¶
We have divided the image into 9 annuli and each annulus into 4 sectors (which do not rotate with the sky) and run KLIP independently on each sector. Picking the geometry depends on the structure of the PSF, but we have found this to be pretty good for GPI data.
annuli_spacing
¶
By default we break the image up into equal sized annuli or concentric rings, with the exception of the last annulus which emcompasses the rest of the image.
Each annulus will undergo its own KL mode calculation. Sometimes we want smaller annuli closer in since the stellar PSF changes rapidly there.
In that case, we suggest setting annuli_spacing="log"
so the widths of the annuli increases logarithmatically. Note that when annuli are further split into
subsections, each sector does its own KL mode calculation.
“Aggressiveness”¶
“Aggressiveness” is a key parameter to tune in the PSF subtraction. Increasing the aggressiveness of the PSF subtraction typically allows you to better model and subtract the stellar PSF. However, doing so also typically causes any astrophysical flux (e.g. planets, disks) to also be subtracted to a higher degree. Generally, there is a sweet spot that balances subtracting the stellar PSF and maintaining the signal of planets and disks. The aggressiveness of the subtraction is tuned via a combination of the “movement” or “minrot” parameters and “numbasis” keywords, as described below.
movement
¶
In our example, we picked PSFs where any potential astrophysical source will have moved by 1 pixel due to ADI (azimuthal motion) and SDI (radial motion) to build the reference library that constructs our principal components. Decreasing the movement parameter increases the aggressiveness of the reduction as it will allow you to pick PSFs that are closer in time and wavelength. However, you will also suffer more selfsubtraction of potential astrophysical sources. We find for GPI data, 1 pixel is good for maximizing the SNR of potential planets in the data.
numbasis
¶
KL modes are the principal components of the KLIP analysis, representing patterns found in the images that will be used to construct our model of the star. We don’t usually pick just one KL basis cutoff for KLIP, but rather an array so we can play aroud with the optimal number.
Increasing the number of KL modes also increases the aggressiveness of the reduction.
For GPI data, we find between 2050 KL modes for planet data and 110 KL modes
for disk data is optimal. However, with both the movement
and numbasis
parameters, it requires a bit
of searching to find the optimal configuration.
mode
¶
The mode
keyword specifies whether we’d like to use Angular Differential Imaging (ADI), Spectral Differential Imaging (SDI), Reference Differential Imaging (RDI),
or any combination of the three to perform starlight subtraction.
ADI creates its principal components by comparing the stillness of a star’s PSF across images taken at different angles to a planet’s apparent rotation in those same images.
SDI makes use of images taken at a wavelength where we expect planets to peak, as well as images taken a nearby continuum wavelength where we expect stars to remain bright
and planets to become dim. By comparing images at the two wavelengths, KLIP is better able to distinguish the star from the planet, and thus better model and subtract starlight.
RDI uses a library of PSFs taken from other stars to model this star.
When RDI is specified, pyKLIP will construct its model of the starlight using a reference library of stellar PSF’s (passed in by the psf_library
parameter).
In order to generate this reference library, see the RDI with a PSF Library documentation.
spectrum
¶
A parameter not specified in this tutorial is the spectral template. Since we know exoplanet spectra should follow
the models (at least roughly), we can use that to better choose reference PSFs to subtract out the stellar PSF.
Currently, the only option is to optimze for Tdwarfs which have sharp methane absorption features. This can be
turned on by setting spectrum='methane'
. By doing this, in channels without methane absorption (i.e. where the
planet signal is strong), we will use reference PSFs from channels where with methane abosrption (i.e. where the planet
signal is weak). The aggressiveness of this is tuned with the movement
keyword (i.e. by decreasing movement
,
we will allow into the reference PSFs images at wavelengths where the ratio of “no methane abospriton”/”some methane
absorption” is smaller). When this keyword is set, we also do a weighted mean collapse in wavelength for the outputted
KLmode cubes.
Other¶
We have also choosen to flux calibrate the data to convert it into contrast units to work in more physical units.
Note
The calibrate_flux
keyword does not correct for algorithm throughput, which is a loss of
flux due to the PSF subtraction process. It merely provides the calibration to convert to contrast units. You
will then need to correct for algorithm throughput by methods such as fake planet injection.
See Calibrating Algorithm Throughput & Generating Contrast Curves which explains how to do this in the context of contrast curves.
There are more parameters that can be tweaked. Read the docstring of pyklip.parallelized.klip_dataset()
for
the full details.
Picking KLIP Parameters for Disks¶
Using KLIP for disks can be difficult since the optimal parameters will depend on the geometry of the disk and the amount of field rotation in the sequence. Below, we describe some starting points for tuning the subtraction. Note that for disks it is suggested to only use mode=”ADI” as SDI can severely distort the disk signal.
Geometry¶
PyKLIP splits divides the image into a number of annuli centered
around the center of the image as defined by the dataset.centers
attribute, and splits each of those annuli into a number of
subsections, set by the annuli
and subsection
keywords,
respectively. For disks, we find subsections=1
to be effective. The
number of annuli can also depend on the geometry of the disk, but we
find that annuli=1
is sufficient for most cases and produces
smoother looking reductions.
Aggressiveness¶
The aggressiveness of a PSF subtraction is influenced by a number of parameters described below. There is often no one optimal aggressiveness, and there is much to be gained from both more aggressive and less aggressive reductions. A more aggressive reduction will allows you to probe features at closer inner working angles at the cost of killing fainter or more extended features. The aggressiveness and the parameters you choose can also be affected by the geometry and strength of the detection. Edgeon disks are more resilient to more aggressive reductions while faceon disks will need less aggressive reductions due to the selfsubtraction associated with ADI.
Numbasis¶
Changing the number of basis vectors subtracted will show different sets of features. More basis vectors will selfsubtract more of the extended PSF structure, showing features in closer inner working angles while subtracting fewer basis vectors will show more extended features of the disk.
Minrot¶
Given the structure of debris disks, it is preferable to use the minrot criterion to select basis vectors rather than the movement parameters as is used in psf subtraction. The choise for this paraeter will depend on the geometry. For thin disks, a smaller minrot is desireable as it will allow for a cleaner subtraction while thicker disks will require a larger minrot to avoid selfsubtraction.
Instrument Tutorials¶
pyKLIP supports data from various instruments. Here we have compiled tutorials on how to use each specific insturment interface.
pyKLIP also has a pyklip.instruments.Instrument.GenericData
class to support generic highcontrast imaging data when
your instrument doesn’t yet have an interface and you don’t yet want to make one. Note that GenericData works fine for standard
KLIP reductions, but may not work for the more sophisticated forward modelling libraries.
GPI Tutorial¶
The following is just duplicated from Basic KLIP Tutorial with GPI, the basic pyKLIP tutorial.
First, you’ll need some reduced GPI datacubes to run KLIP on since pyKLIP does not reduce raw data. If you have raw GPI data you need to reduce, see the GPI Data Reduction Pipeline Documentation page, which has all of the instructions and tutorials to reduce GPI data. After reducing the data, you should have a series of 3D datacubes where the third dimension is either wavelength or polarization depending on if you are working with spectral or polarimetric data. Regardless, the data should have the satellite spot fluxes and locations measured and stored in the header as we will need these to register and calibrate the datacubes. If you don’t have any GPI data or are simply too lazy to reduce some yourself, you can use the reduced Beta Pic datacubes from the GPI Public Data Release.
Once you have reduced your data, we can identify and convert the GPI data from GPI specific information to standardized information for pyKLIP. These initial steps of reading in and parsing the data are demonstrated below:
import glob
import pyklip.instruments.GPI as GPI
filelist = glob.glob("path/to/dataset/*.fits")
dataset = GPI.GPIData(filelist, highpass=True)
pyklip.instruments.GPI.GPIData
returns dataset
, an implementation of the abstract class pyklip.instruments.Instrument.Data
with standardized fields
that are needed to perform the KLIP subtraction, none of which are instrument specific.
Please read the docstring for pyklip.instruments.GPI.GPIData
for more information on the the fields for GPI data.
Note
If you get an error here, you likely did not reduce the raw GPI data correctly, so please check that the satellite spots were measured and stored in the header.
Note
When reading in the GPI data, the data are no longer automatically highpass filtered.
You should explictly high pass filter the data if desired (we find it is typically good for planet SNR
using the optional keyword highpass=True
. You can also apply the highpass filter as preprocessing
step before KLIP in pyklip.parallelized.klip_dataset if you don’t want to do it here as it is slower.
Running KLIP¶
Next, we will perform the actual KLIP subtraction using Angular Differential Imaging and Spectral Differential Imaging (ADI + SDI). To take advantage of the easily parallelizable computation, we will use the
pyklip.parallelized
module to perform the KLIP subtraction, which uses the python multiprocessing
library to parallelize the code (i.e. split the work up across multiple processes on your device).
import pyklip.parallelized as parallelized
parallelized.klip_dataset(dataset, outputdir="path/to/save/dir/", fileprefix="myobject",
annuli=9, subsections=4, movement=1, numbasis=[1,20,50,100],
calibrate_flux=True, mode="ADI+SDI")
pyklip.parallelized.klip_dataset will save the processed KLIP images in the field dataset.output
and as FITS files saved using the directory and fileprefix
specified. The FITS files contain two different kinds of outputs. The first is a “KLmode cube”, a single 3D datacube where the zaxis is all the
different KL mode cutoffs used to model the stellar PSF.
Project 1640 PyKLIP tutorial¶
Usage instructions for Project 1640 are similar to those for GPI with the exception that the grid spot positions must be found before KLIP can be run. Import the P1640 instrument class instead of the GPI instrument class.
The grid spots locations only need to be found once, after which they can be read from a file. Instructions for use are found below.
Overview¶
P1640 Instrument class and support code to interface with PyKLIP PSF subtraction.
Author: Jonathan Aguilar
The code here defines the instrument class for Project 1640 that interacts with the rest of the PyKLIP module. The Instrument class contains the information that is needed to scale and align the datacubes, and to select the reference slicess.
Dependencies¶
 numpy
 scipy
 astropy
 python 2.7 or 3.4
 photutils #### Recommended (required to run the cube and spot verifier tools)
 matplotlib
Instructions for installing photutils can be found here:
http://photutils.readthedocs.io/en/latest/photutils/install.html. Note
that the conda instructions may not work  in that case, you can try
conda install c https://conda.anaconda.org/astropy photutils
Steps¶
The general steps are:
 Collect datacubes
 Vet datacubes
 Fit grid spots
 Vet grid spots
 Run KLIP
A set of tools built into PyKLIP makes this easier to do.
The trickiest part is setting up the grid spot fitting and making sure it succeeds. Once that’s done, the grid spot positions can simply be read in from a file. This is described in more detail below.
TODO: Contrast curves and fake injections require unocculted cubes. Currently there is no way to hook these in. Yeah, I want it too. If you want it so bad, do it yourself.
Tutorial¶
Important This tutorial assumes you are inside the following directory:
pyklip/pyklip/instruments/P1640_support/tutorial
A couple datacubes (with all but the essential information stripped from
them) are available by clicking this link
here
or from the command line with
wget https://sites.google.com/site/aguilarja/otherstuff/pykliptutorialdata/P1640_tutorial_data.tar.gz
Download the tarball and unpack the fits files into the
tutorial/data
folder with the command
tar xvf P1640_tutorial_data.tar.gz
Living On The Edge Version¶
If you trust me, you can do only steps “Collect the datacubes”, “Fit the gridspots”, and “Run KLIP”. This skips visual inspection of the datacubes and spot fitting.
The P1640Data class will automatically check for the presence of the spot files and, if it doesn’t find them, will attempt to do the fitting itself. You’re then trusting that the fitting succeeds. It normally does, but generally I like to fit the grid spots first, visually inspect them, and then move on to the KLIP step. If you don’t think you need to do this  or you already have done the grid spot fitting and vetting  then you can move right on to the Run KLIP step. Otherwise, proceed below to fit the grid spots.
Vet the datacubes¶
This uses the cube checker, a separate commandline tool that lets you quickly decide whether or not you should include a particular cube in your reduction.
Note: there is a new version called P1640_cube_checker_interactive
that is way easier to use, replace P1640_cube_checker
with this in
the lines below if you want to use it. We have noticed that it can take
a long time to load over ssh on Macs (for some reason this doesn’t
affect Linux). A workaround is to enable ssh compression with ssh C.
From an IPython terminal, do: (the syntax here is weird because telling python to evaluate python variables)
:::python
import sys
sys.path.append("..")
import P1640_cube_checker
good_cubes = P1640_cube_checker.run_checker(filelist)
or
%run ../P1640_cube_checker.py files {" ".join(filelist)}
Alternatively, from a bash terminal, do:
:::bash
filelist=`ls data/*Occulted*fits`
python ../P1640_cube_checker.py files ${filelist}
An animation of each cube, along with observing conditions and a comparison to the other cubes in the set, will pop up and the terminal will prompt you Y/N to keep it in the “good cubes” list. These are the files that you will keep for KLIP. If you like the cube, press Y. If you don’t, press N. All the Y’s will be spit out in a copypasteable format at the end, and stored in memory (in this case, in the variable good_cubes). After you’ve looped through all the cubes, you’ll be prompted to quit or reinspect the cubes. If you’re happy with your selection, go ahead and quit (Y), but if you want to revisit your choices, press N to restart the loop. You’ll have redo all of your decisions.
Fit grid spots¶
Note: you should only need to do this once, after which you can just read in the grid spot positions from a file.
First, reassemble your handy list of P1640 data.
Grid spots MUST exist, and (for now) they MUST be in the normal orientation. If this isn’t true, then the code will hang.
In order to fit the spots, we need the P1640spots module:
:::python
import sys
sys.path.append("..")
import P1640spots
# if the variables below are not set, default values will be read from P1640.ini
# for the tutorial, let's set them explicitly
spot_filepath = 'shared_spot_folder/'
spot_filesuffix = 'spot'
spot_fileext = 'csv'
for test_file in good_cubes:
spot_positions = P1640spots.get_single_file_spot_positions(test_file, rotated_spots=False)
P1640spots.write_spots_to_file(test_file, spot_positions, spot_filepath,
spotid=spot_filesuffix, ext=spot_fileext, overwrite=False)
(For now, only normallyoriented gridspots can be used, but in the
future you should be able to set rotated_spots=True
to fit
45degrotated grid spots).
The default values for the spot file filenames and directories (on Dnah
at AMNH) can be found in the P1640.ini
config file. I tend to write
a separate config file specifically for the reduction and define them
again there, with a custom directory if I want. An example reduction
config file will eventually be added to the repo.
Vet grid spots¶
We can run P1640_cube_checker
in “spots” mode to check the spots.
Usage is similar to before except now you need to use the spots
flag and specify the location of the spot file folder.
From IPython, there are two ways:
:::python
import sys
sys.path.append("..")
import P1640_cube_checker
good_spots = P1640_cube_checker.run_spot_checker(good_cubes, spot_path='shared_spot_folder/')
or
%run ../P1640_cube_checker.py files {" ".join(good_cubes)} spots spot_path shared_spot_folder/
From bash, do: (note: check the value of good_cubes before you pass it, make sure it got set properly)
:::bash
good_cubes="copy names of vetted files here"
python ../P1640_cube_checker files ${good_cubes} spots spot_path shared_spot_folder
Again, you will be prompted Y/n
for each cube. Y = keep it, N =
throw it out. At the end, you will be told all the files for which the
spot fitting FAILED and for which it succeeded. For these files, you can
either try to rerun the fitting, or (more likely) remove that cube from
the datacubes that get sent to PyKLIP.
When running in python mode, the variable good_spots
stores the file
names for which you said the spot fitting succeeeded. These are the
files which you will use to run KLIP, and can be used to initialize the
P1640Data object (more below).
Run KLIP¶
Running KLIP on P1640 data is nearly identical to running it on GPI,
with the exception that you have to be careful to only use cubes that
have corresponding grid spot files. We’ll start off by assuming that the
variable filelist
stores a list of the files that you want to
include in your reduction (i.e. they passed all the vetting stages
above).
:::python
import sys
sys.path.append("../../../../")
import pyklip.instruments.P1640 as P1640
dataset = P1640.P1640Data(filelist, spot_directory="shared_spot_folder/")
import pyklip.parallelized as parallelized
parallelized.klip_dataset(dataset, outputdir="output/", fileprefix="woohoo", annuli=5, subsections=4, movement=3, numbasis=[1,20,100], calibrate_flux=False, mode="SDI")
This will run the KLIP PSF subtraction algorithm. The resulting images
are stored in the dataset.output
field and written as FITS files to
the output directory with the file prefix you provided. The P1640 output
header format is that the first header stores the KLIP parameters, and
the subsequent headers store copies of the headers from the original
FITS files that were combined in this analysis. One file containing a
datacube is written for each KL cutoff specified.
SPHERE Tutorials¶
The pyKLIP SPHERE interface supports IFS and IRDIS data produced from Arthur Vigan’s SPHERE tools and IFS data from ESO’s SPHERE pipeline. These tutorials all assume you have processed the raw SPHERE data into a format like what is produced by Arthur Vigan’s SPHERE tools. That is, you will want the following things:
 Input data cube of dimensions (Nfiles, Nwvs, y, x)
 PSF cube of dimensions (Nwvs, y, x)
 A fits table with information like parallactic angle and pupil offset
 For IFS data, a wavelength array that provides the wavelegth solution
The ESO SPHERE IFS pipeline produces similar files, with a parallactic angle FITS file replacing the FITS table described above. One should be able to follow the following tutorial, but replacing the four data products mentioend above with the corresponding products produced by the ESO pipeline.
IFS Data¶
For SPHERE IFS data, we just need to pass in the output FITS files from Arthur Vigan’s pipeline. Outside of that, the only other thing to consider masking pixels into NaNs. Because the output data from the pipeline uses 0 rather than NaN to denote pixels that fall outside of the FOV, there can be edge effects (e.g. downweighting pixels at the edge with 0’s during mean collapse). Here, we will trim the edges of the FOV by masking any pixel within 3 pixels of a 0 as a NaN (i.e. using a 9 pixel box to mask things as NaNs).
datacube = "IFS_YJH_cube_coro.fits"
psfcube = "IFS_YJH_cube_psf.fits"
fitsinfo = "IFS_YJH_info.fits"
wvinfo = "IFS_YJH_wavelength.fits"
print("read in data")
dataset = SPHERE.Ifs(datacube, psfcube, fitsinfo, wvinfo, nan_mask_boxsize=9)
print(dataset.psfs.shape) # instrumental PSF at each wavelength stored here if you need it
parallelized.klip_dataset(dataset, outputdir="path/to/save/dir/", fileprefix="myobject",
annuli=9, subsections=4, movement=1, numbasis=[1,20,50,100],
mode="ADI+SDI", spectrum="methane")
IRDIS Data¶
For IRDIS data, we don’t need to specify the wavelength info. Instead we should pass in a string specifying an IRDIS band name
(options are "Y2Y3"
, "J2J3"
, "H2H3"
, "H3H4"
, "K1K2"
).
datacube = "IRDIS_K1K2_cube_coro.fits"
psfcube = "IRDIS_K1K2_cube_psf.fits"
fitsinfo = "IRDIS_K1K2_info.fits"
print("read in data")
dataset = SPHERE.Irdis(datacube, psfcube, fitsinfo, "K1K2")
print(dataset.psfs.shape) # instrumental PSF at each wavelength stored here if you need it
parallelized.klip_dataset(dataset, outputdir="path/to/save/dir/", fileprefix="myobject",
annuli=9, subsections=4, movement=1, numbasis=[1,20,50,100],
mode="ADI+SDI", spectrum="methane")
For All Else: Generic Data Tutorial¶
The pyklip.instruments.Instrument.GenericData
interface allows anyone to pass in any data set into pyKLIP to do basic
KLIP reductions without having to write an insturment module first. It is recommended that eventually there should be an insturment class
to leverage all the features of pyKLIP. However to test out pyKLIP on data from a new instrument or to handle a dataset you will only
see once (e.g. special lab data), then GenericData is the way to go.
Reading in Generic Data¶
Basically, to read in generic data, you’ll need to do most of the file handling yourself. Then the GenericData class will organize that information to be interfaceable with pyKLIP. For example, you will need to pass in things like the data frames, centers, filenames, inner working angles, etc.
Generic data requires you to pass in the data frames and centers. The data frames are passed in as a 3D datacube with dimensions of
(Nframes, y, x). The centers are passed in as an array of (x,y) coordiantes with dimensions of (Nframes, 2). The rest of the arguments
are optional and depends on your data (e.g. if ou have ADI data, you should pass in the parallactic angles; if you have SDI data you will
need to pass in the wavelengths; if you have RDI data, you will need to pass in the filenames). See the docstring of
pyklip.instruments.Instrument.GenericData
for the details.
Example with Simulated WFIRST Data¶
Here is an example of using GenericData on simulated WFIRST data available publically online (Version 3.1, Observing Scenario 5).
import astropy.io.fits as fits
import numpy as np
from pyklip.instruments.Instrument import GenericData
import pyklip.parallelized as parallelized
# Read in science images, which are taken at 2 roll angles. For each angle, the files come in as a 3D cube
# We want to append the two roll angles together
with fits.open("data/OS5_adi_3_polx_lowfc_random_47_Uma_roll_m13deg_HLC_sequence_with_planets.fits") as input_hdu_1:
with fits.open("data/OS5_adi_3_polx_lowfc_random_47_Uma_roll_p13deg_HLC_sequence_with_planets.fits") as input_hdu_2:
frames_per_roll = input_hdu_1[0].data.shape[0]
# the input science data is the combination of the two roll angles.
input_data = np.append(input_hdu_1[0].data, input_hdu_2[0].data, axis=0) # makes a (2N, y, x) sized cube
# generate roll angle lengths for each frame
pas = np.append([13 for _ in range(frames_per_roll)], [13 for _ in range(frames_per_roll)])
# for each frame, give it a filename
input_filenames = np.append(["OS5_adi_3_polx_lowfc_random_47_Uma_roll_m13deg_HLC_sequence_with_planets.fits" for _ in range(frames_per_roll)],
["OS5_adi_3_polx_lowfc_random_47_Uma_roll_p13deg_HLC_sequence_with_planets.fits" for _ in range(frames_per_roll)])
# all of the files are yet again at (31,31)
input_centers = np.array([[xcenter, ycenter] for _ in range(frames_per_roll*2)])
# set the inner working angle
IWA = 6 # pixels
# now let's generate a dataset to reduce for KLIP. This contains data at both roll angles
dataset = GenericData(input_data, input_centers, IWA=IWA, parangs=pas, filenames=input_filenames)
# set up the KLIP parameters and run KLIP
numbasis=[1,5,10,20,50] # number of KL basis vectors to use to model the PSF. We will try several different ones
maxnumbasis=150 # maximum number of most correlated PSFs to do PCA reconstruction with
annuli=3
subsections=4 # break each annulus into 4 sectors
parallelized.klip_dataset(dataset, outputdir="data/", fileprefix="pyklip_nonoise_k150a3s4m1", annuli=annuli,
subsections=subsections, numbasis=numbasis, maxnumbasis=maxnumbasis, mode="ADI",
movement=1)
If you want to write a new interface into pyKLIP, follow the Adding an Instrument Interface guide for making your own interface.
Calibrating Algorithm Throughput & Generating Contrast Curves¶
Due to oversubtraction and selfsubtraction (see Pueyo (2016) for a good explaination), the shape, flux, and spectrum of the signal of a planet or disk is distoed by PSF subtraction. To calibrate algorithm throughput after KLIP in this tutorial, we will use the standard fake injection technique, which basically is injecting fake planets/disks in the data of known shape, flux, and spectrum to measure the algorithm throughput.
In this tutorial, we will calibrate the throughput of the previous exmaple (Basic KLIP Tutorial with GPI) for the purpose of generating a contrast curve. Note that this same general process can be used to character a planet or disk (e.g. measure astrometry and spectrum of an imaged exoplanet).
Contrast Curves¶
To measure the contrast (ignoring algorithm throughput), we use pyklip.klip.meas_contrast()
, which assumes
azimuthally symmetric noise and computes the 5σ noise level at each separation. It uses a Gaussian cross correlation to
compute the noise as a small optimization to smooth out high frequency noise (since we know our planet is not going to
be smaller than on λ/D scales). It also corrects for small number statistics (i.e. by assuming a Student’s
tdistribution rather than a Gaussian).
This will give us a sense of the contrast to inject fake planets into the data (algorithm throughput is ~50%).
We are calculating broadband contrast so we want to spectrallycollapsed data (if applicable). You can do this
by reading back in the KL mode cube and picking a KL mode cutoff. (The KL mode cutoff is chosen to maximize planet SNR, which we won’t discuss here, but can be determined with fake planet injection.)
Here we will show an example using the pyKLIP output of GPI data, and using KL modes.
import astropy.io.fits as fits
hdulist = fits.open("myobjectKLmodesall.fits")
# pick the 20 KL mode cutoff frame out of [1,20,50,100]
kl20frame = hdulist[1].data[1]
dataset_center = [hdulist[1].header['PSFCENTX'], hdulist[1].header['PSFCENTY'] ]
Then, a convenient pyKLIP function will calculate the contrast, accounting for small sample statistics. We are picking 1.1 arcseconds as the outer boundary of our contrast curve. The low_pass_filter option specifies the size of the Gaussian to use in our cross correlation to smooth low frequency noise. It is typically smaller than the size of the PSF since selfsubtraction from KLIP decreases the PSF size. We also need to specify the FWHM of the PSF in order to account for small sample statistics. It also determines the spacing the contrast curve returns. The function samples the noise with a spacing of FWHM/2.
For our example dataset on Beta Pic, we also need to mask out the planet, Beta Pic b, so that it doesn’t bias our noise estimate.
import numpy as np
import pyklip.klip as klip
import pyklip.instruments.GPI as GPI
dataset_iwa = GPI.GPIData.fpm_diam['J']/2 # radius of occulter
dataset_owa = 1.5/GPI.GPIData.lenslet_scale # 1.5" is the furtherest out we will go
dataset_fwhm = 3.5 # fwhm of PSF roughly
low_pass_size = 1. # pixel, corresponds to the sigma of the Gaussian
# mask beta Pic b
# first get the location of the planet from Wang+ (2016)
betapicb_sep = 30.11 # pixels
betapicb_pa = 212.2 # degrees
betapicb_x = betapicb_sep * np.sin(np.radians(betapicb_pa)) + dataset_center[0]
betapicb_y = betapicb_sep * np.cos(np.radians(betapicb_pa)) + dataset_center[1]
# now mask the data
ydat, xdat = np.indices(kl20frame.shape)
distance_from_planet = np.sqrt((xdat  betapicb_x)**2 + (ydat  betapicb_y)**2)
kl20frame[np.where(distance_from_planet <= 2*dataset_fwhm)] = np.nan
contrast_seps, contrast = klip.meas_contrast(kl20frame, dataset_iwa, dataset_owa, dataset_fwhm, center=dataset_center, low_pass_filter=low_pass_size)
Now we can plot what the contrast curve (missing a calibration for algorithm throughput) looks like.
Injecting Fake Planets¶
KLIP naturally subtracts out planet flux due to oversubtraction and selfsubtraction. To calibrate our sensitivity to planets, we need to inject some fake planets at known brightness into our data to calibrate KLIP attenuation. In this tutorial, we only only inject a few fakes once into the data just to demonstrate the technique with pyKLIP. For your data, it is suggested you inject many planets to explore the attenuation factor as a function of brightness, separation, and KLIP parameters (more aggressive reductions increase attenuation of flux due to KLIP).Fake planets are free, so the more the merrier!
First, let’s read in the data again. This is the same dataset as you read in to run KLIP the first time.
import glob
filelist = glob.glob("path/to/dataset/*.fits")
dataset = GPI.GPIData(filelist, highpass=True)
Now we’ll inject 12 fake planets in each cube. We’ll do this one fake planet at a time using pyklip.fakes.inject_planet()
. As we get further out in the image, we will inject fainter planets, since the throughput does vary with planet flux, so we want the fake planets to be just around the detection threshold (slightly above is preferably to reduce noise). Since we specify a fake planet’s location by it’s separation and position angle, we need to know the orientation of the sky on the image using the frames’ WCS headers. The planets also are injected in raw data units, we need to convert the planet flux from contrast to DN for GPI. For other instruments, each should have its flux calibration and thus own method to convert between data units and contrast.
import pyklip.fakes as fakes
# three sets, planets get fainter as contrast gets better further out
input_planet_fluxes = [1e4, 1e5, 5e6]
seps = [20, 40, 60]
fwhm = 3.5 # pixels, approximate for GPI
for input_planet_flux, sep in zip(input_planet_fluxes, seps):
# inject 4 planets at that separation to improve noise
# fake planets are injected in data number, not contrast units, so we need to convert the flux
# for GPI, a convenient field dn_per_contrast can be used to convert the planet flux to raw data numbers
injected_flux = input_planet_flux * dataset.dn_per_contrast
for pa in [0, 90, 180, 270]:
fakes.inject_planet(dataset.input, dataset.centers, injected_flux, dataset.wcs, sep, pa, fwhm=fwhm)
Now we’ll run KLIP using the example same parameters on this dataset with fake planets.
import pyklip.parallelized as parallelized
parallelized.klip_dataset(dataset, outputdir="path/to/save/dir/", fileprefix="myobjectwithfakes",
annuli=9, subsections=4, movement=1, numbasis=[1,20,50,100],
calibrate_flux=True, mode="ADI+SDI")
Now, the resulting KLIP dataset should have 12 more planets in it! For the Beta Pic dataset, we actually have 13 planets ;).
We now will read in the output of the KLIP reducation with fake planets. Since we’re using the 20 KL mode cutoff frame for our contrast curve, we want the same cutoff for our reduction with fake planets.
kl_hdulist = fits.open("myobjectwithfakesKLmodesall.fits")
dat_with_fakes = kl_hdulist[1].data[1]
dat_with_fakes_centers = [kl_hdulist[1].header['PSFCENTX'], kl_hdulist[1].header['PSFCENTY'] ]
We will measure the flux of each fake in the reduced image using pyklip.fakes.retrieve_planet_flux()
. Our strategy here is to assume the throughput is constant azimuthally, and for each 4 planets at a separation, average their fluxes together to reduce noise. Note that we need to again specify a WCS header (the one corresponding to the output images) to tell the code where to look for the planet in the image. You can grab that from the header of the reduced image, or we will be lazy here are use the dataset.output_wcs
field from our fake dataset, which automatically gets rotated after KLIP.
retrieved_fluxes = [] # will be populated, one for each separation
for input_planet_flux, sep in zip(input_planet_fluxes, seps):
fake_planet_fluxes = []
for pa in [0, 90, 180, 270]:
fake_flux = fakes.retrieve_planet_flux(dat_with_fakes, dat_with_fakes_centers, dataset.output_wcs[0], sep, pa, searchrad=7)
fake_planet_fluxes.append(fake_flux)
retrieved_fluxes.append(np.mean(fake_planet_fluxes))
Now we can calibrate the contrast curves. We know what flux level we injected the planets into the data at. We now have measured the flux value of the planets at each separation, so we can now calculate the “algorithm throughput” which measures how much KLIP attenuates flux. Then for each location on the contrast curve, we will just use the closest fake planet injection separation to assume an algorithm throughput correction. This is why it is good in practice in inject fakes in as many places as possible, so that the fake planets better model the algorithm throughput at each separation.
# fake planet output / fake planet input = throughput of KLIP
algo_throughput = np.array(retrieved_fluxes)/np.array(input_planet_fluxes) # a number less than 1 probably
corrected_contrast_curve = np.copy(contrast)
for i, sep in enumerate(contrast_seps):
closest_throughput_index = np.argmin(np.abs(sep  seps))
corrected_contrast_curve[i] /= algo_throughput[closest_throughput_index]
Finally, we get a calibrated contrast curve!
Coronagraphic Throughput¶
If a coronagraph was used in data collection, it can have a measurable effect on the planet fluxes that we’re able to detect. Closer to the star, we might expect a coronograph to cause major dimming of observed planets, while planets at further separations from the star will likely be unaffected. Each coronagraph will typically have its own transmission profile, a measure of how its throughput changes as a function of distance from the center. It is quite useful for us to incorporate this into our contrast measurements in order to get a more accurate estimation of how bright planets must be in order for us to find them. Luckily, we can account for this coronagraphic effect on light transmission while estimating pyKLIP’s equivalent effect on planet flux, thereby jointly calibrating both coronagraphic and algorithmic throughput. The following will explain how we integrate the coronagraphic throughput correction by making a small modification to our fake planet injection function.
The pyklip.fakes.inject_planet()
function has an optional argument field_dependent_correction
, which accepts
a user provided function that corrects for coronagraphic throughput. The provided function should accept three arguments: the region or ‘stamp’ of your fake planet,
the physical ‘x’ separation of each pixel in the stamp from the center, and the physical ‘y’ separation of each pixel in the stamp from the center.
Provided with the transmission profile of the relevant coronagraph, this function should then scale the input stamp by the necessary amount, then output the throughput corrected stamp.
Prior to creating the function, be sure to read in your coronagraphic transmission profile. This should include columns for the transmission and distance from the star (in pixels).
# Read in transmission profile first
def transmission_correction(input_stamp, input_dx, input_dy):
"""
Args:
input_stamp (array): 2D array of the region surrounding the fake planet injection site
input_dx (array): 2D array specifying the x distance of each stamp pixel from the center
input_dy (array): 2D array specifying the y distance of each stamp pixel from the center
Returns:
output_stamp (array): 2D array of the throughput corrected planet injection site.
"""
# Calculate the distance of each pixel in the input stamp from the center
distance_from_center = np.sqrt((input_dx)**2+(input_dy)**2)
# Select the relevant columns from the coronagraph's transmission profile
transmission = transmission_profile['throughput']
radius = transmission_profile['distance']
# Interpolate to find the transmission value for each pixel in the input stamp
transmission_of_stamp = np.interp(distance_from_center, radius, transmission)
# Reshape the interpolated array to have the same dimensions as the input stamp
transmission_of_stamp = transmission_of_stamp.reshape(input_stamp.shape)
# Make the throughput correction
output_stamp = transmission_of_stamp*input_stamp
return output_stamp
Now, we can include the function as an optional argument in pyklip.fakes.inject_planet()
:
import pyklip.fakes as fakes
# three sets, planets get fainter as contrast gets better further out
input_planet_fluxes = [1e4, 1e5, 5e6]
seps = [20, 40, 60]
fwhm = 3.5 # pixels, approximate for GPI
for input_planet_flux, sep in zip(input_planet_fluxes, seps):
# inject 4 planets at that separation to improve noise
# fake planets are injected in data number, not contrast units, so we need to convert the flux
# for GPI, a convenient field dn_per_contrast can be used to convert the planet flux to raw data numbers
injected_flux = input_planet_flux * dataset.dn_per_contrast
for pa in [0, 90, 180, 270]:
fakes.inject_planet(dataset.input,
dataset.centers,
injected_flux,
dataset.wcs,
sep,
pa,
fwhm=fwhm,
field_dependent_correction = transmission_correction)
RDI with a PSF Library¶
pyKLIP supports RDI with a PSF Library using the pyklip.rdi.PSFLibrary
class. The PSF Library class holds a correlation
matrix of all frames with each other, so it knows which frames are good reference frames for a certain frame. This means that all
the data, science data and reference data, should be in the PSF libary. In this sense, it is good to generate the PSF Library
class once, spend the time to compute the correlation matrix once, and then save the correlation to file using the
save_correlation()
function inside of PSFLibrary. This way, the PSF library can be applied on multiple science targets, and
the correlation matrix needs to just be computed once.
Set up a PSF Library¶
Here we will assume you have a 3D cube of frames from a long series of data (e.g. a night of observing, a full survey) that is
already in the variable psflib_imgs
. This could be made from your own code, or from dataset.input
after you read in a
large list of files into the Data object. All of these images need to be aligned to some defined aligned_center
, a [x,y] array.
If the images haven’t been aligned to this common center already, you can use pyklip.klip.align_and_scale()
to register each
frame. If you are using GPI IFS data, you can also use the pyklip.instruments.Instrument.Data.spectral_collapse()
to
simultaeously center and collapse a dataset. You will also need psflib_filenames
, an array of filenames, so that each frame has a corresponding filename.
This again can be taken from dataset.filenames
if you don’t have it already, but have read in all the files into a Data object.
You want to make sure that the filenames are accurate, as this is how we figure out which frames to exclude for RDI (i.e. we don’t
want to use data in the science sequence as reference images in a RDI reduction).
Now here we demonstrate how to make the PSFLibrary object, compute the correlation matrix, and save the correlation matrix. Note that with a lot of files, generating the correlation matrix can take a long time.
import pyklip.rdi as rdi
# make the PSF library
# we need to compute the correlation matrix of all images vs each other since we haven't computed it before
psflib = rdi.PSFLibrary(psflib_imgs, aligned_center, psflib_filenames, compute_correlation=True)
# save the correlation matrix to disk so that we also don't need to recomptue this ever again
# In the future we can just pass in the correlation matrix into the PSFLibrary object rather than having it compute it
psflib.save_correlation("corr_matrix.fits", overwrite=True)
Then, in the future, we don’t need to recompute the correlation matrix when setting up the PSF library. Instead we can read it in and regenerate the PSFLibrary quickly.
import pyklip.rdi as rdi
import astropy.io.fits as fits
# read in the correlation matrix we already saved
corr_matrix_hdulist = fits.open("corr_matrix.fits")
corr_matrix = corr_matrix_hdulist[0].data
# make the PSF library again, this time we have the correlation matrix
psflib = rdi.PSFLibrary(psflib_imgs, aligned_center, psflib_filenames, correlation_matrix=corr_matrix)
Running RDI on a dataset¶
Now let’s assume you have a dataset
, a object that implements pyklip.instruments.Instrument.Data
. The input files
of dataset
are also part of the PSF library (e.g. part of the data taken in that night, or as part of the survey). We
will then prepare the PSF library for this dataset. This basically invalidates the frames in the PSF library that belong to this
target so they aren’t used as reference images, so there won’t be any selfsubtraction. This is where we use the filenames to match
frames, so it is important that dataset.filenames matches the filenames passed into the PSF library.
Then, it’s as simple as running KLIP. We pass in the same aligned_center
that all the images in the PSF library is aligned to
so that this reduction also aligns the science images to that center.
# now we need to prepare the PSF library to reduce this dataset
# what this does is tell the PSF library to not use files from this star in the PSF library for RDI
psflib.prepare_library(dataset)
# now we can run RDI klip
# as all RDI images are aligned to aligned_center, we need to pass in that aligned_center into KLIP
numbasis=[1,5,10,20,50] # number of KL basis vectors to use to model the PSF. We will try several different ones
maxnumbasis=150 # maximum number of most correlated PSFs to do PCA reconstruction with
annuli=3
subsections=4 # break each annulus into 4 sectors
parallelized.klip_dataset(dataset, outputdir="data/", fileprefix="pyklip_k150a3s4m1", annuli=annuli,
subsections=subsections, numbasis=numbasis, maxnumbasis=maxnumbasis, mode="RDI",
aligned_center=aligned_center, psf_library=psflib, movement=1)
Planet detection¶
Speckle subtraction algorithms like KLIP or LOCI are not planet detection algorithms. An additional step needs to be performed to compute a SNR for example. The SNR can be computed from an aperture photometry, which is then divided by noise standard deviation. The standard deviation is often calculated in an annulus at the same separation as the pixel. This approach is equivalent to a cross correlation of the image with an aperture. In this tutorial, we present simple functions to compute a crosscorrelation for broadband images, a simple matched filter for spectral cubes, a SNR map, and a function to quickly identify the brightest blobs in the image.
Note
The different terminology between cross correlation and matched filter is little arbitrary here since the cross correlation is a kind of matched filter. Here, we say matched filter when a division by the local variance is used.
Please find an example ipython notebook (pyklip/examples/kpop_tutorial.ipynb
) using beta Pictoris test data available in the test directory of pyklip.
Crosscorrelation¶
The cross correlation is the simplest step to perform before computing a SNR map.
calculate_cc
calculate the correlation of an image with a kernel, which represents the shape of the planet PSF.
It ensures that the image isn’t shifted when using even dimensions with scipy.signal.correlate2d
.
A spectrum can also be given to perform a weighted mean if the input image is a cube.
import astropy.io.fits as pyfits
filename = "path/to/image/image.fits"
hdulist = pyfits.open(filename)
image = hdulist[1].data
hdulist.close()
One can use different kernels. We provide two simple kernels: aperture (ie, hat) or 2d gaussian.
from pyklip.kpp.utils.mathfunc import *
x_grid,y_grid= np.meshgrid(np.arange(10,10),np.arange(10,10))
kernel_hat = hat(x_grid,y_grid, radius=3)
kernel_gauss = gauss2d(x_grid,y_grid, amplitude = 1.0, xo = 0.0, yo = 0.0, sigma_x = 1.0, sigma_y = 1.0)
The cross correlated image is then given by:
from pyklip.kpp.metrics.crossCorr import calculate_cc
image_cc = calculate_cc(image, kernel_gauss,spectrum = None, nans2zero=True)
The next step would to calculate the SNR map of image_cc
; see section below.
SNR map¶
There are two routines to compute a SNR map.
The fast version get_image_stat_map
computes the standard deviation in concentric annuli.
The center of the image is defined by center=[cen_x,cen_y]
.
Two consecutive annuli radii are separated by r_step
and their width is Dr
.
A caveat of this routine is that the standard deviation calculation will be biased by the presence of real point sources.
center = []#[cen_x,cen_y]
from pyklip.kpp.stat.stat_utils import get_image_stat_map
SNR_map = get_image_stat_map(image_cc,
centroid = center,
r_step=2,
Dr = 2,
type = "SNR")
A slower version of the routine will perform on similar operation for each pixel in the image.
It will mask a region of radius mask_radius
, and compute the standard deviation in an annulus of width Dr
with the same separation as the current pixel.
center = []#[cen_x,cen_y]
from pyklip.kpp.stat.statPerPix_utils import get_image_stat_map_perPixMasking
SNR_map = get_image_stat_map_perPixMasking(image_cc,
centroid = center,
mask_radius=5,
Dr = 2,
type = "SNR")
Simple matched filter¶
A more optimal way to detect a planet is to divide pixel values by their variance.
If the data is a spectral cube, we can also a template spectrum of the planet to improve our sensitivity.
run_matchedfilter
performs a matched filter using a 3D model of the planet including the planet PSF and a model of the spectrum of the planet planet_sp
.
We illustrate the example with a simple 2D gaussian PSF and a flat spectrum.
The function also estimates the local variance, which is used to normalize the matched filter.
import astropy.io.fits as pyfits
filename = "path/to/spectral/cube/cube.fits"
hdulist = pyfits.open(filename)
cube = hdulist[1].data
nl,ny,nx = cube.shape
hdulist.close()
# Definition of the planet spectrum
planet_sp = np.ones(nz)
# Definition of the PSF
from pyklip.kpp.utils.mathfunc import *
x_grid,y_grid= np.meshgrid(np.arange(10,10),np.arange(10,10))
PSF = gauss2d(x_grid,y_grid, amplitude = 1.0, xo = 0.0, yo = 0.0, sigma_x = 1.0, sigma_y = 1.0)
PSF = np.tile(PSF,(nl,1,1))*planet_sp[:,None,None]
from pyklip.kpp.metrics.matchedfilter import run_matchedfilter
mf_map,cc_map,flux_map = run_matchedfilter(cube, PSF,N_threads=None,maskedge=True)
Pointsource detection¶
The function point_source_detection
identifies the brightest point sources in an SNR map and returns a table including their SNR and location.
The algorithm is iterative. A disk of radius mask_radius
is masked around the brightest candidate at each iteration.
The table includes the following columns described below:
["index","value","PA","Sep (pix)","Sep (as)","x","y","row","col"]
 1/ index of the candidate
 2/ Value of the maximum
 3/ Position angle in degree from North in [0,360]
 4/ Separation in pixel
 5/ Separation in arcsec
 6/ x position in pixel
 7/ y position in pixel
 8/ row index
 9/ column index
import csv
from pyklip.kpp.detection.detection import point_source_detection
detec_threshold = 3 # lower SNR to consider
pix2as = 1 # platescale (pixel to arcsecond)
mask_radius = 15 # Size of the mask to be applied at each iteration
maskout_edge = 10 # Size of the mask to be applied at the edge of the field of view. Works even if the outskirt is full of nans.
candidates_table = point_source_detection(SNR_map, center,detec_threshold,pix2as=pix2as,
mask_radius = mask_radius,maskout_edge=maskout_edge,IWA=None, OWA=None)
The table can optionally be saved on disk:
savedetections = os.path.join(outputDir,"detections.csv")
with open(savedetections, 'w+') as csvfile:
csvwriter = csv.writer(csvfile, delimiter=';')
csvwriter.writerows([["index","value","PA","Sep (pix)","Sep (as)","x","y","row","col"]])
csvwriter.writerows(candidates_table)
Forward Model Matched Filter (FMMF)¶
This tutorial will provide the necessary steps to run the Forward Model Matched Filter (FMMF) that is described in Ruffio et al. (2016).
Why FMMF?¶
Speckle subtraction algorithms like PCA or LOCI are not planet detection algorithms. An additional step needs to be performed to compute the SNR. The SNR can simply be computed from aperture photometry divided by the standard deviation of the noise calculated in an annulus at the same separation, but this is not the optimal approach.
In signal processing, a matched filter is the linear filter maximizing the Signal to Noise Ratio (SNR) of a known signal in the presence of additive noise.
Matched filters are used in Direct imaging to detect point sources using the expected shape of the planet Point Spread Function (PSF) as a template.
Detection of directly imaged exoplanets is challenging since PSF subtraction algorithms (like pyKLIP) distort the PSF of the planet. Pueyo (2016) provide a technique to forward model the PSF of a planet through KLIP.
FMMF uses this forward model as the template of the matched filter therefore improving the planet sensitivity of the algorithm compared to a conventional approach.
FMMF Requirements¶
FMMF is computationally very intensive. It will take up to a few days to run a typical GPI dataset for example on a basic cluster node (but it’s worth it! ;)).
You also need the following pieces of data to forward model the data.
 Data to run PSF subtraction on
 A model or data of the instrumental PSF
 For IFS data, an estimate of the spectrum of the planet that one expects to find.
Running FMMF (Example using GPI data)¶
There are 3 steps in running FMMF:
 Read the data using an instrument class
 Define the MatchedFilter object
 Call
klip_dataset
to run the reduction.
We use the GPI test dataset, which is included in pyklip, to illustrate the method. See For All Else: Generic Data Tutorial for a way to generate a dataset object for any instrument.
Here, we can highpassfilter the data using the highpass
keyword when defining the GPI.GPIData
object.
Experience has shown that the highpass filter is an important step in the reduction, which will need to be performed by the user when using a different instrument.
if __name__ == '__main__':
inputDir = "path/to/dataset/"
outputDir = "path/to/save/dir/"
from pyklip.instruments import GPI
import glob
import os
dataset = GPI.GPIData(glob.glob(os.path.join(inputDir,"*.fits")), highpass=True)
In order to perform a matched filter, we need a model of the planet PSF. For GPI, there is a builtin routine in the dataset object to calculate it from the satellite spots. The PSF model should also be highpass filtered to better represent the signal in data, although we skip this step here for simplicity.
if __name__ == '__main__':
#[...]
import numpy as np
dataset.generate_psf_cube(20,same_wv_only=True)
PSF_cube_arr = dataset.psfs # Shape is [nwvs,ny,nx]
PSF_cube_wvs = np.unique(dataset.wvs)
In addition of the planet PSF, we need to assume a spectrum for the planet. The spectrum should be defined such that:
 it has the total flux of the star, ie correspond to a contrast of 1.
 it represents the total integrated flux of the PSF and not the simple peak value.
 it should be multiplied by the atmospheric and instrumental transmission.
 It has the same size as the number of images in the dataset.
 Note that
MatchedFilter
expects a list, so make it a list of one spectrum like this:[spectrum_vec]
.
We now need to define the fmlib
object, which is the object that will tell klip_dataset
the kind of reduction that we want to do (ie, FMMF).
numbasis
is the number of KL modes to be used. maxnumbasis
is the number of frames to be selected from the dataset and used to compute the covariance matrix.
if __name__ == '__main__':
#[...]
# Flat spectrum
spectrum_vec = np.ones((dataset.input.shape[0],))
# Number KL modes used for KLIP
numbasis = [5]
# Number of images in the reference library
maxnumbasis = [10]
# Build the FM class to do matched filter
import pyklip.fmlib.matchedFilter as mf
fm_class = mf.MatchedFilter(dataset.input.shape,numbasis,
PSF_cube_arr, PSF_cube_wvs,
[spectrum_vec])
FMMF is computationally extremely expensive. We recommend running it on computer or nodes with 16+ cores and 64+GB or RAM depending on the size of the dataset.
The example below can be tested on a laptop, but it will still likely take around 30min.
Before starting the reduction, we still need to define the subdivision of the field of view; the sectors. we restrict the reduction to the separation of the planet using annulus_bounds = [[28,33]]
.
The sectors are then defined to contain a number of pixels that is as close as possible to N_pix_sector = 200
. The annulis will be subdivided according to that constraint.
A significant difference compared to normal klip is that the sectors needs to be padded for the matched filter.
Set the value of padding
to half the width of the PSF array.
if __name__ == '__main__':
#[...]
import pyklip.fm as fm
prefix = "betpic131210J_GPI" #used in the filename of the outputs
annulus_bounds = [[28,33]]# This annulus is centered at the location of bet Pic b in the test dataset
N_pix_sector = 200
padding = PSF_cube_arr.shape[1]//2
movement = 2.0
fm.klip_dataset(dataset, fm_class, outputdir=outputDir, fileprefix=prefix, numbasis=numbasis,
annuli=annulus_bounds, N_pix_sector=N_pix_sector, padding=padding, movement=movement)
This function will produce 6 output files with the following extensions:
[...]FMMFKL#.fits
: This is the matchedfilter map, which should be proportional to the S/N of the planet.[...]FMContKL#.fits
: “Cont” stands for contrast. This is the estimated planet to star flux ratio from the maximum likelihood. Note that the estimated flux ratio, might still be subject to a certain amount of self and oversubtraction and should therefore be calibrated with simulated planet injection and recovery.[...]FMCCKL#.fits
: This is similar to FMMF, but the difference is that the local estimate of the standard deviation is not used as a weight when combining all the data together in a single map.[...]FMN_pixKL#.fits
: This map includes the number of valid pixels used in the dataset at any planet location.[...]klippedKL#speccube.fits
: This is a spectral cube of the klipped reduction.[...]klippedKL#KLmodesall.fits
: This is the 2D image resulting from flattening the klipped spectral cube.
Note that FMMF will not perform optimally on high SNR objects since the forward model will no longer be accurate. The forward model is indeed only a linear approximation of the speckle subtracted planet PSF, which will break when too little ADI/SDI diversity is present or if the planet is too bright.
Because the noise is assumed to be uncorrelated, the estimated SNR in the FMMF map is overestimated.
It needs to be renormalized. This can be done using the function get_image_stat_map_perPixMasking
.
This function is designed to compute the standard deviation of the image in concentric annuli.
It will repeat this operation locally for each pixel in the image by masking the neighboring pixels.
The goal is to prevent a putative planet to contaminate its own SNR estimation by artificially increasing the empirical standard deviation.
If type="stddev"
, the function returns the standard deviation map, and if type="SNR"
, the SNR map is returned.
mask_radius
is the radius of the mask around each pixel, and Dr
is the radial width of the annulus.
if __name__ == '__main__':
#[...]
filename = os.path.join(outputDir,"betpic131210J_GPIFMMFKL5.fits") # Change filename if needed
import astropy.io.fits as pyfits
hdulist = pyfits.open(filename)
FMMF = hdulist[1].data
hdulist.close()
from pyklip.kpp.stat.statPerPix_utils import get_image_stat_map_perPixMasking
FMMF_SNR = get_image_stat_map_perPixMasking(FMMF,
mask_radius = 7,
Dr = 2,
type = "SNR")
The output FMMF_SNR
is the calibrated SNR map that can be used for planet detection.
Planet Evidence Module¶
This tutorial will provide instructions on using the nested sampling implementation to compute the evidence of a detection obtained with the KLIPFM technique being a true point source or just residual noise along with posterior distributions of the source parameters. This module is useful for providing evidence in a Bayesian framework for a point source detection which by eye may appear uncertain. Planet Evidence runs MultiNest with the forward model (i.e. a planet is present at the guessed planet’s location) and with the null hypothesis (i.e. just noise at the guessed planet’s location).
Attribution¶
The PlanetEvidence module has been developed and implemented in pyKLIP by Graça Rocha and Jacob Golomb with contributions from Jason Wang. If you use this module please cite the article by Golomb, J., Rocha, G., et al. 2019.
Requirements¶
To use the Planet Evidence module you must have:
 Data to run PSF subtraction on
 A model or data of the instrumental PSF
 A good guess of the position of the planet (a center of light centroid routine should get the astrometry to a pixel)
 Pymultinest (click link for installation instructions)
 corner
Data Reduction to Prepare the Image and Forward Model¶
To prepare the images for analysis, follow the instructions in ForwardModel Astrometry and Photometry up until the pyklip.fitpsf.FMAstrometry
is called (in the ‘Fitting to the Planet PSF’ section).
Fitting the Planet PSF¶
To do the nested sampling process, we will create pyklip.fitpsf.PlanetEvidence
object and feed in the guessed separation and guessed parallactic angle. Then, stamps of thee data and forward model are created (fit.data_stamp
and fit.fm_stamp
, respectively). MultiNest works by saving its sampling files in a specified directory. If no folder in your working directory has the name you specify, a folder by that name will be created. Because Planet Evidence runs two sampling processes (one for the forward model/planet model, one for the null hypothesis/noise model), we specify that we want the forward model sampling files to be saved in a directory ‘chains/’ with the prefix ‘Planet’ and the null hypothesis sampling files saved in ‘chains/’ with the prefix ‘Null’. A local directory for sampling_outputdir
must be specified. If the specified directory does not already exist, it will be automatically created. If nothing is specified for fm_basename
and null_basename
they will automatically revert to ‘Planet’ and ‘Null’, respectively, overwriting any files in your specified directory that have that same prefix!
Note there is also an optional l_only
boolean argument when calling PlanetEvidence. This is to determine if the null hypothesis should be a 1 or 3parameter model. In the 1parameter model, we set l_only = True
, so the null hypothesis model only samples for the correlation length scale parameter (i.e. the scale of the spatially correlated noise). Setting l_only = False
sets the null hypothesis model to a 3parameter model where the correlation length scale as well as positional parameters are sampled. In this example, we will use the 3parameter model.
# create PlanetEvidence object that does Multinest sampling, set fm_basename and null_basename to the prefixes you want each sampler to save in output_dir
output_dir = 'chains/' #will save sampling files in the specified directory
fit = fitpsf.PlanetEvidence(guesssep, guesspa, 13, sampling_outputdir=output_dir, l_only = False, fm_basename='Planet', null_basename='Null')
# generate FM stamp
# padding should be greater than 0 so we don't run into interpolation problems
fit.generate_fm_stamp(fm_frame, [fm_centx, fm_centy], padding=5)
# generate data_stamp stamp
# not that dr=4 means we are using a 4 pixel wide annulus to sample the noise for each pixel
# exclusion_radius excludes all pixels less than that distance from the estimated location of the planet
fit.generate_data_stamp(data_frame, [data_centx, data_centy], dr=4, exclusion_radius=10)
Next we need to choose the Gaussian process kernel. We currently only support the Matern (ν=3/2) and square exponential kernel, so we will pick the Matern kernel here.
# set kernel, no read noise
corr_len_guess = 3.
corr_len_label = r"$l$"
fit.set_kernel("matern32", [corr_len_guess], [corr_len_label])
Priors are necessary for MultiNest to know where in parameter space to sample. We support the use of uniform priors, with those for x and y being flat in linear space and for flux scaling and kernel parameters being flat in log space. In the function below, we will set the boundaries for these priors. The first two values are for x and y and they say how far away (in pixels) from the guessed position of the planet MultiNest will sample. For the flux_range
and corr_len_range
parameters, the values are the magnitude MultiNest can sample from the guessed value (e.g. a value of 1 means we allow a factor of 10 variation in the value). The flux scaling parameter (α) is a multiplicative correction to guessflux
for measuring the photometry. The correlation length (l) is a Gaussian process hyperparameter.
# set bounds
x_range = 1.5 # pixels
y_range = 1.5 # pixels
flux_range = 1. # flux can vary by an order of magnitude
corr_len_range = 1. # between 0.3 and 30
fit.set_bounds(x_range, y_range, flux_range, [corr_len_range])
Running the Sampler¶
Now we are ready to run MultiNest on the data. We use MultiNest to sample parameter space for the position parameters (x, y), the flux scale, and the correlation length scale. This will sample the “fitting region” around the initial guess values for the position of the planet (specified above). For the forward model case, we sample the four aforementioned parameters within the specified prior range, whereas the null hypothesis case can either be a 3 parameter or 1 parameter noise model. Thus, the evidence for the forward model will be the evidence of the point source detected while the evidence for the null hypothesis will be the evidence that the detection is just noise.
Running the two samplers is simple:
#Run the pymultinest fit
fit.multifit()
This will save the files sampler necessary for analysis in the specified directory (in this case we’ve set it to a local /chains/ directory). The forward model sampling files will have the prefix set by fm_basename
and the null hypothesis model sampling files will have the prefix set by null_basename
.
Analyzing the Data¶
For analyzing the data, the posterior distributions of each of the parameters will be of interest. To visualize these, we provided a wrapper to output the corner plots.
#Output corner plots from these fits (first figure is the H1 hypothesis, followed up H0)
fit.fit_plots()
Here is an example from the beta Pic b Jband GPI data, the posteriors for the forward model followed by the null hypothesis model:
As can be seen by the lack of structure in the first corner plots, the forward model hypothesis converged on the location of the planet, as well as the flux scale and correlation length parameters. The 3parameter null hypothesis model posteriors are uniform in space, but not for the correlation length scale, as there is spatiallycorrected noise present.
The most important benefit of using nested sampling is the ability to obtain the evidence for each model. We will output the statistics from both model runs into a tuple, which will contain two dictionaries.
#Store the Multinest statistics in a tuple called "evidence". evidence[0] will be a dict
#with the statistics for the H1 hypothesis. evidence[1] is the same, for the null hypothesis
evidence = fit.fit_stats()
We have stored the statistics from the forward model run in a dictionary in the 0th element of the ‘evidence’ tuple and those of the null hypothesis run in the 1st element. These dictionaries include the log evidence and the marginal distribution statistics for each parameter. We can store the relevant statistics from these outputs into thee appropriate variables by accessing the information in the dictionaries.
#Forward model evidence
fm_evidence = evidence[0]['nested sampling global logevidence']
#forward model parameter distributions, containing the median and percentiles for each
fm_posteriors = evidence[0]['marginals']
#Null model evidence
null_evidence = evidence[1]['nested sampling global logevidence']
#null parameter distributions, containing the median and percentiles for each
null_posteriors = evidence[1]['marginals']
Note
The evidence value we use here is the nested sampling global log evidence. If interested, evidence
also includes local log evidence.
Each posterior variable now contains a list, each element of which contains the statistics its respective parameter. For example, fm_posteriors[0]
contains the marginal posterior statistics for the x parameter, fm_posteriors[1]
contains those of the y parameter, and so on. null_posteriors
will have only 3 elements, as it is from a 3 parameter model (no flux scale parameter).
Computing the ratio of the evidences gives the relative likelihood between the two models. For example, we will now quantify which model is favored for our runs on beta Pictoris b, and by how much.
#Compute evidence ratio for forward model relative to null hypothesis model
evidence_ratio = np.exp(fm_evidence)/np.exp(null_evidence)
# or
#evidence_ratio= np.exp (fm_evidence  null_evidence)
Where the exponentiation is necessary to get the evidences out of ln space. The evidence ratio in this case is on the order of e^175. This result can be interpreted by consulting a resource such as Table 1 in Trotta (2008) . (Note that this table uses the ln of the evidence ratio.) In our case, this value is on the order of 175, indicating strong evidence in favor of the forward model (planet) model over the null (noise) model.
We can also examine how the residuals appear in the bestfit region when compared with the input data. To do this, we will use the command fm_residuals
as such:
This will also print the signaltonoise ratio (SNR) from the residuals, taking the maximum pixel value from the bestfit model and dividing it by the standard deviation of the residuals.
FowardModel Astrometry and Photometry¶
pyKLIP offers PSF fitting as the most optimal method for extraction of planet astrometry and photometry. Astrometry/photometry of directly imaged exoplanets is challenging since PSF subtraction algorithms (like pyKLIP) distort the PSF of the planet. Pueyo (2016) provide a technique to forward model the PSF of a planet through KLIP. Taking this forward model, you could fit it to the data, but you would underestimate your errors because the noise in direct imaging data is correlated (i.e. each pixel is not independent). To account for the correlated nature of the noise, we use Gaussian process regression to model and account for the correlated nature of the noise. This allows us to obtain accurate measurements and accurate uncertainties on the planet parameters.
The code was originally designed for the Bayesian KLIPFM Astorometry technique (BKA) that is described in Wang et al. (2016) to obtain one milliarcsecond astrometry on β Pictoris b. Since then, it has be extended to also do maximum likelihood fitting in a frequentist framework that can also obtain uncertainties. A frequentist framework is appropriate when astrometric calibration uncertainties are also derived in a frequentist procedure.
Requirements¶
You need the following pieces of data to forward model the data. (If you have a model and some data to fit to, you can skip the section on using KLIPFM to generate a forawrd model PSF.)
 Data to run PSF subtraction on
 A model or data of the instrumental PSF
 A good guess of the position of the planet (a center of light centroid routine should get the astrometry to a pixel)
 For IFS data, an estimate of the spectrum of the planet (it does not need to be very accurate, and 20% errors are fine)
If you want to run BKA, you need the additional packages installed, which should be available readily:
Generating instrumental PSFs for GPI¶
A quick aside for GPI spectral mode data, here is how to generate the instrumental PSF from the satellite spots.
import glob
import numpy as np
import pyklip.instruments.GPI as GPI
# read in the data into a dataset
filelist = glob.glob("path/to/dataset/*.fits")
dataset = GPI.GPIData(filelist)
# generate instrumental PSF
boxsize = 17 # we want a 17x17 pixel box centered on the instrumental PSF
dataset.generate_psfs(boxrad=boxsize//2) # this function extracts the satellite spots from the data
# now dataset.psfs contains a 37x25x25 spectral cube with the instrumental PSF
# normalize the instrumental PSF so the peak flux is unity
dataset.psfs /= (np.mean(dataset.spot_flux.reshape([dataset.spot_flux.shape[0] // 37, 37]),
axis=0)[:, None, None])
Here is an exmaple using three datacubes from the publicly available GPI data on beta Pic. Note that the wings of the PSF are somewhat noisy, due to the fact the speckle noise in Jband is high near the satellite spots. However, this should still give us an acceptable instrumental PSF.
Forward Modelling the PSF with KLIPFM¶
With an estimate of the planet position, the instrumental PSF, and, if applicable, an estimate of the spectrum,
we can use the pyklip.fm
implementation of KLIPFM and pyklip.fmlib.fmpsf.FMPlanetPSF
extension to
forward model the PSF of a planet through KLIP.
First, let us initalize pyklip.fmlib.fmpsf.FMPlanetPSF
to forward model the planet in our data.
For GPI, we are using normalized copies of the satellite spots as our input PSFs, and because of that, we need to pass in
a flux conversion value, dn_per_contrast
, that allows us to scale our guessflux
in contrast units to data units. If
you are not using normalized PSFs, dn_per_contrast
should be the factor that scales your input PSF to the flux of the
unocculted star. If your input PSF is already scaled to the flux of the stellar PSF, dn_per_contrast
is optional
and should not actually be passed into the function.
# setup FM guesses
# You should change these to be suited to your data!
numbasis = np.array([1, 7, 100]) # KL basis cutoffs you want to try
guesssep = 30.1 # estimate of separation in pixels
guesspa = 212.2 # estimate of position angle, in degrees
guessflux = 5e5 # estimated contrast
dn_per_contrast = your_flux_conversion # factor to scale PSF to star PSF. For GPI, this is dataset.dn_per_contrast
guessspec = your_spectrum # should be 1D array with number of elements = np.size(np.unique(dataset.wvs))
# initialize the FM Planet PSF class
import pyklip.fmlib.fmpsf as fmpsf
fm_class = fmpsf.FMPlanetPSF(dataset.input.shape, numbasis, guesssep, guesspa, guessflux, dataset.psfs,
np.unique(dataset.wvs), dn_per_contrast, star_spt='A6',
spectrallib=[guessspec])
Note
When executing the initializing of FMPlanetPSF, you will get a warning along the lines of “The coefficients of the spline returned have been computed as the minimal norm leastsquares solution of a (numerically) rank deficient system.” This is completely normal and expected, and should not be an issue.
Next we will run KLIPFM with the pyklip.fm
module. Before we run it, we will need to pick our
PSF subtraction parameters (see the Basic KLIP Tutorial with GPI for more details on picking KLIP parameters).
For our zones, we will run KLIP only on one zone: an annulus centered on the guessed location of the planet with
a width of 30 pixels. The width just needs to be big enough that you see the entire planet PSF.
# PSF subtraction parameters
# You should change these to be suited to your data!
outputdir = "." # where to write the output files
prefix = "betpic131210jfmpsf" # fileprefix for the output files
annulus_bounds = [[guesssep15, guesssep+15]] # one annulus centered on the planet
subsections = 1 # we are not breaking up the annulus
padding = 0 # we are not padding our zones
movement = 4 # we are using an conservative exclusion criteria of 4 pixels
# run KLIPFM
import pyklip.fm as fm
fm.klip_dataset(dataset, fm_class, outputdir=outputdir, fileprefix=prefix, numbasis=numbasis,
annuli=annulus_bounds, subsections=subsections, padding=padding, movement=movement)
This will now run KLIPFM, producing both a PSF subtracted image of the data and a forwardmodelled PSF of the planet at the gussed location of the planet. The PSF subtracted image as the “klipped” string in its filename, while the forwardmodelled planet PSF has the “fmpsf” string in its filename.
Fitting the Planet PSF¶
Now that we have the forwardmodeled PSF and the data, we can fit the model PSF to the data and obtain the astrometry/photometry of the point source. To do it the Bayesian way with MCMC or the frequentist way with maximum likelihood is very similar in the code, and will produce similar results as both frameworks use the forwardmodeled PSF and Gaussian process regression. The difference is interpretation, and determining which framework makes sense for your analysis (e.g., if the instrument calibration was done in a Bayesian sense, then doing the fit in a Bayesian way is appropriate to properly combine calibration uncertainties with measurement uncertainties).
First, let’s read in the data from our previous forward modelling. We will take the collapsed KL mode cubes, and select the KL mode cutoff we want to use. For the example, we will use 7 KL modes to model and subtract off the stellar PSF.
import os
import astropy.io.fits as fits
# read in outputs
output_prefix = os.path.join(outputdir, prefix)
fm_hdu = fits.open(output_prefix + "fmpsfKLmodesall.fits")
data_hdu = fits.open(output_prefix + "klippedKLmodesall.fits")
# get FM frame, use KL=7
fm_frame = fm_hdu[1].data[1]
fm_centx = fm_hdu[1].header['PSFCENTX']
fm_centy = fm_hdu[1].header['PSFCENTY']
# get data_stamp frame, use KL=7
data_frame = data_hdu[1].data[1]
data_centx = data_hdu[1].header["PSFCENTX"]
data_centy = data_hdu[1].header["PSFCENTY"]
# get initial guesses
guesssep = fm_hdu[0].header['FM_SEP']
guesspa = fm_hdu[0].header['FM_PA']
We will generate a pyklip.fitpsf.FMAstrometry
object that will handle all of the fitting processes.
The first thing we will do is create this object, and feed it in the data and forward model. It will use them to
generate stamps of the data and forward model which can be accessed using fit.data_stmap
and fit.fm_stamp
respectively. When reading in the data, it will also generate a noise map for the data stamp by computing the standard
deviation around an annulus, with the planet masked out. Here, we will also specify whether we will use a maximum likliehood
or MCMC technique to derive the bestfit and uncertainties using the argument method="mcmc"
(default)
or method="maxl"
.
import pyklip.fitpsf as fitpsf
# create FM Astrometry object that does MCMC fitting
fit = fitpsf.FMAstrometry(guesssep, guesspa, 13, method="mcmc")
# alternatively, could use maximum likelihood fitting
# fit = fitpsf.FMAstrometry(guesssep, guesspa, 13, method="maxl")
# generate FM stamp
# padding should be greater than 0 so we don't run into interpolation problems
fit.generate_fm_stamp(fm_frame, [fm_centx, fm_centy], padding=5)
# generate data_stamp stamp
# not that dr=4 means we are using a 4 pixel wide annulus to sample the noise for each pixel
# exclusion_radius excludes all pixels less than that distance from the estimated location of the planet
fit.generate_data_stamp(data_frame, [data_centx, data_centy], dr=4, exclusion_radius=10)
Next we need to choose a Gaussian process kernel to model the correlated noise in our data. We currently only support the Matern (ν=3/2) and square exponential kernel, so we will pick the Matern kernel here. Note that there is the option to add a diagonal (i.e. read/photon noise) term to the kernel, but we have chosen not to use it in this example. If you are not dominated by speckle noise (i.e. around fainter stars or further out from the star), you should enable the read noies term.
# set kernel, no read noise
corr_len_guess = 3.
corr_len_label = r"$l$"
fit.set_kernel("matern32", [corr_len_guess], [corr_len_label])
MCMC takes a Bayesian approach to estimating our parameters of interest by approximating their posterior distributions i.e. their best fit values given the data. In order to perform this type of Bayesian analysis you will need to define priors, which are your own estimations of what these parameter values might be. Since we usually don’t know much about the data, we will use uniform priors, essentially estimating the same probability for each value. All we will need to specify in this case are the bounds/range of values to the uniform prior. This same code can also be used to specify parameter bounds for the maximum likelihood approach, but setting bounds is not required for that technique. The priors in the x/y position will be flat in linear space, and the priors on the flux scaling and kernel parameters will be flat in log space, since they are scale parameters. In the function below, we will set the boundaries of the priors. The first two values are for x/y and they basically say how far away (in pixels) from the guessed position of the planet can the chains wander. For the rest of the parameters, the values specify how many orders of magnitude the chains can go from the guessed value (e.g. a value of 1 means we allow a factor of 10 variation in the value).
# set bounds
x_range = 1.5 # pixels
y_range = 1.5 # pixels
flux_range = 1. # flux can vary by an order of magnitude
corr_len_range = 1. # between 0.3 and 30
fit.set_bounds(x_range, y_range, flux_range, [corr_len_range])
Finally, we are set up to fit to the data. The fit_astrometry()
function is used for both MCMC and maximum
likelihood.
In this example, we will fit for four parameters.
The RA offset and Dec offset are what we are interested in for the purposes of astrometry. The flux scaling
paramter (α) is a multiplicative correction to guessflux
for measuring the photometry.
The correlation length (l) is a Gaussian process hyperparameter. If we had included read noise,
it would have been a fifth parameter.
As the analysis diverges, we will discuss Maximum Likelihood and Bayesian MCMC Analysis separately.
Bayesian MCMC Analysis¶
To run the MCMC sampler (using the emcee package), we need to specify the number of walkers (the number of Markov chains that explore your parameter space), number of steps each walker takes (how many new values of your parameter each chain should explore), and the number of production steps the walkers take (the number of steps we let each chain take to burnin). We also can specify the number of threads to use. If you have not turned BLAS and MKL off, you probably only want one or a few threads, as MKL/BLAS automatically parallelizes the likelihood calculation, and trying to parallelize on top of that just creates extra overhead.
# run MCMC fit
fit.fit_astrometry(nwalkers=100, nburn=200, nsteps=800, numthreads=1)
fit.sampler
stores the emcee.EnsembleSampler
object which contains the full chains and other MCMC fitting information.
For MCMC, we want to check to make sure all of our chains have converged by plotting them. That is, we need to ensure that there are no large scale movements, and that the entire parameter space is being sampled evenly. The maximum likelihood technique skips this step.
import matplotlib.pylab as plt
fig = plt.figure(figsize=(10,8))
# grab the chains from the sampler
chain = fit.sampler.chain
# plot RA offset
ax1 = fig.add_subplot(411)
ax1.plot(chain[:,:,0].T, '', color='k', alpha=0.3)
ax1.set_xlabel("Steps")
ax1.set_ylabel(r"$\Delta$ RA")
# plot Dec offset
ax2 = fig.add_subplot(412)
ax2.plot(chain[:,:,1].T, '', color='k', alpha=0.3)
ax2.set_xlabel("Steps")
ax2.set_ylabel(r"$\Delta$ Dec")
# plot flux scaling
ax3 = fig.add_subplot(413)
ax3.plot(chain[:,:,2].T, '', color='k', alpha=0.3)
ax3.set_xlabel("Steps")
ax3.set_ylabel(r"$\alpha$")
# plot hyperparameters.. we only have one for this example: the correlation length
ax4 = fig.add_subplot(414)
ax4.plot(chain[:,:,3].T, '', color='k', alpha=0.3)
ax4.set_xlabel("Steps")
ax4.set_ylabel(r"$l$")
Here is an example using three cubes of public GPI data on beta Pic.
For MCMC, we can also plot the corner plot to look at our posterior distribution and correlation between parameters. We can use the peak of each posterior distribution to estimate the best fit value of each parameter.
fig = plt.figure()
fig = fit.make_corner_plot(fig=fig)
Hopefully the corner plot does not contain too much structure (the posteriors should be roughly Gaussian). In the example figure from three cubes of GPI data on beta Pic, the residual speckle noise has not been very whitened, so there is some asymmetry in the posterior, which represents the local strucutre of the speckle noise. These posteriors should become more Gaussian as we add more data and whiten the speckle noise.
To continue, skip to Output of FMAstrometry.
Maximum Likelihood¶
For maximum likelihood, we can start with the same FMAstrometry
setup up until fit_astrometry()
.
The execution of fit_astrometry()
will be completely different.
The algorithm with use a NelderMead optimization to find the global maximum, as it is a fairly
robust method. Then, it will use BFGS
algorithm
from scipy.optimize.minimze
that can approximate the Hessian inverse during the optimization. The Hessian
inverse can be used as an approximation of the covariance matrix for the fitted parameters. We take the diagonal
terms of the Hessian inverse as the variance in each parameter.
# if you're running a maxlikelihood fit
fit.fit_astrometry()
We also store the Hessian inverse in fit.hess_inv
.
Note that the algorithm we use is unable to estimate the uncertainity on the Gaussian parameter
hyperparameters, so those entries with all be 0.
Note
If you get a warning here about the optimizer not converging, this means that the BFGS algorithm in
scipy.optimize.minimize
was unable to converge on estimating the Hessian inverse, and thus
the reported uncertainties are likely unreliable. We are working on a solution to this. We recommend
using fake planet injection instead to estimate your uncertainities if this happens.
Output of FMAstrometry¶
Here are some fields to access the fit. Each field is a
pyklip.fitpsf.ParamRange
object, which has fields bestfit
, error
, and error_2sided
. Here, error
is the average 1sigma error, and error_2sided
lists the positive and negative errors separately. Notice the names all begin with “raw”, which is because these are the values from just fitting the data, and do not include instrument calibration.
fit.raw_RA_offset
: RA offset of the planet from the star (in pixels)fit.raw_Dec_offset
: Dec offset of the planet from the star (in pixels)fit.raw_flux
: Multiplicative factor to scale the model flux to match the datafit.covar_params
: hyperparameters on the Gaussian process. This is a list with length equal to the number of hyperparameters.
Note that since fit.covar_params is a list dependent on user input, you need to index that list in order to see the bestfit values for each element. For example, fit.covar_params[0].bestfit will output the bestfit for the first element in the list.
Generally, it is good to look at the fit visually, and examine the residual plot for structure that might be indicative of a bad fit or systematics in either the data or the model.
fig = plt.figure()
fig = fit.best_fit_and_residuals(fig=fig)
And here is the example from the three frames of beta Pic b Jband GPI data:
The data and best fit model should look pretty close, and the residuals hopefully do not show any obvious strcuture that was missed in the fit. The residual ampltidue should also be consistent with noise. If that is the case, we can use the best fit values for the astrometry of this epoch.
The best fit values from the MCMC give us the raw RA and Dec offsets for the planet. We will still need to fold in uncertainties
in the star location and calibration uncertainties. To do this, we use pyklip.fitpsf.FMAstrometry.propogate_errs()
to
include these terms and obtain our final astrometric values. All of the infered parameters are fields
that can be accessed (see pyklip.fitpsf.FMAstrometry
) and each field is a pyklip.fitpsf.ParamRange
object. Here is a brief overview of the fields:
fit.RA_offset
: RA offset of the planet from the star (angular units)fit.Dec_offset
: Dec offset of the planet from the star (angular units)fit.sep
: Radial separation of the planet from the star (angular units)fit.PA
: Position angle of the planet (i.e., Angle from North towards East; in degrees).
There is currently no infrastrucutre to propogate photometric calibration uncertainities in, so it will need to be done by hand.
fit.propogate_errs(star_center_err=0.05, platescale=GPI.GPIData.lenslet_scale*1000, platescale_err=0.007, pa_offset=0.1, pa_uncertainty=0.13)
# show what the raw uncertainites are on the location of the planet
print("\nPlanet Raw RA offset is {0} +/ {1}, Raw Dec offset is {2} +/ {3}".format(fit.raw_RA_offset.bestfit, fit.raw_RA_offset.error,
fit.raw_Dec_offset.bestfit, fit.raw_Dec_offset.error))
# Full error budget included
print("Planet RA offset is at {0} with a 1sigma uncertainity of {1}".format(fit.RA_offset.bestfit, fit.RA_offset.error))
print("Planet Dec offset is at {0} with a 1sigma uncertainity of {1}".format(fit.Dec_offset.bestfit, fit.Dec_offset.error))
# Propogate errors into separation and PA space
print("Planet separation is at {0} with a 1sigma uncertainity of {1}".format(fit.sep.bestfit, fit.sep.error))
print("Planet PA at {0} with a 1sigma uncertainity of {1}".format(fit.PA.bestfit, fit.PA.error))
Correcting for Coronagraphic Throughput¶
Coronagraphs have measurable effects on the planet fluxes that we measure. Typically, we can expect them to diminish the overall image flux at separations closer to host star, while larger separations remain relatively unaffected. In order to improve the accuracy of our forward model, pyKLIP allows users to account for this coronahgraphic effect on planet light transmission when initializing the fmpsf.FMPlanetPSF class. This feature can be accessed by providing the optional argument ‘field_dependent_correction’, which accepts a user provided function to correct for coronagraphic throughput. Each coronagraph has its own transmission profile, a measure of how its throughput changes as a function of distance from the center. As an example of how this would be incoporated, we’ll use the transmission profile of the JWST/NIRCAM MASK210 coronagraph (obtained from the Occulting Masks section of this NIRCAM webpage: https://jwstdocs.stsci.edu/nearinfraredcamera/nircaminstrumentation/nircamcoronagraphicoccultingmas
First, we’ll create a function that performs the coronagraphic throughput correction. It should accept three arguments: the region or ‘stamp’ of your fake planet, the physical ‘x’ separation of each pixel in the stamp from the coronagraph center, and the physical ‘y’ separation of each pixel in the stamp from the coronagraph center. It should then use the transmission profile of the relevant coronagraph to scale the input stamp by the necessary amount, then output the throughput corrected stamp. Be sure to read in your coronagraphic transmission profile with columns for ‘transmission’ and ‘distance from the star (in pixels)’ prior to creating the function.
# Read in transmission profile first
def transmission_correction(input_stamp, input_dx, input_dy):
"""
Args:
input_stamp (array): 2D array of the region surrounding the fake planet injection site
input_dx (array): 2D array specifying the x distance of each stamp pixel from the center
input_dy (array): 2D array specifying the y distance of each stamp pixel from the center
Returns:
output_stamp (array): 2D array of the throughput corrected planet injection site.
"""
# Calculate the distance of each pixel in the input stamp from the center
distance_from_center = np.sqrt((input_dx)**2+(input_dy)**2)
# Select the relevant columns from the coronagraph's transmission profile
transmission = transmission_profile['throughput']
radius = transmission_profile['distance']
# Interpolate to find the transmission value for each pixel in the input stamp
transmission_of_stamp = np.interp(distance_from_center, radius, transmission)
# Reshape the interpolated array to have the same dimensions as the input stamp
transmission_of_stamp = transmission_of_stamp.reshape(input_stamp.shape)
# Make the throughput correction
output_stamp = transmission_of_stamp*input_stamp
return output_stamp
Now we can include the function as an optional argument in the pyklip.fmlib.fmpsf.FMPlanetPSF
class.
The rest of the procedure can proceed unchanged.
import pyklip.fmlib.fmpsf as fmpsf
fm_class = fmpsf.FMPlanetPSF(dataset.input.shape, numbasis, guesssep, guesspa, guessflux, dataset.psfs,
np.unique(dataset.wvs), dn_per_contrast, star_spt='A6',
spectrallib=[guessspec], field_dependent_correction = transmission_correction)
Spectrum Extraction using extractSpec FM¶
This document describes with an example how to use KLIPFM to extract a spectrum, described in Pueyo et al. (2016) to account the effect of the companion signal in the reference library when measuring its spectrum.
Set up:¶
Here we will just read in the dataset and grab the instrumental PSF. The example code here shows how it is done with GPI, but you will want to refer to the Instrument Tutorials for the instrument you are working. As the code notes, it is important what the units of your instrumental PSF is in, as the code will return the spectrum relative to the input PSF model.
import glob
import numpy as np
import pyklip.instruments.GPI as GPI
import pyklip.fmlib.extractSpec as es
import pyklip.fm as fm
import pyklip.fakes as fakes
import matplotlib.pyplot as plt
files = glob.glob("\path\to\dataset\*.fits")
dataset = GPI.GPIData(files, highpass=True)
# Need to specify a model PSF (either via this method, or any other way)
model_psfs = dataset.generate_psf_cube(20)
# in this case model_psfs has shape (N_lambda, 20, 20)
# The units of your model PSF are important, the return spectrum will be
# relative to the input PSF model, see next example
###### Useful values based on dataset ######
N_frames = len(dataset.input)
N_cubes = np.size(np.unique(dataset.filenums))
nl = N_frames // N_cubes
Calibrating stellar flux for GPI example:¶
Converting to contrast units for GPI data is done using the flux of the satellite spots. The GPI dataset object has attribute spot_flux that represent the average peak flux of the four spots. The normalization factor is computed by dividing the spot flux spectrum by the ratio between the stellar flux and the spot flux (stored in spot_ratio) and adjusting for the ratio between the peak and the sum of the spot PSF.
For any instrument you can scale your model PSF by its respective calibration factors if the model PSF is not already scaled to be the flux of the star. Alternatively, you can choose to skip this step and calibrate your spectrum into astrophysical units as the very end.
GPI Example:
# First set up a PSF model and sums  this is necessary for GPI because
# dataset.spot_flux contains peak values of the satellite spots and we
# have to correct for the full aperture.
PSF_cube = dataset.psfs
model_psf_sum = np.nansum(PSF_cube, axis=(1,2))
model_psf_peak = np.nanmax(PSF_cube, axis=(1,2))
# Now divide the sum by the peak for each wavelength slice
aper_over_peak_ratio = model_psf_sum/model_psf_peak
# startospot calibration factor
band = dataset.prihdrs[0]['APODIZER'].split('_')[1]
spot_to_star_ratio = dataset.spot_ratio[band]
spot_peak_spectrum = \
np.median(dataset.spot_flux.reshape(len(dataset.spot_flux)//nl, nl), axis=0)
calibfactor = aper_over_peak_ratio*spot_peak_spectrum / spot_to_star_ratio
# calibrated_PSF_model is the stellar flux in counts for each wavelength
calibrated_PSF_model = PSF_cube*calibfactor
This is your model_psf for generating the forward model and will return the spectrum in contrast units relative to the star.
Computing the forward model and recovering the spectrum with invert_spect_fmodel¶
We will use the ExtractSpec class to forward model the PSF of the planet and the invert_spect_fm function in pyklip.fmlib.extractSpec to recover the spectrum. invert_spect_fm returns a spectrum in units relative to the input PSF.
These are the numbers you change:
###### parameters you specify ######
pars = (45, 222) # replace with known separation and pa of companion
planet_sep, planet_pa = pars
numbasis = [50,] # "k_klip", this can be a list of any size.
# a forward model will be computed for each element.
num_k_klip = len(numbasis) # how many k_klips running
maxnumbasis = 100 # Max components to be calculated
movement = 2.0 # aggressiveness for choosing reference library
stamp_size = 10.0 # how big of a stamp around the companion in pixels
# stamp will be stamp_size**2 pixels
numthreads=4 # number of threads, machine specific
spectra_template = None # a template spectrum, if you want
Generating the forward model with pyKLIP:
###### The forward model class ######
fm_class = ExtractSpec(dataset.input.shape,
numbasis,
planet_sep,
planet_pa,
calibrated_PSF_model,
np.unique(dataset.wvs),
stamp_size = stamp_size)
###### Now run KLIP! ######
fm.klip_dataset(dataset, fm_class,
fileprefix="fmspect",
annuli=[[planet_sepstamp_size,planet_sep+stamp_size]],
subsections=[[(planet_pastamp_size)/180.*np.pi,\
(planet_pa+stamp_size)/180.*np.pi]],
movement=movement,
numbasis = numbasis,
maxnumbasis=maxnumbasis,
numthreads=numthreads,
spectrum=spectra_template,
save_klipped=True, highpass=True,
outputdir="\path\to\output")
# Forward model is stored in dataset.fmout, this is how it is organized:
# the klipped psf
klipped = dataset.fmout[:,:,1,:]
# The rest is the forward model, dimensions:
# [num_k_klip, N_frames, N_frames, stamp_size*stamp_size]
# If numbasis is a list, the first dimension will be the size of that list,
# a forward model calculated at each value of numbasis.
Now you can recover the spectrum:
# If you want to scale your spectrum by a calibration factor:
units = "scaled"
scaling_factor = my_calibration_factor
#e.g., for GPI this could be the startospot ratio
# otherwise, the defaults are:
units = "natural" # (default) returned relative to input PSF model
scale_factor=1.0 # (default) not used if units not set to "scaled"
exspect, fm_matrix = es.invert_spect_fmodel(dataset.fmout, dataset, units=units,
scaling_factor=scaling_factor,
method="leastsq")
# method indicates which matrix inversion method to use, they all tend
# to yield similar results when things are wellbehaved. Here are the options:
# "JB" matrix inversion adds up over all exposures, then inverts
# "leastsq" uses a leastsq solver.
# "LP" inversion adds over frames and one wavelength axis, then inverts
# (LP is not generally recommended)
The units of the spectrum, FM matrix, and klipped data are all in raw data units in this example. Calibration of instrument and atmospheric transmmission and stellar spectrum can be done via the input PSF model and optionally applying the scaling factor to invert_spect_fmodel. It can also be done after extracting the spectrum.
Simulating + recovering a simulated source¶
Example:
# PSF model template for each cube observation, copies of the PSF model:
inputpsfs = np.tile(calibrated_PSF_model, (N_cubes, 1, 1))
bulk_contrast = 1e2
fake_psf = inputpsfs*bulk_contrast
fake_flux = bulk_contrast*np.ones(dataset.wvs.shape)
#for ll in range(N_cubes):
# fake_flux[ll*nl:(ll+1)*nl] = exspect[0, :]
pa = planet_pa+180
tmp_dataset = GPI.GPIData(files, highpass=False)
fakes.inject_planet(tmp_dataset.input, tmp_dataset.centers, fake_psf,\
tmp_dataset.wcs, planet_sep, pa)
fm_class = es.ExtractSpec(tmp_dataset.input.shape,
numbasis,
planet_sep,
pa,
calibrated_PSF_model,
np.unique(dataset.wvs),
stamp_size = stamp_size)
fm.klip_dataset(tmp_dataset, fm_class,
fileprefix="fakespect",
annuli=[[planet_sepstamp_size,planet_sep+stamp_size]],
subsections=[[(pastamp_size)/180.*np.pi,\
(pa+stamp_size)/180.*np.pi]],
movement=movement,
numbasis = numbasis,
maxnumbasis=maxnumbasis,
numthreads=numthreads,
spectrum=spectra_template,
save_klipped=True, highpass=True,
outputdir="demo_output/")
fake_spect, fakefm = es.invert_spect_fmodel(tmp_dataset.fmout, tmp_dataset,
method="leastsq", units="scaled", scaling_factor=2.0)
Comparing the klipped data to the FM¶
You may want to look at how well your forward model represents the klipped data, measure residual error, etc. All the information you need is in the output of invert_spect_fmodel: the spectrum and FM matrix.
Recall the klipped data is in fmout:
klipped_data = tmp_dataset.fmout[:,:,1, :]
klipped_coadd = np.zeros((num_k_klip, nl, stamp_size*stamp_size))
for ll in range(N_cubes):
klipped_coadd = klipped_coadd + klipped_data[0, ll*nl:(ll+1)*nl, :]
# turn it back into a 2D arrat at each wavelength, k_klip
klipped_coadd.shape = [nl, int(stamp_size), int(stamp_size)]
# summed over each wavelength channel, but you can view them individually
plt.imshow(klipped_coadd.sum(axis=0), interpolation="nearest")
plt.colorbar()
Plot the forward model by taking the dot product with the extracted spectrum:
k=0 # choose which numbasis
fm_image_k = np.dot(fakefm[k,:,:], fake_spect[k].transpose())
# reshape the image back to 2D
fm_image_k = fm_image_k.reshape(nl, stamp_size, stamp_size)
# summed over each wavelength channel
plt.imshow(fm_image_k.sum(axis=0), interpolation="nearest")
plt.colorbar()
Calculating Errobars¶
One may want to calculate errorbars by injecting signals at an annulus of same separation as the real signal and measuring the spread of the recovered spectra (loop through the procedure above):
def recover_fake(files, position, fake_flux):
# We will need to create a new dataset each time.
# PSF model template for each cube observation, copies of the PSF model:
inputpsfs = np.tile(calibrated_PSF_model, (N_cubes, 1, 1))
bulk_contrast = 1e2
fake_psf = inputpsfs*fake_flux[0,None,None]
pa = planet_pa+180
tmp_dataset = GPI.GPIData(files, highpass=False)
fakes.inject_planet(tmp_dataset.input, tmp_dataset.centers, fake_psf,\
tmp_dataset.wcs, planet_sep, pa)
fm_class = es.ExtractSpec(tmp_dataset.input.shape,
numbasis,
planet_sep,
pa,
calibrated_PSF_model,
np.unique(dataset.wvs),
stamp_size = stamp_size)
fm.klip_dataset(tmp_dataset, fm_class,
fileprefix="fakespect",
annuli=[[planet_sepstamp_size,planet_sep+stamp_size]],
subsections=[[(pastamp_size)/180.*np.pi,\
(pa+stamp_size)/180.*np.pi]],
movement=movement,
numbasis = numbasis,
maxnumbasis=maxnumbasis,
numthreads=numthreads,
spectrum=spectra_template,
save_klipped=True, highpass=True,
outputdir="demo_output/")
fake_spect, fakefm = es.invert_spect_fmodel(tmp_dataset.fmout,
tmp_dataset, method="leastsq",
units="scaled", scaling_factor=2.0)
del tmp_dataset
return fake_spect
# This could take a long time to run
# Define a set of PAs to put in fake sources
npas = 11
pas = (np.linspace(planet_pa, planet_pa+360, num=npas+2)%360)[1:1]
# For numbasis "k"
# repeat the spectrum over each cube in the dataset
input_spect = np.tile(exspect[k,:], N_cubes)[0,:]
fake_spectra = np.zeros((npas, nl))
for p, pa in enumerate(pas):
fake_spectra[p,:] = recover_fake(files, (planet_sep, pa), input_spect)
Other details, like the forward model or klipped data for the injected signal could be useful.
If the real companion signal is too bright, the forward model may fail to capture all the flux It could be helpful to look at whether the recovered spectra for the simulated signal are evenly distributed around the simulated spectrum or if they are systematically lower flux:
offset[ii] = estim_spec[ii]  np.median(fake_spectra, axis=0)
Disk Foward Modelling (DiskFM)¶
This tutorial presents how to use the forward modelling routines specific to disk modelling and disk parameter retrieval.
Why DiskFM?¶
As noted in Pueyo (2016), “in practice Forward Modeling with disks is complicated by the fact that [it] cannot be simplified using a simple PSF as the astrophysical model: every hypothetical disk morphology must be explored”. Indeed, because of their complex geometries, the forward modelling have to be repeated a lot of time on disks with slightly different parameters. All these geometries are then compared to the klipped reduced image of the data, within an MCMC or a Chisquare wrapper.
However, once measured for a set of reduction parameters, the KarhunenLoeve (KL) basis do not change. One can save the KL vectors in a file once so they do not have to be recomputed every time. For a new disk model, the forward modelling is therfore only a array reformating and a matrix multiplication, which can be optimized to be only a few seconds. These routines are implemented in PyKLIP and showed on this page. DiskFM currently only supports KLIP ADI, SDI ADI+SDI and RDI reductions (but currently not RDI+ADI/SDI or NMF).
DiskFM Requirements¶
diskFM on a single model can be done on a personnal computer. However, the full parameter space exploration with the Chisquare or MCMC wrapper (out of the scope of this this tutorial) can be computationally very intensive, taking easily a few days, even parallized on a large server.
You also need the following pieces of data to forward model the data:
 A model of disk (this tutorial do not include disk modelling)
 The instrument PSF or a model of the PSF
 A set of to run PSF subtraction on
Set up¶
First import an instrument data set and convolve your 2D disk model by the instrument PSF:
import glob
import numpy as np
import pyklip.instruments.GPI as GPI
from astropy.convolution import convolve
from pyklip.fmlib.diskfm import DiskFM
import pyklip.fm as fm
# read in the data into a dataset
filelist = sorted(glob.glob("path/to/dataset/*.fits"))
dataset = GPI.GPIData(filelist)
# convolved the 2D disk model
disk_model_convolved = convolve(disk_model,instrument_psf, boundary = 'wrap')
Simple disk Forward Modelling¶
This code then shows how to initialize the DiskFM object and to do a forward modelling:
numbasis = [3, 10, 20] # different KL numbers we applied to the disk.
aligned_center=[140, 140] # indicate the position of the star
# initialize the DiskFM object
diskobj = DiskFM(dataset.input.shape, numbasis, dataset, disk_model_convolved,
aligned_center=aligned_center)
To run the forward modelling, just run:
fm.klip_dataset(dataset, diskobj, outputdir="output_path/", fileprefix="my_favorite_disk",
numbasis=numbasis, maxnumbasis=100, aligned_center=aligned_center,
mode='ADI', annuli=1, subsections=1, movement=1)
the code will save two fits files in outputdir, containing the klipped data and the associated disk forward model.
Most of the parameters implemented for psf forward model KLIP correction with pyklip can be used (see Picking KLIP Parameters for Disks) with the following exceptions:
 spectrum specific keywords (spectrum, flux_overlap, calibrate_flux)
 specific correction modes filtering the data (corr_smooth, highpass)
 other specific correction parameters (N_pix_sector, padding, annuli_spacing)
Mode parameter can be set only to ‘ADI’, ‘SDI’ and ‘ADI+SDI’.`aligned_center` is the position were the klip reduction will center the reduced image. The code will raise an error if it is not set to the position to which you set the star in your model.
DiskFM for MCMC or ChiSquare¶
For an MCMC or ChiSquare you can create the KL basis and then save them to forward model multiple models on a dataset without recomputing them every time. If you would like save the KL basis then you will need to signal it during the initialization of the DiskFM object, then apply fm.klip_dataset to measure and ave the forward model KL basis and parameters:
diskobj = DiskFM(dataset.input.shape, numbasis, dataset,
disk_model_convolved, aligned_center=aligned_center,
basis_filename = 'path/to/dir/klipbasis.h5', save_basis = True)
fm.klip_dataset(dataset, diskobj, outputdir="output_path/", fileprefix="my_favorite_disk",
numbasis=numbasis, maxnumbasis=100, aligned_center=aligned_center,
mode='ADI', annuli=1, subsections=1, movement=1)
Then, in any python session you can create a disk object and you can forward model disks with the loaded KL basis vectors without needing to measure this basis. The disk forward model will be output to fmout:
diskobj = DiskFM(dataset.input.shape, numbasis, dataset,
disk_model_convolved, aligned_center=aligned_center,
basis_filename='path/to/dir/klipbasis.h5', load_from_basis=True)
# do the forward modelling on a new model
new_disk_model_convolved=convolve(new_disk_model,instrument_psf, boundary='wrap')
diskobj.update_disk(new_disk_model_convolved)
fmout=diskobj.fm_parallelized()
# do the forward modelling on a third model
third_disk_model_convolved=convolve(third_disk_model,instrument_psf, boundary='wrap')
diskobj.update_disk(third_disk_model_convolved)
fmout=diskobj.fm_parallelized()
These last 3 lines are specifically what should be repeated withinin the MCMC or ChiSquare wapper.
Note that even if you have already created a DiskFM object to save the FM (ie even if you have runned diskFM with save_basis = True) in this python session, you still need to recreate the DiskFM object and load it (ie, you still need diskFM with load_from_basis = True).
In previous version, the dataset itself (input images) were not saved in the .h5 files, only the KL coeficients. This caused problems because you could run the same KL coefficients with slightly different datasets (for example the order of the frames were not identical) the code would run but provide wrong forward models. This has now beed solved and all the information necessary is saved inside the .h5 file, including intial frames and reduction paramters.
Speeding up DiskFM¶
The time is a key element here if you want to produce hundreds of thousands of forward modelling models. A smart choice of pyklip parameters can reduce the time for a single disk forward model:
 use OWA to limit only in the zone where the disk is.
 limit the number of sections (small annuli and subsections number).
 reduce the number of wavelengths. We recall this very usefull pyklip function to rebin quickly the number of wavelength, which should be applied immediatly after loading the dataset:
dataset.spectral_collapse(collapse_channels=1, align_frames=True)
 determine the best KL number parameters in advance and use only one, e.g.:
numbasis = [3]
Finally, due to the fact that numpy also parallelizes linear algebra routines across multiple cores, performance can actually sharply decrease when multiprocessing in a mcmc. Please read Note on parallelized performance on this subject.
Multiwavelength DiskFM¶
If you put a multiwavelenght dataset (e.g. IFS), the code will produce a multiwavelenght forward model. In that case, you can use a simple 2D model for the disk and the code will duplicate this model and apply the forward modelling separately on each of those at every wavelengths. Or you can use a 3D model (n_wl, x, y) and the code will apply the forward modelling separately on each of those at every wavelengths.
Alhtough everything we said in the previous sections on saving and loading the KL basis still apply multiwavelength disk forward modelling is long (it can take up to a few minutes or hours for a single forward modelling depending on the number of wavelengths) and we do not recommand to use this in an MCMC wrapper.
Full DiskFM tutorial¶
We recall all the steps in a single block
import glob
import numpy as np
import pyklip.instruments.GPI as GPI
from astropy.convolution import convolve
from pyklip.fmlib.diskfm import DiskFM
import pyklip.fm as fm
# read in the data into a dataset
filelist = sorted(glob.glob("path/to/dataset/*.fits"))
dataset = GPI.GPIData(filelist)
# in case of multiWL data, you might want to stack them first to speed things up
dataset.spectral_collapse(collapse_channels=1, align_frames=True)
numbasis = [3] # different KL numbers we applied to the disk.
aligned_center=[140, 140] # indicate the position of the star
# convolved the disk model
disk_model_convolved = convolve(disk_model,instrument_psf, boundary = 'wrap')
# initialize the DiskFM class
diskobj = DiskFM(dataset.input.shape, numbasis, dataset,
disk_model_convolved, aligned_center=aligned_center,
basis_filename = 'path/to/dir/klipbasis.pkl', save_basis = True)
# run klip to find and save FM basis
fm.klip_dataset(dataset, diskobj, outputdir="path/", fileprefix="my_favorite_disk",
numbasis=numbasis, maxnumbasis=100, aligned_center=aligned_center,
mode='ADI', annuli=2, subsections=1, minrot=3)
# 
# starting from here you can close the session and reopen later if you want
# 
# load Klip parameters and FM basis
diskobj = DiskFM(dataset.input.shape, numbasis, dataset,
disk_model_convolved, aligned_center=aligned_center,
basis_filename='path/to/dir/klipbasis.h5', load_from_basis=True)
# do the forward modelling on a new model
new_disk_model_convolved=convolve(new_disk_model,instrument_psf, boundary='wrap')
diskobj.update_disk(new_disk_model_convolved)
fmout=diskobj.fm_parallelized()
Developing for pyKLIP¶
Adding Modules¶
pyKLIP is fairly modular and allow you to add modules for various functionality like support for different instruments or different forward modelling methods. Here’s some guides on how to make your own.
Adding an Instrument Interface¶
To add an instrument interface, one needs to implement a subclass of the abstract class
pyklip.instruments.Instrument.Data
, overriding the abstract methods and fields required by the class. Some fields
may not not be relevant a particular instrument (e.g. wavelengths for a broadband instrument with only one channel) and we will
provide suggestions on what to default the value to.
For a simple example, look at how pyklip.instruments.Instrument.GenericData
implements the interface.
Now we will discuss what are the necessary steps to make your own instrument class.
Extending Instrument.Data¶
The first thing we need to do is create a new object that is a subclass of pyklip.instruments.Instrument.Data
. To do
this, we specify our new class inherits pyklip.instruments.Instrument.Data
and we need to call the __init__()
function
of the super class (i.e. pyklip.instruments.Instrument.Data
) as the first step of our new __init__()
function.
import pyklip.instruments.Instrument.Data as Data
class MyInstrument(Data):
"""
This is my new instrument class
"""
# Constructor
def __init__(self, args):
# initialize the super class
super(MyInstrument, self).__init__()
# run some more initialization code here
# the rest of the class goes here
Fields¶
For each required field, we use the following syntax to implement each required field. The field itself is just a wrapper
for an internal field named _field
, which should be set by the __init__()
function. This is a little clunky, but
allows python to ensure these fields are implemented. You should have one of the following code blocks for each required field:
@property
def field(self):
return self._field
@input.setter
def field(self, newval):
self._field = newval
Here are the fields that need to be implemented (some are optional and are marked as such). For the optional fields, you do not need to use the previous code block and make getter/setter methods; you can just set them like you normally set attributes.
input
¶This is the input data. It should be a numpy array of dimensions (Nframes, y, x). For dualband or IFS data, the wavelength dimension should be merged into the total number of files so that Nframes = Nwvs x Nfiles. Basically, this should always be a 3D cube, and we will use other bookkeeping fields to track the wavelengths and filenames for each frame.
output
¶This is where the output data gets stored. For initialization, set this equal to None so that the variable is defined. Othewise, you can expect that after pyKLIP, the 5D output cube gets stored here with dimension (KL, Nfiles, Nwvs, y, x) where KL is the numbe of KL cutoffs you specified, Nfiles is the number of unique filenames, Nwvs is the number of unique wavelengths.
fmout
(Optional)¶This is where the output of the forward modelling is stored (unless otherwise noted by the specific KLIPFM library). Refer to each KLIPFM feature for how to make sense of this data.
centers
¶This is the image centers for each input frame of data. It should be a numpy array of dimensions (Nframes, 2) where the second dimension is the (x,y) center for that frame. This is required for all datasets.
filenames
¶This is an array of filenames that correspond to each frame. Depending on how the data is formatted, filenames can be duplicated so that more than one frame has the same filename (e.g. each frame in an IFS datacube). For RDI, filenames are required, so that the PSF library will exclude the science frames from the PSF library that was built. If you really don’t care about them, set them to something generic (e.g. for the first frame, “0.fits”)
filenums
¶This is an array of file numbers so that each filename corresponds to a certain number. This allows for easier manipulating of frames, since it is easier to sort and slice numbers than strings. For easy implementation, make the first filename corespond to 0, the second correspond to 1, and so on.
wvs
¶This is an array of wavelengths, required for SDI to figure out how to rescale the speckles. If you are working with broadband data or generally wavelength agnostic, make this an array of the same number (e.g. use the central wavelength of the filter or set it all to 1).
PAs
¶This is an array of angles for each frame. This is defined as the angle needed to rotate the image north up, which means it is a combination of the parallactic angle (angle from North to zenith in the direction towards East) and any instrumental angles (e.g. the angle from the image to zenith). For ADI, PAs are required to determine field rotation. If you don’t know or don’t want image rotation, set this to an array of 0 with length equal to the number of frames.
flipx
(Optional)¶This specifies a boolean that at the end, when derotating and stacking the images, whether to flip the xaxis. By default, this is set to True. In the end we want to rotate images Northup Eastleft (i.e. East CCW of North). If your image starts out with East clockwise of North, then flipx should be set to True. Otherwise, set it to False.
wcs
¶This is an array of astropy.wcs.WCS objects that specifies the orientation of each input image. Since pyKLIP primiarily uses PAs
and
flipx
to figure out image orientation, this keyword isn’t strictly necessary, but could be good to have (e.g. pyklip.fakes
uses it for fake planet injection, and it is generally nice to have in your final PSF subtracted images). Note that WCS objects
have the method deepcopy
which allows you to replicate WCS headers, so if you have multiple frames that share the same WCS,
it is an easy to to give each frame its own WCS info. If you don’t have WCS info or don’t want to deal with it, set wcs
to an
array of None with length equal to the number of frames.
IWA
¶This is the inner working angle (radius, in pixels) of your data. pyKLIP will not reduce this part of your data and instead mask it as NaNs. If you don’t have an inner working angle well defined for your instrument, make this a parameter the user could pass in, guess one, or set it to 0. Note that this is a single number and not an array.
OWA
(Optional)¶This is the outer working angle for your data. By default this is None, and pyKLIP will use either the closest NaN to the center of the first image, or (if there are no NaNs) the edge of the image as the outer working angle. This is also a single number.
output
, output_centers
, output_wcs
(Requiredish)¶These fields corresponds to the output data, an array of dimensions (KLcutoffs, Nframes, y, x), the (x,y) center for each output
frame, an array of dimensions (Nframes, 2), and an array of WCS headers corresponding to the output images, which are typically
rotated Northup and Eastleft. The one you must explicitly define in your class is output
, but you should expect the other
two fields to also get populated after a KLIP reduction, so it could be useful to refer to those fields in savedata
. Note that
if you pass an array of None to wcs
, output_wcs
will also be an array of None. Also note that output_center
is the same
(x,y) coordinate repeated for each frame since the images are aligned together after KLIP.
Methods¶
Here are some required methods that need to be implemented. Of them, you definely want to make sure savedata()
works.
readdata(self, filepaths)
¶This function should be able to read in files from a list of filenames, compile them together, and set up the fields for this dataset.
Typically this function is called by the __init__()
function. If your data doesn’t come in a form where reading it in like this
is a very elegant solution, feel free to skip this function (by implementing this funciton with pass
as the only command) and
writing a different way to initalize your dataset.
savedata(self, filepath, data, klipparams=None, filetype="", zaxis=None, more_keywords=None)
¶This is the most important function to implement since it saves your pyKLIP reductions. The filepath is where to save the
file to, and the data is what to save. klipparams
is a string listing all of the pyKLIP parameters, and is typically
saved into the histroy of a FITS file. filetype
tells you the type of datacube this data is (i.e. “KL Mode Cube”,
“PSF Subtracted Spectral Cube”). For data with just one wavelength, the data will only be KL Mode cubes where the third
dimension is the KL mode cutoff. zaxis
is used for KL Mode cubes to specify the KL mode cutoffs for each slice.
more_keywords
is additional keywords to save into the header for bookkeeping.
calibrate_output(self, img, spectral=False)
¶This handles the flux calibration of the image passed in via img
. For spectral data cubes (i.e. the third dimension is
wavelength), then the spectral flag is set to true.
Docker¶
One very useful tool to have is a local build environment of the pyKLIP package for testing and validation purposes. We will be using a software container platform called Docker and this tutorial will provide a brief overview on what it is, how to set it up, and how to use it for pyKLIP.
Here you will find everything you need to know about Docker for pyKLIP.
Setup¶
Here you will learn how to install and setup your Docker.
Installation¶
We will be using the community edition of Docker.
For Ubuntu Linux
sudo aptget update
sudo aptget install dockerce
For all other OSes, installation instructions and requirements can be found here.
Setup¶
From a fresh install, there are a few steps to getting your container up and running.
1. Download and run the pyKLIP image. You can do this by pulling and running, but simply running the pyKLIP image will do both steps in one. Executing the run command will first check your local machine for the appropriate images and use them if Docker finds them, or download them from Docker Hub if it fails. For now we’ll start with the pull command:
$ docker pull simonko/pyklip
2. From here, to check if the appropriate image has been set up use the docker images
command, and you should get
something similar to the following
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
simonko/pyklip latest e9a584c685bb 4 hours ago 2.37 GB
3. Running the command below creates a container of the pyklip image and gives us an interactive shell to interact with
container. The i t
flags allows for interactive mode and allocates a pseudotty for the container respectively.
This is usually combined into the flag it
. If you don’t specify a tag, it’ll generate some random name for you.
(ex. sad_lovelace, agitated_saha, ecstatic_pare, etc)
$ docker run it simonko/pyklip:latest /bin/bash
4. When you’re done with the container, simply type exit
and your session will end. If you get the message that
states there is a process running, simply type exit
again and it’ll exit the session.
5. After you’ve made your container you should be able to see it with
$ docker ps a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c6695e4d9a63 simonko/pyklip:latest "/usr/bin/tini  ..." 6 seconds ago Exited (0) 3 seconds ago zealous_goldwasser
6. To get into the container, you have to first start the container again, then use the attach command to get back into the interactive shell.
$ docker start <container name>
$ docker attach <container name>
For a very basic tutorial on Docker and how to use it, check out the Docker docs and tutorials here. There are a lot of helpful tutorials and information there.
Working With Docker¶
Here you will learn how some basics on working with Docker.
Using Local Files¶
Once you have your image, you can cp over local files into the container. To do this you have to use the attach
command and d
flag like so
$ docker run it d simonko/pyklip:latest
exit
$ docker cp <source file/directory> <container name>:<destination>
$ docker start <container name>
$ docker attach <container name>
It should be noted that if the specified destination does not exist, it will create the destination for you. For example if I were to do the following
$ docker cp <somefile/directory> zealous_goldwasser:/pyklip
inside the zealous_goldwasser container and it did not already have a pyklip directory, docker would create the directory for me and place the file in it, just like the normal cp command.
Deleting Images and Containers¶
You may find that your docker is getting a bit cluttered after playing around with it. The following section will show you how to delete images and containers. You can also refer to this cheat sheet for more on deleting images and containers. The below is just a few basic and useful commands.
To delete a container, first locate the container(s) you wish to delete, then use docker rm <ID or NAME>
to delete:
$ docker ps a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c6695e4d9a63 simonko/pyklip:latest "/usr/bin/tini  ..." 6 seconds ago Exited (0) 3 seconds ago zealous_goldwasser
$ docker rm <container ID (c6695e4d9a63) or Name (zealous goldwasser)>
To delete multiple containers at once use the filter flag. For example, if you want to delete all exited containers
$ docker rm $(docker ps a f status=exited q)
You can also find all containers all exited containers using just the command in the parenthesis without the q flag. This is particularly useful if there are many exited containers and you don’t remember which ones you wanted to delete.
To delete your images first you must find which ones you wish to delete. It should also be noted that to delete an image, there can be no containers associated with it. You must delete all containers from the image before deleting the image.
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
pyklippipeline latest e9a584c685bb 13 days ago 2.37 GB
simonko/pyklip latest e9a584c685bb 13 days ago 2.37 GB
localrepo latest dc74a96e5ef0 2 weeks ago 2.25 GB
ubuntu latest 0ef2e08ed3fa 3 weeks ago 130 MB
continuumio/anaconda3 latest 26043756c44f 6 weeks ago 2.23 GB
$ docker rmi <repository name>
Note
Before you delete an image, all containers using the image must be DELETED, not exited.
To delete ALL of your images
$ docker rmi $(docker images a q)
Sharing Images¶
Here you will learn how to create and upload your own images onto Docker Hub for others to use.
Creating Images¶
In this section, you will learn how to create and upload your own image. To do this you need to make a dockerfile. If you wish to share the image for others to use, you need to create a Docker Hub account and push your image into a repository. This section will go over all of these steps. For a more detailed tutorial use this link. Otherwise here are the very basics.
Docker images are created from a set of commands in a dockerfile. What goes on this file is entirely up to you. Docker uses these commands to create an image, and it can be an entirely new one or an image based off of another existing image.
 Create a file and name it dockerfile. There are three basic commands that go on a dockerfile.
 FROM <Repository>:<Build>  This command will tell docker that this image is based off of another image. You can specify which build to use. To use the most uptodate version of the image, use “latest” for build.
 RUN <Command>  This will run commands in a new layer and creates a new image. Typically used for installing necessary packages. You can have multiple RUN statements.
 CMD <Command>  This is the default command that will run once the image environment has been set up. You can only have ONE CMD statement.
For more information on RUN vs CMD here is a useful link.
After you’ve made your file run the following command to create your image
$ docker build t <Image Name> <Path to Directory of Dockerfile>
The t
flag lets you name the image.
For example, the docker file used for the pyklip image I set up above (under the “Using Docker” section) is made using a dockerfile with the following content:
FROM continuumio/anaconda3:latest
RUN git clone https://bitbucket.org/pyKLIP/pyklip.git \
&& pip install coveralls \
&& pip install emcee \
&& pip install corner \
&& conda install c https://conda.anaconda.org/astropy photutils
Uploading Images¶
If you haven’t already, create a Docker Hub account.
After you’ve made your account, sign in and click on “Create Repository” and fill out the details. Make sure visibility is set to PUBLIC. Press create.
Find your image ID. Using a previous example
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE pyklippipeline latest e9a584c685bb 13 days ago 2.37 GB
The image ID would be e9a584c685bb.
Tag the image using
$ docker tag <Image ID> <DockerHub Account Name>/<Image Name>:<Version or Tag>
So for the pyklip pipeline image my command would be:
$ docker tag e9a584c685bb simonko/pyklip:latest
Check that the image has been tagged
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
pyklippipeline latest e9a584c685bb 13 days ago 2.37 GB
simonko/pyklip latest e9a584c685bb 13 days ago 2.37 GB
Login to Docker on terminal
$ docker login Username: ***** Password: ***** Login Succeeded
Push your tagged image to docker hub
$ docker push <Repository Name>
To pull from the repo now, all you have to do is run the repo. Docker will automatically pull from docker hub if it cannot find it locally.
Tests¶
Here we will lay out the testing infrastructure used for pyKLIP.
Testing¶
Here we will go over how we test and what our testing infrastructure looks like for pyKLIP.
All of our tests can be found in the tests
folder located in the base directory. In the folder, each module or feature gets it’s own test
file, and inside every file, each function is a different test for the module/feature.
The testing workflow for pyKLIP can be broken down into the following steps:
 Creating the tests
 Documenting the tests
 Running the tests
Creating Tests¶
All tests for pyKLIP can be found in the tests
directory. We use pytest to run all of our tests in this directory.
All tests should be named “test_<module/purpose>”, and within the test files, each function should be named “test_<function
name>” to give an idea of what the test is for. The docstring for the function will go into detail as to what the test
is testing and a summary of how it works.
Our testing framework is organized so that each file tests an individual module or feature, and each function inside each test file tests different aspects of the module/feature.
During the test, you may find it necessary to look at input or output files. In this case, all pathing should be agnostic of the directory structure outside of the pyKLIP folder. It is suggested you first construct relative paths with respect to the current test file or the pyklip base directory, and then convert it to an absolute path before using it in the function.
Some commands you may find helpful to find files:
 os.path.abspath(path)  returns the absolute path of the path provided
 os.path.dirname(path)  returns the name of the directory of the filepath provided.
 os.path.exists(path)  returns True if the path exists, False otherwise.
 os.path.sep = path separator. This is important because different OSes can have different path separators. For example Ubuntu Linux uses “/” while Windows uses “". This will take care of that.
 os.path.join(args)  returns a string with all the args separated by the appropriate path separator. For exmaple
os.path.join("this", "is", "a", "path")
would return"this/is/a/path"
in Ubuntu Linux.  __file__  When used on its own, filepath of this python file. All python modules should also have this as an attirbute (e.g.
pyklip.__file__
)
Documenting Tests¶
Docstring for tests should follow the Google Python stylguide. Here is an exmaple of a function docstring:
"""
Summary of what your tests does goes here.
Args:
param1: First param.
param2: Second param.
etc: etc
Returns:
A description of what is returned.
Raises:
Error: Exception.
"""
Use the following link for more details on docstrings as well as Python style in general.
Running Tests¶
All of our tests are run automatically using pytest
on a Docker image using a continuous integration build system (Bitbucket Pipelines).
This allows us to test pyKIP against a fresh and updated Python installation to ensure functionality is not broken and is comptable with the newest Python version.
If these terms seem unfamiliar, please refer to our Developing for pyKLIP page under the “Docker” section for more
information on Docker.
Here is a simple overview of the steps invovled in our automated testing framework:
 Bitbucket Pipelines reads our pipeline yml file to build the pipeline.
 Creates a docker image of the latest continuum anaconda3.
 Git clones the pyklip repository inside image.
 Installs all necessary packages.
 Runs tests using pytest on the test directory.
 Runs coverage analysis on our tests.
 Submits coverage report.
You can also run tests locally. This is typically useful when you make changes and want to check that the changes does not break any functionality. It can also be useful if you write a test before writing the function code, and debug your code as you develop your function. That way, you will have validation code from the start. In this case, you may not want to run the full suite of tests.
To simply run a single test you can either call the file directly using:
$ python <path/to/test file name>.py
To run all tests simply call:
$ pytest
The general command for pytest is as follows and there are two ways to invoke it:
$ python m pytest [args]
$ pytest [args]
The above line will invoke pytest through the Python interpreter and add the current directory to sys.path. Otherwise the second line is equivalent to the first. There are many arguments and many different ways to use pytest. To run a single test simply enter the path to the test file to run, to test all files in a directory use the path to the directory instead of a single file. For more information on how to use pytest and some of its various usages, visit this link.
Code Coverage¶
Here we will go over code coverage, the analysis of what lines of code are tested in our tests.
Our code coverage is set up using two different tools  Coverage and Coveralls. Coverage is what we use to report the coverage statistics on our code and tests, while Coveralls is the service we use to hook our reports to our pipeline, giving us a website to read coverage reports for each build.
Coverage¶
The documentation for the coverage package can be found here.
There are several different ways of reporting code coverage. I highly recommend reading the How Coverage.py Works section to learn what it means to say your tests have x% code coverage. Basically, there are three phases to the code coverage we use:
 Execution: Executes code and records information.
 Analysis: Analyzes codebase for total executable lines.
 Reporting: Combines execution and analysis phases to give coverage report.
When tests are run, coverage.py runs a trace function that records each file and line that is executed when a test is
run. It records this information in a JSON file (usually) named .coverage
. This is called the execution phase.
During “Analysis,” coverage looks at the compiled python files to get the set of executable lines, and filters through to leave out lines that shouldn’t be considered (e.g. blank lines, docstrings).
Finally, the Reporting phase handles the format in which to report its findings. There are several different outputs for the reports that you can use.
Coverage also has a configuration file that allows the user to specify different options for coverage to handle, such as
multithreading. The coverage configuration file is named .coveragerc
by default. Information on the syntax for the
file can be found here. Through the configuration
file you can specify lines to skip, ignoring specific errors, where to output the coverage report, etc.
Note
When running multiple coverage reports or using the multithread option, the command coverage combine
is useful
in that it will combine all the reports into one. Multithreading will spawn multiple processes which will each
have their own report so combining is very important for getting an accurate report. Note that all the reports must
be in the same directory when running the command.
As a final note, it is important to note that, although code coverage is a great tool to have and use, it is not by itself enough to say the code is bug free. 100% code coverage, in the end, does not mean much. It simply means all the executable lines of code have been run in one way or another, but there is no real way to test ALL possible branches and situations your code can take, especially for larger code bases. Read this article for more on why code coverage can be flawed as well as a few examples. Just know that code coverage is a useful tool but not foolproof.
Coveralls¶
Coveralls is the web service used to track our code coverage and report on our automated pipeline builds. Every time code is pushed to our Bitbucket repo and our tests are run, we first obtain our report using coverage, then we send the report to coveralls which in turn organizes our report with each build and displays the information for us on both the coverage website and a badge on the bitbucket repo.
For information on how to setup a coveralls hook to a repo, look here. For our pipeline, we use Bitbucket Pipelines, so use the “Usage (another CI)” section.
pyklip package¶
Subpackages¶
pyklip.fmlib package¶
Submodules¶
pyklip.fmlib.diskfm module¶
pyklip.fmlib.extractSpec module¶

class
pyklip.fmlib.extractSpec.
ExtractSpec
(inputs_shape, numbasis, sep, pa, input_psfs, input_psfs_wvs, input_psfs_pas=None, datatype='float', stamp_size=None)[source]¶ Bases:
pyklip.fmlib.nofm.NoFM
Planet Characterization class. Goal to characterize the astrometry and photometry of a planet

alloc_fmout
(output_img_shape)[source]¶ Allocates shared memory for the output of the shared memory
Parameters: output_img_shape – shape of output image (usually N,y,x,b) Returns: mp.array to store FM data in fmout_shape: shape of FM data array Return type: fmout

cleanup_fmout
(fmout)[source]¶ After running KLIPFM, we need to reshape fmout so that the numKL dimension is the first one and not the last
Parameters: fmout – numpy array of ouput of FM Returns: same but cleaned up if necessary Return type: fmout

fm_from_eigen
(klmodes=None, evals=None, evecs=None, input_img_shape=None, input_img_num=None, ref_psfs_indicies=None, section_ind=None, section_ind_nopadding=None, aligned_imgs=None, pas=None, wvs=None, radstart=None, radend=None, phistart=None, phiend=None, padding=None, IOWA=None, ref_center=None, parang=None, ref_wv=None, numbasis=None, fmout=None, perturbmag=None, klipped=None, flipx=True, **kwargs)[source]¶ Generate forward models using the KL modes, eigenvectors, and eigenvectors from KLIP. Calls fm.py functions to perform the forward modelling
Parameters:  klmodes – unpertrubed KL modes
 evals – eigenvalues of the covariance matrix that generated the KL modes in ascending order (lambda_0 is the 0 index) (shape of [nummaxKL])
 evecs – corresponding eigenvectors (shape of [p, nummaxKL])
 input_image_shape – 2D shape of inpt images ([ysize, xsize])
 input_img_num – index of sciece frame
 ref_psfs_indicies – array of indicies for each reference PSF
 section_ind – array indicies into the 2D xy image that correspond to this section. Note needs be called as section_ind[0]
 pas – array of N parallactic angles corresponding to N reference images [degrees]
 wvs – array of N wavelengths of those referebce images
 radstart – radius of start of segment
 radend – radius of end of segment
 phistart – azimuthal start of segment [radians]
 phiend – azimuthal end of segment [radians]
 padding – amount of padding on each side of sector
 IOWA – tuple (IWA,OWA) where IWA = Inner working angle and OWA = Outer working angle both in pixels. It defines the separation interva in which klip will be run.
 ref_center – center of image
 numbasis – array of KL basis cutoffs
 parang – parallactic angle of input image [DEGREES]
 ref_wv – wavelength of science image
 fmout – numpy output array for FM output. Shape is (N, y, x, b)
 perturbmag – numpy output for size of linear perturbation. Shape is (N, b)
 klipped – PSF subtracted image. Shape of ( size(section), b)
 kwargs – any other variables that we don’t use but are part of the input

generate_models
(input_img_shape, section_ind, pas, wvs, radstart, radend, phistart, phiend, padding, ref_center, parang, ref_wv, flipx, stamp_size=None)[source]¶ Generate model PSFs at the correct location of this segment for each image denoated by its wv and parallactic angle
Parameters:  pas – array of N parallactic angles corresponding to N images [degrees]
 wvs – array of N wavelengths of those images
 radstart – radius of start of segment
 radend – radius of end of segment
 phistart – azimuthal start of segment [radians]
 phiend – azimuthal end of segment [radians]
 padding – amount of padding on each side of sector
 ref_center – center of image
 parang – parallactic angle of input image [DEGREES]
 ref_wv – wavelength of science image
 stamp_size – size of the stamp for spectral extraction
 flipx – if True, flip x coordinate in final image
Returns: array of size (N, p) where p is the number of pixels in the segment
Return type: models


pyklip.fmlib.extractSpec.
invert_spect_fmodel
(fmout, dataset, method='JB', units='natural', scaling_factor=1.0)[source]¶  Greenbaum Nov 2016
Parameters:  fmout – the forward model matrix which has structure: [numbasis, n_frames, n_frames+1, npix]
 dataset – from GPI.GPIData(filelist) – typically set highpass=True also
 method – “JB” or “LP” to try the 2 different inversion methods (JB’s or Laurent’s)
 units – “natural” means the answer is scaled to the input PSF (default) fmout will be in these units. “scaled” means the output is scaled to “scaling_factor” argument
 scaling_factor – multiplies output spectrum and forward model, user set for desired calibration factor. units=”scaled” must be set in args for this to work!
Returns: A tuple containing the spectrum and the forward model (spectrum, forwardmodel) spectrum shape:(len(numbasis), nwav)
pyklip.fmlib.fmpsf module¶

class
pyklip.fmlib.fmpsf.
FMPlanetPSF
(inputs_shape, numbasis, sep, pa, dflux, input_psfs, input_wvs, flux_conversion=None, spectrallib=None, spectrallib_units='flux', star_spt=None, refine_fit=False, field_dependent_correction=None, input_psfs_pas=None)[source]¶ Bases:
pyklip.fmlib.nofm.NoFM
Forward models the PSF of the planet through KLIP. Returns the forward modelled planet PSF

alloc_fmout
(output_img_shape)[source]¶ Allocates shared memory for the output of the shared memory
Parameters: output_img_shape – shape of output image (usually N,y,x,b) Returns: mp.array to store FM data in fmout_shape: shape of FM data array Return type: fmout

alloc_perturbmag
(output_img_shape, numbasis)[source]¶ Allocates shared memory to store the fractional magnitude of the linear KLIP perturbation Stores a number for each frame = max(oversub + selfsub)/std(PCA(image))
Parameters:  output_img_shape – shape of output image (usually N,y,x,b)
 numbasis – array/list of number of KL basis cutoffs requested
Returns: mp.array to store linaer perturbation magnitude perturbmag_shape: shape of linear perturbation magnitude
Return type: perturbmag

cleanup_fmout
(fmout)[source]¶ After running KLIPFM, we need to reshape fmout so that the numKL dimension is the first one and not the last
Parameters: fmout – numpy array of ouput of FM Returns: same but cleaned up if necessary Return type: fmout

fm_from_eigen
(klmodes=None, evals=None, evecs=None, input_img_shape=None, input_img_num=None, ref_psfs_indicies=None, section_ind=None, section_ind_nopadding=None, aligned_imgs=None, pas=None, wvs=None, radstart=None, radend=None, phistart=None, phiend=None, padding=None, IOWA=None, ref_center=None, parang=None, ref_wv=None, numbasis=None, fmout=None, perturbmag=None, klipped=None, covar_files=None, flipx=True, **kwargs)[source]¶ Generate forward models using the KL modes, eigenvectors, and eigenvectors from KLIP. Calls fm.py functions to perform the forward modelling
Parameters:  klmodes – unpertrubed KL modes
 evals – eigenvalues of the covariance matrix that generated the KL modes in ascending order (lambda_0 is the 0 index) (shape of [nummaxKL])
 evecs – corresponding eigenvectors (shape of [p, nummaxKL])
 input_image_shape – 2D shape of inpt images ([ysize, xsize])
 input_img_num – index of sciece frame
 ref_psfs_indicies – array of indicies for each reference PSF
 section_ind – array indicies into the 2D xy image that correspond to this section. Note needs be called as section_ind[0]
 pas – array of N parallactic angles corresponding to N reference images [degrees]
 wvs – array of N wavelengths of those referebce images
 radstart – radius of start of segment
 radend – radius of end of segment
 phistart – azimuthal start of segment [radians]
 phiend – azimuthal end of segment [radians]
 padding – amount of padding on each side of sector
 IOWA – tuple (IWA,OWA) where IWA = Inner working angle and OWA = Outer working angle both in pixels. It defines the separation interva in which klip will be run.
 ref_center – center of image
 numbasis – array of KL basis cutoffs
 parang – parallactic angle of input image [DEGREES]
 ref_wv – wavelength of science image
 fmout – numpy output array for FM output. Shape is (N, y, x, b)
 perturbmag – numpy output for size of linear perturbation. Shape is (N, b)
 klipped – PSF subtracted image. Shape of ( size(section), b)
 kwargs – any other variables that we don’t use but are part of the input

generate_models
(input_img_shape, section_ind, pas, wvs, radstart, radend, phistart, phiend, padding, ref_center, parang, ref_wv, flipx)[source]¶ Generate model PSFs at the correct location of this segment for each image denoated by its wv and parallactic angle
Parameters:  pas – array of N parallactic angles corresponding to N images [degrees]
 wvs – array of N wavelengths of those images
 radstart – radius of start of segment
 radend – radius of end of segment
 phistart – azimuthal start of segment [radians]
 phiend – azimuthal end of segment [radians]
 padding – amount of padding on each side of sector
 ref_center – center of image
 parang – parallactic angle of input image [DEGREES]
 ref_wv – wavelength of science image
 flipx – if True, flip x coordinate in final image
Returns: array of size (N, p) where p is the number of pixels in the segment
Return type: models

save_fmout
(dataset, fmout, outputdir, fileprefix, numbasis, klipparams=None, calibrate_flux=False, spectrum=None, pixel_weights=1)[source]¶ Saves the FM planet PSFs to disk. Saves both a KL mode cube and spectral cubes
Parameters:  dataset – Instruments.Data instance. Will use its dataset.savedata() function to save data
 fmout – the fmout data passed from fm.klip_parallelized which is passed as the output of cleanup_fmout
 outputdir – output directory
 fileprefix – the fileprefix to prepend the file name
 numbasis – KL mode cutoffs used
 klipparams – string with KLIPFM parameters
 calibrate_flux – if True, flux calibrate the data in the same way as the klipped data
 spectrum – if not None, spectrum to weight the data by. Length same as dataset.wvs
 pixel_weights – weights for each pixel for weighted mean. Leave this as a single number for simple mean

pyklip.fmlib.matchedFilter module¶

class
pyklip.fmlib.matchedFilter.
MatchedFilter
(inputs_shape, numbasis, input_psfs, input_psfs_wvs, spectrallib, save_per_sector=None, datatype='float', fakes_sepPa_list=None, disable_FM=None, true_fakes_pos=None, ref_center=None, flipx=None, rm_edge=None, edge_threshold=None, planet_radius=None, background_width=None, save_bbfm=None, save_fm=None, save_fmout_path=None)[source]¶ Bases:
pyklip.fmlib.nofm.NoFM
Matched filter with forward modelling.

alloc_fmout
(output_img_shape)[source]¶ Allocates shared memory for the output of the shared memory
Parameters: output_img_shape – Not used Returns: mp.array to store auxilliary data in fmout_shape: shape of auxilliary array = (3,self.N_spectra,self.N_numbasis,self.N_frames,self.ny,self.nx)  The 3 is for saving the different term of the matched filter:
 0: dot product 1: square of the norm of the model 2: Local estimated variance of the data 3: Number of pixels used in the matched filter
Return type: fmout

fm_end_sector
(interm_data=None, fmout=None, sector_index=None, section_indicies=None)[source]¶ Save the fmout object at the end of each sector if save_per_sector was defined when initializing the class.

fm_from_eigen
(klmodes=None, evals=None, evecs=None, input_img_shape=None, input_img_num=None, ref_psfs_indicies=None, section_ind=None, section_ind_nopadding=None, aligned_imgs=None, pas=None, wvs=None, radstart=None, radend=None, phistart=None, phiend=None, padding=None, IOWA=None, ref_center=None, parang=None, ref_wv=None, numbasis=None, fmout=None, perturbmag=None, klipped=None, flipx=True, **kwargs)[source]¶ Calculate and project the FM at every pixel of the sector. Store the result in fmout.
Parameters:  klmodes – unpertrubed KL modes
 evals – eigenvalues of the covariance matrix that generated the KL modes in ascending order (lambda_0 is the 0 index) (shape of [nummaxKL])
 evecs – corresponding eigenvectors (shape of [p, nummaxKL])
 input_img_shape – 2D shape of inpt images ([ysize, xsize])
 input_img_num – index of sciece frame
 ref_psfs_indicies – array of indicies for each reference PSF
 section_ind – array indicies into the 2D xy image that correspond to this section. Note needs be called as section_ind[0]
 pas – array of N parallactic angles corresponding to N reference images [degrees]
 wvs – array of N wavelengths of those referebce images
 radstart – radius of start of segment
 radend – radius of end of segment
 phistart – azimuthal start of segment [radians]
 phiend – azimuthal end of segment [radians]
 padding – amount of padding on each side of sector
 IOWA – tuple (IWA,OWA) where IWA = Inner working angle and OWA = Outer working angle both in pixels. It defines the separation interva in which klip will be run.
 ref_center – center of image
 numbasis – array of KL basis cutoffs
 parang – parallactic angle of input image [DEGREES]
 ref_wv – wavelength of science image
 fmout – numpy output array for FM output. Shape is (N, y, x, b)
 klipped – array of shape (p,b) that is the PSF subtracted data for each of the b KLIP basis cutoffs. If numbasis was an int, then sub_img_row_selected is just an array of length p
 kwargs – any other variables that we don’t use but are part of the input

generate_model_sci
(input_img_shape, section_ind, pa, wv, radstart, radend, phistart, phiend, padding, ref_center, parang, ref_wv, sep_fk, pa_fk, flipx)[source]¶ Generate model PSFs at the correct location of this segment of the science image denotated by its wv and parallactic angle.
Parameters:  input_img_shape – 2D shape of inpt images ([ysize, xsize])
 section_ind – array indicies into the 2D xy image that correspond to this section. Note needs be called as section_ind[0]
 pa – parallactic angle of the science image [degrees]
 wv – wavelength of the science image
 radstart – radius of start of segment (not used)
 radend – radius of end of segment (not used)
 phistart – azimuthal start of segment [radians] (not used)
 phiend – azimuthal end of segment [radians] (not used)
 padding – amount of padding on each side of sector
 ref_center – center of image
 parang – parallactic angle of input image [DEGREES] (not used)
 ref_wv – wavelength of science image
 sep_fk – separation of the planet to be injected.
 pa_fk – position angle of the planet to be injected.
 flipx – if True, flip x coordinate in final image
 Return: (models, mask)
models: vector of size p where p is the number of pixels in the segment mask: vector of size p where p is the number of pixels in the segment
if pixel == 1: arc shape where to calculate the standard deviation if pixel == 2: 7 pixels disk around the position of the planet.

generate_models
(input_img_shape, section_ind, pas, wvs, radstart, radend, phistart, phiend, padding, ref_center, parang, ref_wv, sep_fk, pa_fk, flipx)[source]¶ Generate model PSFs at the correct location of this segment for each image denotated by its wv and parallactic angle.
Parameters:  input_img_shape – 2D shape of inpt images ([ysize, xsize])
 section_ind – array indicies into the 2D xy image that correspond to this section. Note needs be called as section_ind[0]
 pas – array of N parallactic angles corresponding to N images [degrees]
 wvs – array of N wavelengths of those images
 radstart – radius of start of segment (not used)
 radend – radius of end of segment (not used)
 phistart – azimuthal start of segment [radians] (not used)
 phiend – azimuthal end of segment [radians] (not used)
 padding – amount of padding on each side of sector
 ref_center – center of image
 parang – parallactic angle of input image [DEGREES] (not used)
 ref_wv – wavelength of science image
 sep_fk – separation of the planet to be injected.
 pa_fk – position angle of the planet to be injected.
 flipx – if True, flip x coordinate in final image
Returns: array of size (N, p) where p is the number of pixels in the segment
Return type: models

save_fmout
(dataset, fmout, outputdir, fileprefix, numbasis, klipparams=None, calibrate_flux=False, spectrum=None, pixel_weights=1)[source]¶ Saves the fmout data to disk following the instrument’s savedata function
Parameters:  dataset – Instruments.Data instance. Will use its dataset.savedata() function to save data
 fmout – the fmout data passed from fm.klip_parallelized which is passed as the output of cleanup_fmout
 outputdir – output directory
 fileprefix – the fileprefix to prepend the file name
 numbasis – KL mode cutoffs used
 klipparams – string with KLIPFM parameters
 calibrate_flux – if True, flux calibrate the data (if applicable)
 spectrum – if not None, the spectrum to weight the data by. Length same as dataset.wvs
 pixel_weights – weights for each pixel for weighted mean. [Not used in matched filter]

skip_section
(radstart, radend, phistart, phiend, flipx=True)[source]¶ Returns a boolean indicating if the section defined by (radstart, radend, phistart, phiend) should be skipped. When True is returned the current section in the loop in klip_parallelized() is skipped.
Parameters:  radstart – minimum radial distance of sector [pixels]
 radend – maximum radial distance of sector [pixels]
 phistart – minimum azimuthal coordinate of sector [radians]
 phiend – maximum azimuthal coordinate of sector [radians]
 flipx – if True, flip x coordinate in final image
Returns: False so by default it never skips.
Return type: Boolean


pyklip.fmlib.matchedFilter.
calculate_fm_opti
(delta_KL, original_KL, sci, model_sci_fk, delta_KL_fk, original_KL_fk)[source]¶ Optimized version for calculate_fm() (if numbas) for a single numbasis.
Calculate what the PSF looks up postKLIP using knowledge of the input PSF, assumed spectrum of the science target, and the partially calculated KL modes (Delta Z_k^lambda in Laurent’s paper). If inputflux is None, the spectral dependence has already been folded into delta_KL_nospec (treat it as delta_KL).
Note: if inputflux is None and delta_KL_nospec has three dimensions (ie delta_KL_nospec was calculated using pertrurb_nospec() or perturb_nospec_modelsBased()) then only klipped_oversub and klipped_selfsub are returned. Besides they will have an extra first spectral dimension.
Parameters:  delta_KL_nospec – perturbed KL modes but without the spectral info. delta_KL = spectrum x delta_Kl_nospec. Shape is (numKL, wv, pix). If inputflux is None, delta_KL_nospec = delta_KL
 orignal_KL – unpertrubed KL modes (array of size [numbasis, numpix])
 sci – array of size p representing the science data
 model_sci – array of size p corresponding to the PSF of the science frame
 input_spectrum – array of size wv with the assumed spectrum of the model
If delta_KL_nospec does NOT include a spectral dimension or if inputflux is not None: :returns:
 array of shape (b,p) showing the forward modelled PSF
 Skipped if inputflux = None, and delta_KL_nospec has 3 dimensions.
klipped_oversub: array of shape (b, p) showing the effect of oversubtraction as a function of KL modes klipped_selfsub: array of shape (b, p) showing the effect of selfsubtraction as a function of KL modes Note: psf_FM = model_sci  klipped_oversub  klipped_selfsub to get the FM psf as a function of K Lmodes
(shape of b,p)Return type: fm_psf If inputflux = None and if delta_KL_nospec include a spectral dimension: :returns: Sum(<SKL>KL) with klipped_oversub.shape = (1,Npix)
klipped_selfsub: Sum(<NDKL>KL) + Sum(<NKL>DKL) with klipped_selfsub.shape = (1,N_lambda or N_ref,N_pix)Return type: klipped_oversub
pyklip.fmlib.nofm module¶

class
pyklip.fmlib.nofm.
NoFM
(inputs_shape, numbasis)[source]¶ Bases:
object
Super class for all forward modelling classes. Has fallback functions for all fm dependent calls so that each FM class does not need to implement functions it doesn’t want to. Should do no forward modelling and just do regular KLIP by itself

alloc_fmout
(output_img_shape)[source]¶ Allocates shared memory for the output of the forward modelling
Parameters: output_img_shape – shape of output image (usually N,y,x,b) Returns: mp.array to store FM data in fmout_shape: shape of FM data array Return type: fmout

alloc_interm
(max_sector_size, numsciframes)[source]¶ Allocates shared memory array for intermediate step
Intermediate step is allocated for a sector by sector basis
Parameters: max_sector_size – number of pixels in this sector. Max because this can be variable. Stupid rotating sectors Returns: mp.array to store intermediate products from one sector in interm_shape:shape of interm array (used to convert to numpy arrays) Return type: interm

alloc_output
()[source]¶ Allocates shared memory array for final output
Only use multiprocessing data structors as we are using the multiprocessing class
Args:
Returns: mp.array to store final outputs in (shape of (N*wv, y, x, numbasis)) outputs_shape: shape of outputs array to convert to numpy arrays Return type: outputs

alloc_perturbmag
(output_img_shape, numbasis)[source]¶ Allocates shared memory to store the fractional magnitude of the linear KLIP perturbation
Parameters:  output_img_shape – shape of output image (usually N,y,x,b)
 numbasis – array/list of number of KL basis cutoffs requested
Returns: mp.array to store linaer perturbation magnitude perturbmag_shape: shape of linear perturbation magnitude
Return type: perturbmag

cleanup_fmout
(fmout)[source]¶ After running KLIPFM, if there’s anything to do to the fmout array (such as reshaping it), now’s the time to do that before outputting it
Parameters: fmout – numpy array of ouput of FM Returns: same but cleaned up if necessary Return type: fmout

fm_end_sector
(**kwargs)[source]¶ Does some forward modelling at the end of a sector after all images have been klipped for that sector.

fm_from_eigen
(**kwargs)[source]¶ Generate forward models using the KL modes, eigenvectors, and eigenvectors from KLIP This is called immediately after regular KLIP

save_fmout
(dataset, fmout, outputdir, fileprefix, numbasis, klipparams=None, calibrate_flux=False, spectrum=None, pixel_weights=1)[source]¶ Saves the fmout data to disk following the instrument’s savedata function
Parameters:  dataset – Instruments.Data instance. Will use its dataset.savedata() function to save data
 fmout – the fmout data passed from fm.klip_parallelized which is passed as the output of cleanup_fmout
 outputdir – output directory
 fileprefix – the fileprefix to prepend the file name
 numbasis – KL mode cutoffs used
 klipparams – string with KLIPFM parameters
 calibrate_flux – if True, flux calibrate the data (if applicable)
 spectrum – if not None, the spectrum to weight the data by. Length same as dataset.wvs
 pixel_weights – weights for each pixel for weighted mean. Leave this as a single number for simple mean

skip_section
(radstart, radend, phistart, phiend, flipx=None)[source]¶ Returns a boolean indicating if the section defined by (radstart, radend, phistart, phiend) should be skipped. When True is returned the current section in the loop in klip_parallelized() is skipped.
Parameters:  radstart – minimum radial distance of sector [pixels]
 radend – maximum radial distance of sector [pixels]
 phistart – minimum azimuthal coordinate of sector [radians]
 phiend – maximum azimuthal coordinate of sector [radians]
 flipx – if True, flip x coordinate in final image
Returns: False so by default it never skips.
Return type: Boolean

Module contents¶
pyklip.instruments package¶
Subpackages¶
Given a datacube, find the four corresponding spot files. Plot the calculated positions on top of the original cube.
Run from an ipython terminal with: %run spot_checker.py full/path/to/cube.fits

class
pyklip.instruments.P1640_support.P1640_cube_checker.
ConfigAction
(option_strings, dest, nargs=None, **kwargs)[source]¶ Bases:
argparse.Action
Create a custom action to parse the

pyklip.instruments.P1640_support.P1640_cube_checker.
dnah_spot_directory
= '/data/p1640/data/users/spot_positions/jonathan/'¶  Load list of files
 Create the “good files” dictionary
3. For each file: 3a. Split offt a thread for drawing the cube 3b. Ask for user input 4. When the user provides ‘y’ or ‘n’, update the dictionary and kill the drawing thread 5. Move on to the next file
Type: Pseudocode

pyklip.instruments.P1640_support.P1640_cube_checker.
draw_cube
(cube, cube_name, header, seeing, airmass, cube_ix)[source]¶ Make a figure and draw cube slices on it

pyklip.instruments.P1640_support.P1640_cube_checker.
draw_spot_cube
(cube, cube_name, spots)[source]¶ Make a figure and draw cube slices on it spots are a list of [row, col] positions for each spot

pyklip.instruments.P1640_support.P1640_cube_checker.
get_total_exposure_time
(fitsfiles, unit=Unit("min"))[source]¶ Accept a list of fits files and return the total exposure time Input:
fitsfiles: single fits file or list of files with keyword ‘EXPTIME’ in the header units: [minute] astropy.units unit for the output Output:
 tot_exp_time: the sum of the exposure times for each cube, in minutes
Given a datacube, find the four corresponding spot files. Plot the calculated positions on top of the original cube.
Run from an ipython terminal with: %run spot_checker.py full/path/to/cubes.fits

class
pyklip.instruments.P1640_support.P1640_spot_checker.
ConfigAction
(option_strings, dest, nargs=None, **kwargs)[source]¶ Bases:
argparse.Action
Create a custom action to parse the
Utilities specific to generating contrast curves

pyklip.instruments.P1640_support.P1640contrast.
calc_contrast_multifile
(core_files, datacube)[source]¶ Assemble a median core PSF out of the list of core_files, and return a datacube scaled by the core flux

pyklip.instruments.P1640_support.P1640contrast.
calc_contrast_single_file
(filename, core_info=None, chans='all')[source]¶ Calculate the radiallaveraged variance for a pyklipreduced file for a single channel Input:
filename: full path to a pyklipprocessed datacube core_info: pandas DataFrame with ‘radius’ and ‘flux’ columns chans: iterative type list of channels

pyklip.instruments.P1640_support.P1640contrast.
make_contrast_plot
(contrast_map, title=None, contrast_range=None, plate_scale=19.2, pckwargs=None)[source]¶ Plot contrast against separation and channel number. Input:
contrast_map: Pandas DataFrame. See required columns below. name: plot title (preferably the file name corresponding to the contrast map) plate_scale (19.2 mas/pix): convert between pixels and mas contrast_range (None): tuple of upper and lower bounds for contrast plot. If none, minmax. pckwargs: dictionary of arguments that can be passed to plt.pcolorReturns: plt.figure() object Return type: fig contrast_map: one column is titled ‘rad’, this is the separation in pixels The rest of the columns are titled ‘chan##’, where ## is the channel number.
This library has method for operating on P1640 core images

pyklip.instruments.P1640_support.P1640cores.
aperture_convolve_cube
(orig_cube, aperture_radii, apkwargs={'method': 'subpixel', 'subpixels': 4})[source]¶ Perform apeture photometry on every pixel in a datacube or image Wrapper for _aperture_convolve_img to handle 2D and 3D data Input:
orig_cube: [Nlambda x] Npix x Npix datacube aperture_radii: Nlambda array of aperture radii apkwargs: dictionary of arguments to pass to aperture_photometry
Default: {‘method’:’subpixel’,’subpixels’:4}Returns: cube with shape of orig_cube of the aperture photometry Return type: phot_cube

pyklip.instruments.P1640_support.P1640cores.
centroid_image
(orig_img)[source]¶ Centroid an image  weighted sum of pixels Input:
orig_img: 2D arrayReturns: [y,x] floatingpoint coordinates of the centroid

pyklip.instruments.P1640_support.P1640cores.
combine_multiple_cores
(multiple_core_info)[source]¶ Combine the stellar flux and radii information from multiple cores in the proper way.

pyklip.instruments.P1640_support.P1640cores.
get_PSF_center
(orig_cube, refchan=26, fine=False)[source]¶ Return the PSF center at the pixel level (default) or subpixel level (fine=True) Input:
orig_cube: [Nlambda x] Npix x Npix datacube (or image) refchan(=26): Reference channel for the initial center estimate fine_centering(=False): After getting a rough estimate of the center, centroid the imageReturns: Nlamdba x 2 array of pixel indices for the PSF center

pyklip.instruments.P1640_support.P1640cores.
get_cube_xsection
(orig_cube, center, width)[source]¶ Select the crosssection of a cube centered in center with 1/2width width Input:
orig_cube: Nlambda x Npix x Npix datacube center: [row, col] index width: 1/2width of crosssectionReturns: Nlambda x (2*width+1) x (2*width+1) cube crosssection Return type: cube_cutout

pyklip.instruments.P1640_support.P1640cores.
get_encircled_energy_cube
(orig_cube, frac=0.5)[source]¶ Get the fractional encircled energy of a PSF in each channel of a datacube. Basically a wrapper for _get_encircled_energy_image. Accepts 2D and 3D input. Input:
orig_cube: unocculted core cube [Nlambda x ]Npix x Npix frac: encircled energy cutoffReturns: [starx, stary, radius, flux] Return type: Pandas Dataframe with Nlambda columns

pyklip.instruments.P1640_support.P1640cores.
get_injection_core
(core_cubes)[source]¶ Remember, the injected PSF needs to be the SAME as the reference PSF, except for a scaling factor! Combine multiple cubes into a single core file for injection. Make sure that the total injected flux is the sum of the pixels! Outline: 1. Get the encircled energy fraction for all the cores (frac=1) 2. For each core, prepare a zerocube with width of the largest radius + 1 3. Add the core from each channel to the zerocube

class
pyklip.instruments.P1640_support.P1640spots.
P1640params
[source]¶ Bases:
object

aperture_refchan
= 3.5¶

channels
= array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31])¶

nchan
= 32¶

num_spots
= 4¶

refchan
= 26¶

reflambda
= 1.663451612903226¶

scale_factors
= array([0.58252371, 0.59858049, 0.61463727, 0.63069405, 0.64675083, 0.66280761, 0.67886439, 0.69492117, 0.71097795, 0.72703473, 0.74309151, 0.75914829, 0.77520507, 0.79126185, 0.80731863, 0.82337541, 0.8394322 , 0.85548898, 0.87154576, 0.88760254, 0.90365932, 0.9197161 , 0.93577288, 0.95182966, 0.96788644, 0.98394322, 1. , 1.01605678, 1.03211356, 1.04817034, 1.06422712, 1.0802839 ])¶

wlsol
= array([0.969 , 0.99570968, 1.02241935, 1.04912903, 1.07583871, 1.10254839, 1.12925806, 1.15596774, 1.18267742, 1.2093871 , 1.23609677, 1.26280645, 1.28951613, 1.31622581, 1.34293548, 1.36964516, 1.39635484, 1.42306452, 1.44977419, 1.47648387, 1.50319355, 1.52990323, 1.5566129 , 1.58332258, 1.61003226, 1.63674194, 1.66345161, 1.69016129, 1.71687097, 1.74358065, 1.77029032, 1.797 ])¶


pyklip.instruments.P1640_support.P1640spots.
check_bad_channels
(rad_spot)[source]¶ Check that the spot positions increase monotonically in radius. If they don’t, return the positions that do not follow monotonically. Input:
rad_spot: Nchan array of radial sep of a spot from star Output:
 bad_chans: a list of channel pairs that fail the check and need fixing

pyklip.instruments.P1640_support.P1640spots.
check_bad_spots
(spot, centers)[source]¶  Input:
 spot: y,x positions for a spot centers: y,x positions for the center
 Output:
 fixed_spot: a fixed spot position y,x

pyklip.instruments.P1640_support.P1640spots.
fit_grid_spot
(img, center, loc=None)[source]¶ Fit spot with a 2D Gaussian Inputs:
img: 2D masked_array to fit center: center of the image in (row, col) order loc (optional): initial position guess in (row, col) order

pyklip.instruments.P1640_support.P1640spots.
fit_grid_spots
(masked_cubes, centers, spots_guesses)[source]¶ Wrapper for fit_grid_spot to loop over all four spots Input:
masked_cubes: Nspot x Nchan x Npix x Npix array of masks centers: Nchan x 2 array of (row, col) guesses for spot centers spot_guesses: Nspot x Nchan x 2 (row, col) guesses for spot locations Output:
 spot_fits: List of astropy.model fits spot_locs: Nspot x Nchan x 2 array of spot locations from fitting

pyklip.instruments.P1640_support.P1640spots.
fit_poly
(ind, dep, order)[source]¶ Takes in an array of positions in row, col format and returns a polynomial that fits them to col = b + C*row, where C is a vector of coefficients Fitting is done by least squares Input:
ind: dependent variable (prob channel number) dep: independent variable (prob x or y spot position)Returns: array of polynomial coefficients

pyklip.instruments.P1640_support.P1640spots.
fix_bad_channels
(spot, centers, bad_chans)[source]¶  Two cases:
 a spot jumps inwards
 a spot jumps outwards
In either case, remove both the failing spot and the one before it and fit a cubic to the remaining points. Then, fix the spot that is further from the fit. Input:
spot: Nchan x 2 array of positions for one spot centers: y,x positions for the star in each channel bad_chan: index of a spot that does not monotonically increase in radius Output:
 fixed_spot: Nchan x 2 array of fixed positions for the spot

pyklip.instruments.P1640_support.P1640spots.
get_centered_grid
(img_shape, center)[source]¶ Return a coordinate grid shifted to the center

pyklip.instruments.P1640_support.P1640spots.
get_initial_spot_guesses
(cube, rotated_spots=False)[source]¶

pyklip.instruments.P1640_support.P1640spots.
get_points_from_poly
(ind, coeffs)[source]¶ Get a polynomials coefficients and return the yvalues given the independent values

pyklip.instruments.P1640_support.P1640spots.
get_rotated_grid
(img_shape, center, angle)[source]¶ Rotate a coordinate grid by some angle around a center

pyklip.instruments.P1640_support.P1640spots.
get_scaling
(spot_array, star_array=None, return_mean=True)[source]¶ Wrapper for get_single_cube_scaling_factors, to handle multiple cubes Input:
spot_array: (Nfiles) x Nspots x Nchan x 2 array in (row, col) order star_array: (Nfiles) x Nchan x 2 array of (row, column) star positions.
If not supplied, spot_array will be calculated from spot_array return_mean: (default False) If true, return mean scaling factor for
 each channel (useful for multiple cubes)
 Output:
 scaling_array: (Nfiles) x Nchan array of scaling fators

pyklip.instruments.P1640_support.P1640spots.
get_scaling_and_centering_from_files
(files, mean_scaling=True)[source]¶ Take some csv spot files, and return the star positions and scaling factors for each datacube Wrapper for get_scaling_and_centering_from_spots Input:
 files: a list of fits files with data cubes
 if the files end in fits or csv, call appropriate routines
mean_scaling: [True] return the mean scaling of the 4 spots
 Output:
 scaling_factors: scaling factors for each slice of each cube star_positions: star positions in each slice of each cube

pyklip.instruments.P1640_support.P1640spots.
get_scaling_and_centering_from_spots
(spot_positions, mean_scaling=True)[source]¶ Accepts an array of spots, and returns the scaling factors and centers. See also: get_scaling_and_centering_from_files Input:
spot_positions: Ncube x Nspot x Nchan x 2 array of (row, col) spot positions mean_scaling: [True] return the average scaling of the 4 spots Output:
 scaling_factors: Ncube x Nchan array of scaling factors star_positions: Ncube x Nchan x 2 array of (row, col) star positions

pyklip.instruments.P1640_support.P1640spots.
get_single_cube_scaling_factors
(spot_array, star_array=None)[source]¶ Get the scaling factors for a single cube Input:
spot_array: Nspots x Nchan x 2 array of (row, column) spot positions star_array: Nchan x 2 array of (row, column) star positions.
If not supplied, spot_array will be calculated from spot_array Output:
 scaling: Nspots x Nchan array of scaling factors, normalized to
 P1640params.refchan

pyklip.instruments.P1640_support.P1640spots.
get_single_cube_spot_photometry
(cube, spot_positions)[source]¶ Do aperture photometry on the spots. Will need to be careful about aperture size for future comparison Input:
cube: Nchan x Npix x Npix data cube to do photometry spot_positions: Nspot x Nchan x 2 spot positions for apertures scaling_factors: Nchan array for scaling apertures with wavelength Output:
 spot_phot: Nspot x Nchan array of spot photometry and spot errors

pyklip.instruments.P1640_support.P1640spots.
get_single_cube_spot_positions
(cube, rotated_spots=False)[source]¶ Return the spot positions for a single cube Input:
cube: a data cube from P1640 rotated_spots: (False) if True, use the rotated masks Output:
 spot_array: Nspots x Nchan x 2 array of spot positions.

pyklip.instruments.P1640_support.P1640spots.
get_single_cube_spot_positions_and_photometry
(cube)[source]¶ Wrapper that combines get_single_cube_spot_positions and get_single_cube_spot_photometry Input:
cube: a datacube in P1640 format Output:
 spot_positions: Nspots x Nchan x 2 array of spot positions. spot_photometry: Nspots x Nchan x 1 array of spot fluxes

pyklip.instruments.P1640_support.P1640spots.
get_single_cube_star_positions
(spot_array)[source]¶ Using the spot positions for a single cube, find the star position at each wavelength. Input:
spot_array: Nspots x Nchan x 2 array of (row, column) spot positions Output:
 star_array: Nchan x 2 array of (row, column) star positions

pyklip.instruments.P1640_support.P1640spots.
get_single_file_scaling_and_centering
(fitsfile)[source]¶ Take a single fits file, and return the star positions and scaling factors See also: get_scalign_and_centering_from_spots Input:
fitsfile: a single fits file with a P1640 cube Output:
 scaling_factors: scaling factors for each slice of the cube star_positions: star positions in each slice of the cube

pyklip.instruments.P1640_support.P1640spots.
get_single_file_spot_positions
(fitsfile, rotated_spots=False)[source]¶ Wrapper for get_single_cube_spot_positions

pyklip.instruments.P1640_support.P1640spots.
get_spot_positions
(fitsfiles)[source]¶ Return the spot positions for a set of data cubes. Really just a wrapper for get_single_cube_spot_positions Input:
fitsfiles: a list of P1640 fits files Output:
 spot_array: Nfile x 4 x Nchan x 2 array of spot positions

pyklip.instruments.P1640_support.P1640spots.
get_spot_positions_and_photometry
(fitsfiles)[source]¶ Wrapper that combines get_single_cube_spot_positions and get_single_cube_spot_photometry Accept a list of fits files and returns the spot positions and spot photometry Input:
fitsfiles: a list of P1640 fits files Output:
 spot_array: Nfiles x Nspots x Nchan x 2 array of (row, col) positions spot_phot: Nfils x Nspots x Nchan array of spot photometry

pyklip.instruments.P1640_support.P1640spots.
get_star_positions
(spot_array)[source]¶ Get the center of a set of 4 spots for a single cube Input:
spot_locations: Nspot x Nlambda x 2 array of [row, col] spot positionsReturns: Nlambda x 2 array of [row, col] star positions Return type: star_array

pyklip.instruments.P1640_support.P1640spots.
guess_grid_spot_loc
(img)[source]¶ get max pixel as initial guess of location

pyklip.instruments.P1640_support.P1640spots.
make_mask_bar
(img_shape, center, angle, width)[source]¶ Make a bar mask where all the pixels inside a bar through the center of the image within some width are 1 and everything outside is 0 Inputs:
img_shape: the shape of the image in (row, col) center: the center of the mask, in (row, col) angle: angle measured counterclockwise from vertical, default in deg width: with of bar in pixelsReturns:  a masked array where the values inside the bar are False and
 outside the bar are True
Return type: mask

pyklip.instruments.P1640_support.P1640spots.
make_mask_circle
(img_shape, center, R)[source]¶ Make a circular mask, where everything inside a radius R around the center is False and outside is True Input:
img_shape: the shape of the image in (row, col) center: the center of the mask, in (row, col) R: the radius of the circleReturns: a masked array of shape img_shape Return type: mask

pyklip.instruments.P1640_support.P1640spots.
make_mask_donut
(img_shape, center, R0, R1)[source]¶ Make a donut mask centered on ‘center’ where the inside of the donut is False and the outside of the donut is True

pyklip.instruments.P1640_support.P1640spots.
make_mask_grid_spots
(img_shape, centers, rotated_spots=False, nchan=32)[source]¶ Make a mask that shows only the grid spots Input:
img_shape: the shape of the image to mask in (row, col) centers: (Nchan x 2) array of centers of the mask in (row, col) rotated_spots: [False] make mask for normal (False) or rotated (True) grid spots nchan: number of spectral channels in the cubeReturns: Nspot x Nchan cube of masks Return type: masks

pyklip.instruments.P1640_support.P1640spots.
make_mask_half_img
(img_shape, center, angle)[source]¶ Mask half the image, cutting it through the center at an arbitrary angle. Angle is measured counterclockwise from vertical, and should be an astropy units object, otherwise assume degrees. Input:
img_shape: shape of image in (row, col) center: the center of the mask in (row, col) angle: angle measured counterclockwise from vertical, default in deg Output:
 mask: masked_array with a plane running through point (center) at angle
 (angle)

pyklip.instruments.P1640_support.P1640spots.
make_mask_refined_grid_spots
(img_shape, centers, spots, nchan=32)[source]¶ Make a new set of masks that are centered on the interpolated grid spot locations Input:
img_shape: x and ydimensions of image centers: nchan x 2 array of star positions spots: num_spots x nchan x 2 array of spot positions

pyklip.instruments.P1640_support.P1640spots.
write_spots_to_file
(data_filepath, spot_positions, output_dir=None, spotid=None, ext=None, overwrite=True)[source]¶ Write one file for each spot to the directory defined at the top of this file. Output file name is data_filename fits +spoti.csv. Format is (row, col). Will overwrite existing files. Input:
data_filename: the base name of the file with the spots spot_positions: Nspot x Nchan x 2 array of spot positions output_dir: directory to write the output files overwrite: (True) overwrite existing spot files spotid: (spoti) identifier for the 4 different spot files ext: (csv) file extensionReturns: None writes a file to the output dir whose name corresponds to the cube used to generate the spots + spotidN.ext (N is 03)

pyklip.instruments.P1640_support.P1640utils.
centroid_image
(orig_img)[source]¶ Centroid an image  weighted sum of pixels Input:
orig_img: 2D arrayReturns: [y,x] floatingpoint coordinates of the centroid

pyklip.instruments.P1640_support.P1640utils.
clean_bad_pixels
(img, boxrad=2, thresh=3)[source]¶ Clean the image of outlier pixels using a median filter. Input:
img: 2d array boxrad: 1/2 the fitler size (2*boxrad+1) thresh: threshold (in stdev) for deciding a hot pixelReturns: 2_d array where hot pixels have been replaced by median values Return type: cleaned_img

pyklip.instruments.P1640_support.P1640utils.
clean_bad_pixels_cube
(cube, boxrad=2, thresh=10)[source]¶ Clean the image of outlier pixels using a median filter. Input:
cube: 3D data cube boxrad: 1/2 the fitler size (2*boxrad+1) thresh: threshold (in stdev) for deciding a hot pixel

pyklip.instruments.P1640_support.P1640utils.
find_bad_pix
(img, median_img, std_img, thresh=3)[source]¶ Find the bad pixels

pyklip.instruments.P1640_support.P1640utils.
get_PSF_center
(cube, refchan=26, fine=False)[source]¶ Return the PSF center at the pixel level (default) or subpixel level (fine=True) Input:
cube: Nlambda x Npix x Npix datacube refchan(=26): Reference channel for the initial center estimate fine_centering(=False): After getting a rough estimate of the center, centroid the imageReturns: Nlamdba x 2 array of pixel indices for the PSF center

pyklip.instruments.P1640_support.P1640utils.
get_cube_xsection
(orig_cube, center, width)[source]¶ Select the crosssection of a cube centered in center with 1/2width width Input:
orig_cube: Nlambda x Npix x Npix datacube center: [row, col] index width: 1/2width of crosssectionReturns: Nlambda x (2*width+1) x (2*width+1) cube crosssection Return type: cube_cutout

pyklip.instruments.P1640_support.P1640utils.
get_encircled_energy_cube
(cube, frac=0.5, refchan=26)[source]¶ Get the fractional encircled energy of a PSF in each channel of a datacube. Basically a wrapper for get_encircled_energy_image Input:
core_cube: unocculted core cube Nlambda x Npix x Npix frac: encircled energy cutoffReturns: [starx, stary, radius, flux] Return type: Pandas Dataframe with Nlambda columns

pyklip.instruments.P1640_support.P1640utils.
get_encircled_energy_image
(im, center, frac=0.5)[source]¶ Given an image, find the fraction of encircled energy around the center. Input:
im: unocculted core cube Npix x Npix frac: encircled energy cutoffReturns: [starx, stary, radius, flux, bgnd_mean, bgnd_std, bgnd_npix] Return type: Pandas Series with the following indices

pyklip.instruments.P1640_support.P1640utils.
get_spot_files
(fitsfile, spot_file_dir)[source]¶ Search in spot_file_dir for the spot files associated with fitsfile Return spot files if they are found, otherwise return an empty list

pyklip.instruments.P1640_support.P1640utils.
set_zeros_to_nan
(data)[source]¶ PyKLIP expects values outside the detector to be set to nan. P1640 sets these (and also saturated pixels) to identically 0. Find all the zeros and convert them to nans Input:
data: N x Npix x Npix datacube or appended set of datacubesReturns: data with nans instead of zeros Return type: nandata

pyklip.instruments.P1640_support.P1640utils.
table_to_TableHDU
(table, kwargs={})[source]¶ Accept a table with a .colnames element and return it as an astropy fits.TableHDU object. Only works with floatingpoint data atm. Input:
table: astropy.table.Table object kwargs: dict of keywords and arguments to pass to the HDUReturns: fits TableHDU with an empty header Return type: TableHDU

class
pyklip.instruments.utils.global_centroid.
Consumer
(task_queue, result_queue)[source]¶ Bases:
multiprocessing.context.Process

pyklip.instruments.utils.global_centroid.
fitallrelcen
(cubes, ivars, r1=15, r2=50, smooth=True, maxcpus=1)[source]¶ Function fitallrelcen. Fit for the relative centroids between all pairs of frames at the central wavelength using the PSF in an annulus around the center. Return the bestfit relative offets.
Parameters:  cubes – all image cubes in the data set, shape (ncube, nwv, ny, nx)
 ivars – inverse variance frames corresponding to cubes
 r1 – int, minimum separation in lenslets from the image center for annular reference region. Default 15.
 r2 – int, maximum separation in lenslets from the image center for annular reference region. Default 50.
 smooth – whether to smooth the reference image before fitting
 maxcpus – int, maximum number of CPUs to allocate for parallelization. Default 1/2 of the available CPUs.
Returns: 1D ndarray of the relative centers in x ysol : 1D ndarray of the relative centers in y
Return type: xsol

pyklip.instruments.utils.global_centroid.
fitcen
(cube, ivar, lam, spotsep=None, guess_center_loc=None, i1=1, i2=1, r1=15, r2=35, spot_dx=4, astrogrid='XYdiag', smooth=True)[source]¶ Function fitcen. Fit for the center of a CHARIS data cube using the satellite spots by maximizing the agreement between scaled cube slices around the spot locations. If no spot locations are provided, use only the diffraction pattern itself in an annulus around the image center.
Parameters:  cube – 3D ndarray, CHARIS data cube
 ivar – 3D ndarray, inverse variance of the CHARIS data cube
 lam – 1D ndarray, wavelengths in microns
 spotsep – float or None. If float, separation of the satellite spots in units of lambda/D. If None, only use the diffraction pattern in an annulus between r1 and r2.
 guess_center_loc – manually specify initial location of image center if necessary, in [x, y] format
 i1 – int, first slice of the data cube to use. Default 1 (skip slice 0)
 i2 – int, high limit of slices to use. Default 1 (skip last slice)
 r1 – float, minimum separation from approximate center for the annulus of the diffraction pattern to use in centroiding. Default 15
 r2 – float, maximum separation from approximate center for the annulus of the diffraction pattern to use in centroiding. Default 35
 spot_dx – float, radius around spot location to cut out in order to match the spot location as a function of wavelength. Default 4
 astrogrid – astrogrid status read from the header, determines the pattern of the diffraction spots
 smooth – whether to smooth image before fitting
Returns:  list of floats
p[0] is the angle of spots in radians p[1] is the separation in lambda/D p[2]  p[1] are the coefficients of the polynomial fit to the centroid
Return type: p

pyklip.instruments.utils.global_centroid.
fitcen_parallel
(infiles, cubes, ivars, prihdrs, astrogrid_status=None, astrogrid_sep=None, smooth_coef=True, guess_center_loc=None, smooth_cubes=True, maxcpus=1)[source]¶ Function fitcen_parallel. Centroid a series of CHARIS data cubes in parallel using fitcen. By default, get the wavelengths and astrogrid parameters from the headers. This might fail on early CHARIS data before the standardization of headers.
Parameters:  infiles – input files
 cubes – all image cubes in the data set corresponding to filenames, shape (ncube, nwv, ny, nx)
 ivars – inverse variance frames corresponding to cubes
 prihdrs – primary headers for the cubes
 astrogrid_status – None or list of astrogrid configurations for SCExAO. If None, try to read the astrogrid configuration from the header. If this fails, assume there is no astrogrid and centroid using the general diffraction pattern. Default None.
 astrogrid_sep – None or list of astrogrid spot separations in units of lambda/D. If None, try to read from the header. If that fails, centroid using the general diffraction pattern. Default None.
 smooth_coef – boolean. smooth the nonlinear coefficients of the centroid fit (the terms proportional to lambda and lambda^2) over the sequence of cubes? Default True.
 guess_center_loc – manually specify initial location of image center if necessary, in [x, y] format
 smooth_cubes – whether to smooth the data before fitting
 maxcpus – int, maximum number of CPUs to use in parallelization
Returns: [centroid_params, x, y, mask]
 centroid_params : 2D array of centroid parameters, first dimension is the number of files.
Second dimension is the length of the wavelengthdependent model.
x : xcoordinates of the centroid at the middle wavelength y : ycoordinates of the centroid at the middle wavelength mask : 1D boolean array, True if astrogrid was on, or None if the astrogrid was never on.

pyklip.instruments.utils.global_centroid.
fitrelcen
(image1, image2, x, y, method='Powell')[source]¶ Function fitrelcen fits for the offset between two images without any wavelength dependence by calling _resid_xy, minimizing the sum of the squared differences between the images (after optimizing over the wavelengthdependent relative normalization).
Parameters:  image1 – 2D ndarray, first image
 image2 – 2D ndarray, second image
 x – ndarray, x coordinates of pixels/lenslets to use
 y – ndarray, y coordinates of pixels/lenslets to use
 method – method passed to scipy.optimize.minimize. Default ‘Powell’.
Returns:  two floating point numbers giving the bestfit offset between
image1 and image2
Return type: xc, yc

pyklip.instruments.utils.global_centroid.
get_sats_satf
(p, cube, lam, astrogrid='XYdiag')[source]¶ retrieves the pixel locations of all four satellite spots at each wavelength, and the negative sum of the four spot intensities at each wavelength
Parameters:  p – list of floats, the coefficients of the polynomial fit to the centroid
 cube – ndarray, data cube for which the centroid is fitted
 lam – ndarray, wavelengths for the datacube
 astrogrid – astrogrid status read from the header, determines the pattern of the diffraction spots
Returns: pixel locations (in [x,y] format) of all four satellite spots at each wavelength, shape (wvs, 4, 2) satf: float, peak fluxes (interpolated pixel value) at the fitted spot locations
Return type: sats

pyklip.instruments.utils.global_centroid.
polyfit
(x, y, order=2, clip=2.5, niter=5, mask=None, return_y=True)[source]¶ Smooth a series of points with a polynomial, iteratively clipping outliers.
Parameters:  x – 1D ndarray of x coordinates for the polynomial fit
 y – 1D ndarray of y coordinates for the polynomial fit
 order – int, order of the polynomial to fit. Default 2.
 clip – float, number of sigma outliers to clip. Default 2.5.
 niter – int, number of iterations of sigma clipping. Default 5.
 mask – boolean ndarray or None: mask each y value? Default None
 return_y – boolean, return smoothed y values (as opposed to coefficients of the polynomial fit)? Default True.
Returns: 1D array, if return_y=True coef : array of the polynomial coefficients if return_y=False
Return type: y_smoothed

pyklip.instruments.utils.global_centroid.
recen
(p, cube, lam, sats, satf, n=None, scale=False, head=None, outfile=None, mask=None, data_HDU1=True)[source]¶ Function recen recenters the input data according to the offset parameters given in the argument p. Optionally scale the data by wavelength to undo the scaling of diffraction. Return the recentered cube.
Parameters:  p – list of floats, the coefficients of the polynomial fit to the centroid
 cube – 3D ndarray, data cube
 lam – 1D array of wavelengths
 sats – fitted satellite spot indices before recentering
 satf – fitted satellite spot fluxes
 n – integer, spatial dimension of recentered cube. Default original cube size
 scale – boolean, rescale by wavelength? Default False
 head – fits header for output file. Default None
 outfile – string or None, name of output file. Default None
 mask – boolean lenslet mask
 data_HDU1 – boolean, write data to HDU1 and leave HDU0 with no data? Default True.
Returns: 3D ndarray, nlam x n x n, recentered (optionally scaled by wavelength) data cube.

pyklip.instruments.utils.global_centroid.
specphotcal
(infiles, cubes, prihdrs, cencoef, aperture=1.0)[source]¶ Function specphotcal. Computes approximate photometry from the satellite spots (using aperture photometry) and scale each wavelength to this photometric value. This should crudely put the cubes in units of contrast, though it omits the scaling of satellite spot intensity with 1/lambda^2.
Parameters:  infiles – list of file names with CHARIS data cubes
 cubes – all image cubes in the data set corresponding to filenames, shape (ncube, nwv, ny, nx)
 prihdrs – primary headers for the cubes
 cencoef – 2D ndarray with coefficeitns
 aperture – float, radius of aperture for photometry in units of lambda/D
Returns: photocalibration coefficients
Return type: all_phot

pyklip.instruments.utils.nair.
get_coeff_mathar
(i, P, T, H, wvrange=1)[source]¶ Calculate the coefficients for the polynomial series expansion of index of refraction (Mathar (2008)) ***Only valid for between 1.3 and 2.5 microns! and 2.8 through 4.2 microns!
 Inputs:
 i: degree of expansion in wavenumber P: Pressure in Pa T: Temperature in Kelvin H: relative humiditiy in % (i.e. between 0 and 100) wvrange (int): 1 = (1.32.5 um), 2 = (2.8  4.2 um)
Returns: Coefficient [cm^i] Return type: coeff

pyklip.instruments.utils.nair.
nMathar
(wv, P, T, H=10)[source]¶ Calculate the index of refraction as given by Mathar (2008): http://arxiv.org/pdf/physics/0610256v2.pdf ***Only valid for between 1.3 and 2.5 microns!
 Inputs:
 wv: wavelength in microns P: Pressure in Pa T: Temperature in Kelvin H: relative humiditiy in % (i.e. between 0 and 100)
Returns: index of refratoin Return type: n

pyklip.instruments.utils.nair.
nRoe
(wv, P, T, H=10)[source]¶ Compute n for air from the formula in Henry Roe’s PASP paper: http://arxiv.org/pdf/astroph/0201273v1.pdf which in turn is referenced from Allen’s Astrophysical Quantities.
 Inputs:
 wv: wavelength in microns P: pressure in Pascal T: temperature in Kelvin H: relative humidity in % (0100)
Returns: index of refraction of air Return type: n

pyklip.instruments.utils.radonCenter.
samplingRegion
(size_window, theta=[45, 135], m=0.2, M=0.8, step=1, decimals=2, ray=False)[source]¶ This function returns all the coordinates of the sampling region, the center of the region is (0,0) When applying to matrices, don’t forget to SHIFT THE CENTER! Input:
size_window: the radius of the sampling region. The whole region should thus have a length of 2*size_window+1. theta: the angle range of the sampling region, default: [45, 135] for the antidiagonal and diagonal directions. m: the minimum fraction of size_window, default: 0.2 (i.e., 20%). In this way, the saturated region can be excluded. M: the maximum fraction of size_window, default: 0.8 (i.e., 80%). Just in case if there’s some star along the diagonals. step: the seperation between sampling dots (units: pixel), default value is 1pix. decimals: the precisoin of the sampling dots (units: pixel), default value is 0.01pix. ray: only half of the line? Output: (xs, ys)
 xs: x indecies, flattend. ys: y indecies, flattend.
Example
1. If you call “xs, ys = samplingRegion(5)”, you will get: xs: array([2.83, 2.12, 1.41, 0.71, 0.71, 1.41, 2.12, 2.83, 2.83, 2.12, 1.41, 0.71, 0.71, 1.41, 2.12, 2.83] ys: array([2.83, 2.12, 1.41, 0.71, 0.71, 1.41, 2.12, 2.83, 2.83, 2.12, 1.41, 0.71, 0.71, 1.41, 2.12, 2.83])) 2. For “radonCenter.samplingRegion(5, ray=True)”, you will get: xs: array([ 0.71, 1.41, 2.12, 2.83, 0.71, 1.41, 2.12, 2.83]) ys: array([ 0.71, 1.41, 2.12, 2.83, 0.71, 1.41, 2.12, 2.83])

pyklip.instruments.utils.radonCenter.
searchCenter
(image, x_ctr_assign, y_ctr_assign, size_window, m=0.2, M=0.8, size_cost=5, theta=[45, 135], ray=False, smooth=2, decimals=2, output_cost=False)[source]¶ This function searches the center in a grid, calculate the cost function of Radon Transform (Pueyo et al., 2015), then interpolate the cost function, get the center which corresponds to the maximum value in the cost function.
 Input:
image: 2d array. x_ctr_assign: the assigned xcenter, or starting xposition; for STIS, the “CRPIX1” header is suggested. x_ctr_assign: the assigned ycenter, or starting yposition; for STIS, the “CRPIX2” header is suggested. size_window: half width of the sampling region; size_window = image.shape[0]/2 is suggested.
m & M: The sampling region will be (M*size_window, m*size_window)U(m*size_window, M*size_window).size_cost: search the center within +/ size_cost pixels, i.e., a square region. theta: the angle range of the sampling region; default: [45, 135] for the antidiagonal and diagonal directions. ray: is the theta a line or a ray? Default: line. smooth: smooth the cost function, for one pixel, replace it by the average of its +/ smooth neighbours; defualt = 2. decimals: the precision of the centers; default = 2 for a precision of 0.01.
 Output:
 x_cen, y_cen

pyklip.instruments.utils.radonCenter.
smoothCostFunction
(costFunction, halfWidth=0)[source]¶ smoothCostFunction will smooth the function within +/ halfWidth, i.e., to replace the value with the average within +/ halfWidth pixel. This function can be genrally used to smooth any 2D matrix. Input:
costFunction: original cost function, a matrix. halfWdith: the half width of the smoothing region, default = 0 pix. Output:
 newFunction: smoothed cost function.
Submodules¶
pyklip.instruments.CHARIS module¶

class
pyklip.instruments.CHARIS.
CHARISData
(filepaths, guess_spot_index=0, guess_spot_locs=None, guess_center_loc=None, skipslices=None, PSF_cube=None, update_hdrs=None, sat_fit_method='global', IWA=None, OWA=None, platecal=False, smooth=True, photcal=False)[source]¶ Bases:
pyklip.instruments.Instrument.Data
A sequence of CHARIS Data. Each CHARISData object has the following fields and functions
Parameters:  filepaths – list of filepaths to files
 skipslices – a list of datacube slices to skip (supply index numbers e.g. [0,1,2,3])
 PSF_cube – 3D array (nl,ny,nx) with the PSF cube to be used in the flux calculation.
 update_hrs – if True, update input file headers by making sat spot measurements. If None, will only update if missing hdr info

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

wcs
¶ Array of N wcs astormetry headers for each image.

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

OWA
¶ a floating point scalar (not array). Specifies to outer working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

wv_indices
¶ Array of N indicies specifying the slice of datacube this frame comes frame (accounts of skipslices) You can use this to index into the header to grab info for the respective slice

spot_flux
¶ Array of N of average satellite spot flux for each frame

dn_per_contrast
¶ Flux calibration factor in units of DN/contrast (divide by image to “calibrate” flux) Can also be thought of as the DN of the unocculted star

flux_units
¶ units of output data [DN, contrast]

prihdrs
¶ Array of N primary headers

exthdrs
¶ Array of N extension headers

bad_sat_spots
¶ a list of up to 4 elements indicating if a sat spot is systematically bad. Indexing is based on sat spot x location. Possible values are 0,1,2,3. [0,3] would mark upper left and lower right sat spots bad

savedata
()[source]¶ save a specified data in the CHARIS datacube format (in the 1st extension header)

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

OWA

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

calibrate_output
(img, spectral=False, units='contrast')[source] Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending on if the spectral flag is set.
Assumes the broadband flux calibration is just multiplication by a single scalar number whereas spectral datacubes may have a separate calibration value for each wavelength
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
 units – currently only support “contrast” w.r.t central star
Returns: calibrated image of the same shape (this is the same object as the input!!!)
Return type: img

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

centralwave
= {}¶

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

flux_zeropt
= {}¶

fpm_diam
= {}¶

generate_host_star_psfs
(boxrad=7)[source]¶ Generates PSF of the unocculted host star for each frame of input data. Only works on spectral mode data.
Parameters: boxrad – the halflength of the size of the extracted PSF (in pixels) Returns: saves PSFs to self.host_star_psfs as an array of size(N,psfy,psfx) where psfy=psfx=2*boxrad + 1

generate_psfs
(boxrad=7, spots_collapse='mean', mask_locs=None, mask_rad=15, mask_ref_ind=0, bg_sub='global')[source]¶ Generates PSF for each frame of input data. Only works on spectral mode data.
Parameters:  boxrad – the halflength of the size of the extracted PSF (in pixels)
 spots_collapse – the method to collapse the psfs in this frame, currently supports: ‘mean’, ‘median’
 mask_locs – ndarray, array of approximate [x, y] pixel locations where bright companions are located. To be masked out when calculating background noise level. Shape (2,) or (n, 2), where n is the number of bright sources that needs to be masked.
 mask_rad – scalar, radius of the mask in pixels.
 mask_ref_ind – the cube index (range: 1ncube) with respect to which mask_locs are specified, used to account for its movement in the FOV due to field rotation.
 bg_sub – the method for background estimate, ‘global’, ‘local’, or None. ‘global’ method is the default method. It estimates the background level at the spot using an annulus around the central star. It is less prone to variances on small scales, and generally better for background that is relatively symmetric azimuthally. ‘local’ method estimates the background using a small annulus around the spot. It is prone to speckle variations near the spot, and should be used when the background level is asymmetric.
Returns: saves PSFs of all frames to self.uncollapsed_psfs. saves time collapsed PSFs to self.psfs as an array of size(nwv,psfy,psfx) where psfy=psfx=2*boxrad + 1

ifs_rotation
= 0.0¶

input
Input Data. Shape of (N, y, x)

ivars
¶

lenslet_scale
= 0.01615¶

lenslet_scale_x
= 0.01615¶

lenslet_scale_x_err
= 5e05¶

lenslet_scale_y
= 0.01615¶

lenslet_scale_y_err
= 7e05¶

measure_spot_over_star_ratio
(opaque_indices=[], boxrad=7, spots_collapse='mean')[source]¶ Obtain satellite spot flux to primary star flux ratio from unsaturated frames
Parameters:  opaque_ind – array of indices of wavelength slices that are opaque and should be interpolated in flux ratio fit
 spots_collapse – the method to collapse the psfs in this frame, currently supports: ‘mean’, ‘median’
Returns: array of spot flux / star flux values for all wavelengths

obs_latitude
= 19.828611111111112¶

obs_longitude
= 155.48055555555555¶

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

readdata
(filepaths, guess_spot_index=0, guess_spot_locs=None, guess_center_loc=None, skipslices=None, PSF_cube=None, update_hdrs=None, sat_fit_method='global', IWA=None, OWA=None, platecal=False, smooth=True, photcal=False)[source] Method to open and read a list of CHARIS data
Parameters:  filespaths – a list of filepaths
 guess_spot_index – the wavelength index for which the initial guess is given
 guess_spot_locs – initial guess of the satellite spot pixel indices. If None, will default to rough guesses for the four satellite spots of a typical CHARIS data cube at the first wavelength slice, in [[x, y],…] format
 guess_center_loc – initial guess of the primary star center in [x, y] format
 skipslices – a list of wavelenegth slices to skip for each datacube (supply index numbers e.g. [0,1,2,3])
 PSF_cube – 3D array (nl,ny,nx) with the PSF cube to be used in the flux calculation.
 update_hrs –
If True, update input file headers with new sat spot measurements. If False, no measuremnets will be carried out and original headers will not be modified If None and sat_fit_method==’global’, all cubes will be updated with new global measurement as
long as there is a single cube without sat spot measurements. If None and sat_fit_method==’local’, only cubes without existing sat spot measurements will be
 updated with new measurements.
 sat_fit_method – ‘global’ or ‘local’
 platecal – bool, whether to calibrate the plate scales of the data, should always be True because the pipeline will automatically skip already calibrated cubes only set to False if running on noncalibrated data and don’t want them calibrated
 smooth – bool, whether to smooth final data slightly
 photcal – bool, whether to photocalibration all frames
 IWA – a floating point scalar (not array). Specifies to inner working angle in pixels
 OWA – a floating point scalar (not array). Specifies to outer working angle in pixels
Returns: Technically none. It saves things to fields of the CHARISData object. See object doc string

savedata
(filepath, data, klipparams=None, filetype=None, zaxis=None, more_keywords=None, center=None, astr_hdr=None, fakePlparams=None, user_prihdr=None, user_exthdr=None, extra_exthdr_keywords=None, extra_prihdr_keywords=None)[source] Save data and header in the first extension header
 Note: In principle, the function only works inside klip_dataset(). In order to use it outside of klip_dataset,
 you need to define the following attributes:
 dataset.output_wcs = np.array([w.deepcopy() if w is not None else None for w in dataset.wcs]) dataset.output_centers = dataset.centers
Parameters:  filepath – path to file to output
 data – 2D or 3D data to save
 klipparams – a string of klip parameters
 filetype – filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”)
 zaxis – a list of values for the zaxis of the datacub (for KL mode cubes currently)
 more_keywords (dictionary) – a dictionary {key: value, key:value} of header keywords and values which will written into the primary header
 astr_hdr – wcs astrometry header
 center – center of the image to be saved in the header as the keywords PSFCENTX and PSFCENTY in pixels. The first pixel has coordinates (0,0)
 fakePlparams – fake planet params
 user_prihdr – User defined primary headers to be used instead
 user_exthdr – User defined extension headers to be used instead
 extra_exthdr_keywords – Fits keywords to be added to the extension header before saving the file
 extra_prihdr_keywords – Fits keywords to be added to the primary header before saving the file

spot_ratio
= {}¶

update_input_cubes
()[source]¶ Updates the input spectral data cubes with the current headers. This is useful to permanately save the measured sat spot locations to disk.

wcs
Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”

pyklip.instruments.CHARIS.
generate_psf
(frame, locations, boxrad=5, spots_collapse='mean', bg_sub=None, mask_locs=None, mask_rad=15)[source]¶ Generates a GPI PSF for the frame based on the satellite spots TODO: normalize psfs? GPI module normalizes these to DN units in the generate_PSF_cube() function.
Parameters:  frame – 2d frame of data
 locations – array of (N,2) containing [x,y] coordinates of all N satellite spots
 boxrad – half length of box to use to pull out PSF
 spots_collapse – the method to collapse the psfs in this frame, currently supports: ‘mean’, ‘median’
 bg_sub – str or None, method used to perform background subtraction
 mask_locs – ndarray, array of approximate [x, y] pixel locations where bright companions are located. To be masked out when calculating background noise level. Shape (2,) or (n, 2), where n is the number of bright sources that needs to be masked.
 mask_rad – scalar, radius of the mask in pixels.
Returns: 2d frame of size (2*boxrad+1, 2*boxrad+1) with average PSF of satellite spots
Return type: genpsf
pyklip.instruments.GPI module¶

class
pyklip.instruments.GPI.
GPIData
(filepaths=None, skipslices=None, highpass=False, meas_satspot_flux=False, numthreads=1, PSF_cube=None, recalc_wvs=True, recalc_centers=True, bad_sat_spots=None, quiet=False)[source]¶ Bases:
pyklip.instruments.Instrument.Data
A sequence of GPI Data. Each GPIData object has the following fields and functions
Parameters:  filepaths – list of filepaths to files
 skipslices – a list of datacube slices to skip (supply index numbers e.g. [0,1,2,3])
 highpass – if True, run a Gaussian high pass filter (default size is sigma=imgsize/10) can also be a number specifying FWHM of box in pixel units
 meas_satspot_flux – if True, remeasure the satellite spot fluxes (would be down after hp filter)
 numthreads – Number of threads to be used. Default 1 sequential sat spot flux calc. If None, numthreads = mp.cpu_count().
 PSF_cube – 3D array (nl,ny,nx) with the PSF cube to be used in the flux calculation.
 recalc_wvs – if True, uses sat spot positions and the central wavelength to recalculate wavelength solution
 recalc_centers – if True, uses a least squares fit and the satellite spots to recalculate the img centers

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

wcs
¶ Array of N wcs astormetry headers for each image.

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

wv_indices
¶ Array of N indicies specifying the slice of datacube this frame comes frame (accounts of skipslices) You can use this to index into the header to grab info for the respective slice

spot_flux
¶ Array of N of average satellite spot flux for each frame

dn_per_contrast
¶ Flux calibration factor in units of DN/contrast (divide by image to “calibrate” flux) Can also be thought of as the DN of the unocculted star

flux_units
¶ units of output data [DN, contrast]

prihdrs
¶ Array of N primary GPI headers (these are written by Gemini Observatory + GPI DRP Pipeline)

exthdrs
¶ Array of N extension GPI headers (these are written by GPI DRP Pipeline)

bad_sat_spots
¶ a list of up to 4 elements indicating if a sat spot is systematically bad. Indexing is based on sat spot x location. Possible values are 0,1,2,3. [0,3] would mark upper left and lower right sat spots bad

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

band
= 'K2'¶

bands
= ['Y', 'J', 'H', 'K1', 'K2']¶

calibrate_output
(img, spectral=False, units='contrast')[source] Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending on if the spectral flag is set.
Assumes the broadband flux calibration is just multiplication by a single scalar number whereas spectral datacubes may have a separate calibration value for each wavelength
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
 units – currently only support “contrast” w.r.t central star
Returns: calibrated image of the same shape (this is the same object as the input!!!)
Return type: img

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

centralwave
= {'H': 1.6473029, 'J': 1.2344508, 'K1': 2.0425281, 'K2': 2.2515272, 'Y': 1.0473831}¶

config
= <configparser.ConfigParser object>¶

configfile
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments/GPI.ini'¶

coronagraph_throughputs
= {'H': (array([0. , 0.007, 0.014, 0.021, 0.028, 0.035, 0.042, 0.049, 0.056, 0.063, 0.07 , 0.077, 0.084, 0.091, 0.098, 0.105, 0.112, 0.119, 0.126, 0.133, 0.14 , 0.147, 0.154, 0.161, 0.168, 0.175, 0.182, 0.189, 0.196, 0.203, 0.21 , 0.217, 0.224, 0.231, 0.238, 0.245, 0.252, 0.259, 0.266, 0.273, 0.28 , 0.287, 0.294, 0.301, 0.308, 0.315, 0.322, 0.329, 0.336, 0.343, 0.35 , 0.357, 0.364, 0.371, 0.378, 0.385, 0.392, 0.399, 0.406, 0.413, 0.42 , 0.427, 0.434, 0.441, 0.448, 0.455, 0.462, 0.469, 0.476, 0.483]), array([8.49097439e04, 8.37072387e04, 9.94797298e04, 1.87987893e03, 4.29712735e03, 9.01064484e03, 1.62980489e02, 2.55382495e02, 3.51561829e02, 4.31943901e02, 4.85002097e02, 5.21188622e02, 5.81902996e02, 7.36763548e02, 1.06662657e01, 1.63638448e01, 2.46742706e01, 3.52153953e01, 4.70442971e01, 5.88925256e01, 6.95196102e01, 7.80514848e01, 8.41778709e01, 8.81453422e01, 9.05699680e01, 9.21635829e01, 9.34900390e01, 9.48369923e01, 9.62256099e01, 9.75188213e01, 9.85565766e01, 9.92533646e01, 9.96276539e01, 9.97723707e01, 9.97998226e01, 9.97957353e01, 9.98010674e01, 9.98201375e01, 9.98409753e01, 9.98529165e01, 9.98538427e01, 9.98481945e01, 9.98415341e01, 9.98368276e01, 9.98341302e01, 9.98323042e01, 9.98305833e01, 9.98288920e01, 9.98272863e01, 9.98254721e01, 9.98229252e01, 9.98193858e01, 9.98151619e01, 9.98109458e01, 9.98073293e01, 9.98044413e01, 9.98019603e01, 9.97994101e01, 9.97964499e01, 9.97929605e01, 9.97889579e01, 9.97845010e01, 9.97797039e01, 9.97748019e01, 9.97701391e01, 9.97660228e01, 9.97625336e01, 9.97594571e01, 9.97564169e01, 9.97531267e01])), 'J': (array([0. , 0.007, 0.014, 0.021, 0.028, 0.035, 0.042, 0.049, 0.056, 0.063, 0.07 , 0.077, 0.084, 0.091, 0.098, 0.105, 0.112, 0.119, 0.126, 0.133, 0.14 , 0.147, 0.154, 0.161, 0.168, 0.175, 0.182, 0.189, 0.196, 0.203, 0.21 , 0.217, 0.224, 0.231, 0.238, 0.245, 0.252, 0.259, 0.266, 0.273, 0.28 , 0.287, 0.294, 0.301, 0.308, 0.315, 0.322, 0.329, 0.336, 0.343, 0.35 , 0.357, 0.364, 0.371, 0.378, 0.385, 0.392, 0.399, 0.406, 0.413, 0.42 , 0.427, 0.434, 0.441, 0.448, 0.455, 0.462, 0.469, 0.476, 0.483]), array([9.25064760e04, 9.59303547e04, 1.71519818e03, 4.88162953e03, 1.21684536e02, 2.35931347e02, 3.63247609e02, 4.60611319e02, 5.14877388e02, 5.92210014e02, 8.45993573e02, 1.45488848e01, 2.51472603e01, 3.95389645e01, 5.53845771e01, 6.97504085e01, 8.04939636e01, 8.71252045e01, 9.06585588e01, 9.26930718e01, 9.44321630e01, 9.62527089e01, 9.79222136e01, 9.91025469e01, 9.97002942e01, 9.98854386e01, 9.99042027e01, 9.99120466e01, 9.99423160e01, 9.99709810e01, 9.99812000e01, 9.99788653e01, 9.99759579e01, 9.99768346e01, 9.99796005e01, 9.99825147e01, 9.99855602e01, 9.99883553e01, 9.99896453e01, 9.99889790e01, 9.99874732e01, 9.99865000e01, 9.99862971e01, 9.99861913e01, 9.99856444e01, 9.99845958e01, 9.99830614e01, 9.99809744e01, 9.99784894e01, 9.99760176e01, 9.99737097e01, 9.99711181e01, 9.99676483e01, 9.99632841e01, 9.99586112e01, 9.99540710e01, 9.99493859e01, 9.99438476e01, 9.99370762e01, 9.99293813e01, 9.99214762e01, 9.99140383e01, 9.99075449e01, 9.99022771e01, 9.98982186e01, 9.98949328e01, 9.98917133e01, 9.98879924e01, 9.98836315e01, 9.98788239e01])), 'K1': (array([0. , 0.007, 0.014, 0.021, 0.028, 0.035, 0.042, 0.049, 0.056, 0.063, 0.07 , 0.077, 0.084, 0.091, 0.098, 0.105, 0.112, 0.119, 0.126, 0.133, 0.14 , 0.147, 0.154, 0.161, 0.168, 0.175, 0.182, 0.189, 0.196, 0.203, 0.21 , 0.217, 0.224, 0.231, 0.238, 0.245, 0.252, 0.259, 0.266, 0.273, 0.28 , 0.287, 0.294, 0.301, 0.308, 0.315, 0.322, 0.329, 0.336, 0.343, 0.35 , 0.357, 0.364, 0.371, 0.378, 0.385, 0.392, 0.399, 0.406, 0.413, 0.42 , 0.427, 0.434, 0.441, 0.448, 0.455, 0.462, 0.469, 0.476, 0.483]), array([4.43740966e04, 4.63263651e04, 6.12527612e04, 1.15138792e03, 2.46611383e03, 4.98412551e03, 9.04319870e03, 1.47361281e02, 2.17810099e02, 2.94856331e02, 3.68670612e02, 4.29483107e02, 4.71906427e02, 4.99528495e02, 5.28248341e02, 5.86847162e02, 7.13856893e02, 9.50802659e02, 1.33306487e01, 1.88054605e01, 2.59068750e01, 3.43598022e01, 4.36701624e01, 5.32063723e01, 6.23130818e01, 7.04292655e01, 7.71816449e01, 8.24316974e01, 8.62678337e01, 8.89494585e01, 9.08218907e01, 9.22269800e01, 9.34324321e01, 9.45946134e01, 9.57581280e01, 9.68846583e01, 9.78966600e01, 9.87200822e01, 9.93139118e01, 9.96810130e01, 9.98617495e01, 9.99169062e01, 9.99082127e01, 9.98834181e01, 9.98695323e01, 9.98741342e01, 9.98919452e01, 9.99128742e01, 9.99283706e01, 9.99345182e01, 9.99319900e01, 9.99241102e01, 9.99145492e01, 9.99057656e01, 9.98985814e01, 9.98926362e01, 9.98871659e01, 9.98816057e01, 9.98757918e01, 9.98698265e01, 9.98638226e01, 9.98577380e01, 9.98513802e01, 9.98445348e01, 9.98371003e01, 9.98291376e01, 9.98208138e01, 9.98122933e01, 9.98036523e01, 9.97948661e01])), 'K2': (array([0. , 0.007, 0.014, 0.021, 0.028, 0.035, 0.042, 0.049, 0.056, 0.063, 0.07 , 0.077, 0.084, 0.091, 0.098, 0.105, 0.112, 0.119, 0.126, 0.133, 0.14 , 0.147, 0.154, 0.161, 0.168, 0.175, 0.182, 0.189, 0.196, 0.203, 0.21 , 0.217, 0.224, 0.231, 0.238, 0.245, 0.252, 0.259, 0.266, 0.273, 0.28 , 0.287, 0.294, 0.301, 0.308, 0.315, 0.322, 0.329, 0.336, 0.343, 0.35 , 0.357, 0.364, 0.371, 0.378, 0.385, 0.392, 0.399, 0.406, 0.413, 0.42 , 0.427, 0.434, 0.441, 0.448, 0.455, 0.462, 0.469, 0.476, 0.483]), array([4.74253298e04, 6.13139462e04, 1.14431434e03, 2.37218828e03, 4.68282734e03, 8.39309813e03, 1.35974807e02, 2.00599860e02, 2.71957263e02, 3.41714521e02, 4.01272254e02, 4.44867503e02, 4.72900233e02, 4.94593568e02, 5.29080051e02, 6.04246224e02, 7.53144911e02, 1.00839917e01, 1.39561560e01, 1.92722309e01, 2.59821846e01, 3.38497942e01, 4.24765367e01, 5.13580607e01, 5.99621911e01, 6.78121032e01, 7.45570075e01, 8.00158092e01, 8.41858923e01, 8.72175952e01, 8.93628306e01, 9.09116028e01, 9.21316624e01, 9.32240842e01, 9.43021540e01, 9.53943036e01, 9.64658557e01, 9.74505555e01, 9.82820021e01, 9.89169893e01, 9.93464754e01, 9.95940995e01, 9.97055924e01, 9.97342269e01, 9.97274175e01, 9.97180604e01, 9.97219577e01, 9.97405259e01, 9.97665869e01, 9.97906533e01, 9.98056499e01, 9.98090700e01, 9.98026852e01, 9.97907114e01, 9.97776015e01, 9.97664174e01, 9.97582345e01, 9.97525001e01, 9.97479135e01, 9.97433084e01, 9.97381650e01, 9.97326394e01, 9.97272406e01, 9.97224083e01, 9.97182253e01, 9.97143809e01, 9.97103511e01, 9.97056647e01, 9.97001071e01, 9.96937687e01])), 'Y': (array([0. , 0.007, 0.014, 0.021, 0.028, 0.035, 0.042, 0.049, 0.056, 0.063, 0.07 , 0.077, 0.084, 0.091, 0.098, 0.105, 0.112, 0.119, 0.126, 0.133, 0.14 , 0.147, 0.154, 0.161, 0.168, 0.175, 0.182, 0.189, 0.196, 0.203, 0.21 , 0.217, 0.224, 0.231, 0.238, 0.245, 0.252, 0.259, 0.266, 0.273, 0.28 , 0.287, 0.294, 0.301, 0.308, 0.315, 0.322, 0.329, 0.336, 0.343, 0.35 , 0.357, 0.364, 0.371, 0.378, 0.385, 0.392, 0.399, 0.406, 0.413, 0.42 , 0.427, 0.434, 0.441, 0.448, 0.455, 0.462, 0.469, 0.476, 0.483]), array([0.00126878, 0.00135917, 0.00287518, 0.00872691, 0.02064626, 0.03569239, 0.04721173, 0.05361069, 0.06866578, 0.12011037, 0.23112447, 0.39778805, 0.58427982, 0.74362636, 0.84750364, 0.89977528, 0.92470934, 0.94427994, 0.96538791, 0.98373541, 0.99461201, 0.99850757, 0.99910112, 0.99921351, 0.99951706, 0.99972671, 0.99974038, 0.9997049 , 0.99970786, 0.99973339, 0.99976638, 0.99980435, 0.99982675, 0.9998186 , 0.99979679, 0.99978469, 0.99978277, 0.99977871, 0.99976696, 0.99974748, 0.9997205 , 0.99969075, 0.99966534, 0.99964255, 0.9996124 , 0.99957165, 0.99952919, 0.99949363, 0.99946411, 0.9994362 , 0.99940887, 0.99938193, 0.99935295, 0.99932133, 0.99929076, 0.99926393, 0.99923786, 0.99920742, 0.99917087, 0.99912977, 0.9990861 , 0.99904193, 0.99899998, 0.99896186, 0.99892673, 0.99889297, 0.99886043, 0.99882978, 0.99880053, 0.99877085]))}¶

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

flux_zeropt
= {'H': 1.138292e06, 'J': 3.1868226e06, 'K1': 5.3415049e07, 'K2': 3.7012793e07, 'Y': 5.1508988e06}¶

fpm_diam
= {'H': 17.37165454417061, 'J': 12.993432667184521, 'K1': 21.608643457382954, 'K2': 21.608643457382954, 'Y': 11.016171174352094}¶

generate_psf_cube
(boxw=20, threshold=0.01, tapersize=0, zero_neg=False, same_wv_only=True)[source]¶ Generates an average PSF from all frames of input data. Only works on spectral mode data. Overall cube is normalized to have the average sat spot spectrum in DN units. The spectrum is built by combining all the estimated sat spot fluxes. It can take a while as this function is not parallelized…
The center of the PSF is exactly on the central pixel of the PSF. The center pixel index is always (nx/2,nx/2) assuming integer division.
The output PSF cube shape doesn’t depend on the underlying sat spot flux calculation. The sat spot fluxes are only used to set the spectrum of the PSF at the very end.
//!CAUTION 1: I think same_wv_only = False has a bug even in the rescaling of the coordinates //!CAUTION 2: Currently hard coded assuming 37 spectral channels!!!
This function is not compatible with skipslices.Parameters:  boxw – the width the extracted PSF (in pixels). Should be bigger than 20 because there is an interpolation of the background by a plane which is then subtracted to remove linear biases.
 threshold – fractional pixel value of max pixel value below which everything gets set to 0’s
 tapersize – if > 0, apply a hann window on the edges with size equal to this argument
 zero_neg – if True, set all negative values to zero instead
 same_wv_only – If true (default), it only combines sat spot from the same wavelength. Otherwise it rescales them to each wavelengths. CAUTION: I think same_wv_only = False has a bug even in the rescaling of the coordinates
Returns: ,:] is the PSF for a given wavelength.
Return type: A cube of shape 37*boxw*boxw. Each slice [k,

generate_psfs
(boxrad=7, time_collapse=True)[source]¶ Generates PSF for each frame of input data. Only works on spectral mode data.
Parameters:  boxrad – the halflength of the size of the extracted PSF (in pixels)
 time_collapse – if True, averages PSF in time.
Returns: saves PSFs to self.psfs as an array of size(N_wvs, psfy, psfx) where psfy=psfx=2*boxrad + 1 unless time_collapse=False, in which case it has shape (N_cubes, N_wvs, psfy, psfx).

get_radial_psf
(save=None)[source]¶ Return a pure radial PSF by averaging the original psf. The new PSF is invariant by rotation. A call to generate_psf_cube() is required prior to calling this function. The center pixel index is always (nx/2,nx/2) assuming integer division.
Parameters: save – Optionally automatically save the radial psf cube as a fits file with filename: save+”original_radial_PSF_cube.fits” Returns: a (37,nx,nx) cube with the radial psf. Return type: rad_psf_cube

hdulist
= [<astropy.io.fits.hdu.image.PrimaryHDU object>, <astropy.io.fits.hdu.table.BinTableHDU object>]¶

ifs_rotation
= 23.5¶

input
Input Data. Shape of (N, y, x)

lenslet_scale
= 0.014161¶

observatory_latitude
= 30.24075¶

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

package_directory
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments'¶

profile_dir
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments/GPI_profiles'¶

profile_filename
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments/GPI_profiles/gpi_offaxis_throughput_K2.fits'¶

readdata
(filepaths, skipslices=None, highpass=False, meas_satspot_flux=False, numthreads=1, PSF_cube=None, recalc_wvs=True, recalc_centers=True, bad_sat_spots=None, quiet=False)[source] Method to open and read a list of GPI data
Parameters:  filespaths – a list of filepaths
 skipslices – a list of wavelenegth slices to skip for each datacube (supply index numbers e.g. [0,1,2,3])
 highpass – if True, run a Gaussian high pass filter (default size is sigma=imgsize/10) can also be a number specifying FWHM of box in pixel units
 meas_satspot_flux – if True, remeasure the satellite spot fluxes (would be done after hp filter)
 numthreads – Number of threads to be used. Default 1 sequential sat spot flux calc. If None, numthreads = mp.cpu_count().
 PSF_cube – 3D array (nl,ny,nx) with the PSF cube to be used in the flux calculation.
 recalc_wvs – if True, uses sat spot positions and the central wavelength to recalculate wavelength solution
 recalc_centers – if True, uses a least squares fit and the satellite spots to recalculate the img centers
 bad_sat_spots – a list of up to 4 elements indicating if a sat spot is systematically bad.
Returns: Technically none. It saves things to fields of the GPIData object. See object doc string

savedata
(filepath, data, klipparams=None, filetype=None, zaxis=None, more_keywords=None, center=None, astr_hdr=None, fakePlparams=None, user_prihdr=None, user_exthdr=None, extra_exthdr_keywords=None, extra_prihdr_keywords=None)[source] Save data in a GPIlike fashion. Aka, data and header are in the first extension header
 Note: In principle, the function only works inside klip_dataset(). In order to use it outside of klip_dataset,
 you need to define the following attributes:
 dataset.output_wcs = np.array([w.deepcopy() if w is not None else None for w in dataset.wcs]) dataset.output_centers = dataset.centers
Parameters:  filepath – path to file to output
 data – 2D or 3D data to save
 klipparams – a string of klip parameters
 filetype – filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”)
 zaxis – a list of values for the zaxis of the datacub (for KL mode cubes currently)
 more_keywords (dictionary) – a dictionary {key: value, key:value} of header keywords and values which will written into the primary header
 astr_hdr – wcs astrometry header
 center – center of the image to be saved in the header as the keywords PSFCENTX and PSFCENTY in pixels. The first pixel has coordinates (0,0)
 fakePlparams – fake planet params
 user_prihdr – User defined primary headers to be used instead
 user_exthdr – User defined extension headers to be used instead
 extra_exthdr_keywords – Fits keywords to be added to the extension header before saving the file
 extra_prihdr_keywords – Fits keywords to be added to the primary header before saving the file

seps
= array([0. , 0.007, 0.014, 0.021, 0.028, 0.035, 0.042, 0.049, 0.056, 0.063, 0.07 , 0.077, 0.084, 0.091, 0.098, 0.105, 0.112, 0.119, 0.126, 0.133, 0.14 , 0.147, 0.154, 0.161, 0.168, 0.175, 0.182, 0.189, 0.196, 0.203, 0.21 , 0.217, 0.224, 0.231, 0.238, 0.245, 0.252, 0.259, 0.266, 0.273, 0.28 , 0.287, 0.294, 0.301, 0.308, 0.315, 0.322, 0.329, 0.336, 0.343, 0.35 , 0.357, 0.364, 0.371, 0.378, 0.385, 0.392, 0.399, 0.406, 0.413, 0.42 , 0.427, 0.434, 0.441, 0.448, 0.455, 0.462, 0.469, 0.476, 0.483])¶

spectral_collapse
(collapse_channels=1, align_frames=True, aligned_center=None, numthreads=None)[source]¶ GPI wrapper of spectral_collapse(). Adds GPI values to collapse
Collapses the dataset spectrally, bining the data into the desired number of output wavelengths. This bins each cube individually; it does not bin the data tempoarally. If number of wavelengths / output channels is not a whole number, some output channels will have more frames that went into the collapse
Parameters:  collapse_channels (int) – number of output channels to evenlyish collapse the dataset into. Default is 1 (broadband)
 align_frames (bool) – if True, aligns each channel before collapse so that they are centered properly
 aligned_center – Array of shape (2) [x_cent, y_cent] for the centering the images to a given value if align_frames is True
 numthreads (bool,int) – number of threads to parallelize align and scale. If None, use default which is all of them

spot_ratio
= {'H': 0.000174, 'J': 0.000184, 'K1': 0.000212, 'K2': 0.0001905, 'Y': 0.00016}¶

spot_ratio2
= {'J': 0.000147, 'Y': 0.000142}¶

spot_ratio_h
= {'J': 0.000192, 'K1': 0.00018, 'K2': 0.0001905, 'Y': 0.000171}¶

spot_ratio_h2
= {'J': 0.000141, 'Y': 0.000137}¶

throughputs
= array([4.74253298e04, 6.13139462e04, 1.14431434e03, 2.37218828e03, 4.68282734e03, 8.39309813e03, 1.35974807e02, 2.00599860e02, 2.71957263e02, 3.41714521e02, 4.01272254e02, 4.44867503e02, 4.72900233e02, 4.94593568e02, 5.29080051e02, 6.04246224e02, 7.53144911e02, 1.00839917e01, 1.39561560e01, 1.92722309e01, 2.59821846e01, 3.38497942e01, 4.24765367e01, 5.13580607e01, 5.99621911e01, 6.78121032e01, 7.45570075e01, 8.00158092e01, 8.41858923e01, 8.72175952e01, 8.93628306e01, 9.09116028e01, 9.21316624e01, 9.32240842e01, 9.43021540e01, 9.53943036e01, 9.64658557e01, 9.74505555e01, 9.82820021e01, 9.89169893e01, 9.93464754e01, 9.95940995e01, 9.97055924e01, 9.97342269e01, 9.97274175e01, 9.97180604e01, 9.97219577e01, 9.97405259e01, 9.97665869e01, 9.97906533e01, 9.98056499e01, 9.98090700e01, 9.98026852e01, 9.97907114e01, 9.97776015e01, 9.97664174e01, 9.97582345e01, 9.97525001e01, 9.97479135e01, 9.97433084e01, 9.97381650e01, 9.97326394e01, 9.97272406e01, 9.97224083e01, 9.97182253e01, 9.97143809e01, 9.97103511e01, 9.97056647e01, 9.97001071e01, 9.96937687e01])¶

wcs
Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”

pyklip.instruments.GPI.
calc_center
(prihdr, exthdr, wvs, ignoreslices=None, skipslices=None, bad_sat_spots=None)[source]¶ calcualte the center position of a spectral data cube
Parameters:  prihdr – primary GPI header
 exthdr – extention GPI header
 wvs – wvs of the datacube
 ignoreslices – slices to ignore in the fit. A list of wavelength slice indicies to ignore if none, ignores slices 0,1, len2, len1 (first and last two)
 skipslices – slices that were already skipped in processing
 bad_sat_stots – of the 4 sat spots, which are bad and should be ignored. Indexed 03 based on x coordinate
Returns: star center
Return type: centx, centy

pyklip.instruments.GPI.
calc_center_least_squares
(xpos, ypos, wvs, orderx, ordery, displacement)[source]¶ calcualte the center position, linear least squares fit to 4 parameters
Parameters:  xpos – array of length n of x positions of satellite spots
 ypos – array of length n of y positions of satellite spots
 wvs – the wavelength of each pair of positoins
 orderx – the x order (can be 1 or 1 in this case. 1 is under the center, 1 is above the center)
 ordery – the y order (e.g. pos0 is at pox=1, posy=1).
 displacment – the displacement from zenith
Returns: four fit parameters (xcenter, ycenter, adrx, adry). xcenters = xcenter + ardx * displacement

pyklip.instruments.GPI.
generate_psf
(frame, locations, boxrad=5, medianboxsize=30)[source]¶ Generates a GPI PSF for the frame based on the satellite spots
Parameters:  frame – 2d frame of data
 location – array of (N,2) containing [x,y] coordinates of all N satellite spots
 boxrad – half length of box to use to pull out PSF
 medianboxsize – size in pixels of box for median filter
Returns: 2d frame of size (2*boxrad+1, 2*boxrad+1) with average PSF of satellite spots
Return type: genpsf

pyklip.instruments.GPI.
get_gpi_wavelength_sampling
(filter_name)[source]¶ Return GPI wavelength sampling for a given band.
Parameters: filter_name – ‘H’, ‘J’, ‘K1’, ‘K2’, ‘Y’. Wavelength samples are linearly spaced between the first and the last wavelength of the band. Returns: is the gpi sampling of the considered band in micrometer. Return type: wavelengths

pyklip.instruments.GPI.
measure_sat_spot_fluxes
(img, spots_x, spots_y, psfs_func_list=None, wave_index=None, residuals=False)[source]¶ Measure satellite spot peak fluxes using a Gaussian matched filter
Parameters:  img – 2D frame with 4 sat spots
 spots_x – list of 4 satellite spot x coordinates
 spots_y – list of 4 satellite spot y coordinates
 psfs_func_list – List of spline fit function for the PSF_cube. If None (default) a gaussian fit is used.
 wave_index – Index of the current wavelength. In [0,36] for GPI. Only used when psfs_func_list is not None.
 residuals – If True (Default = False) then calculate the residuals of the sat spot fit (gaussian or PSF cube).
Returns: list of 4 satellite spot fluxes
Return type: spots_f

pyklip.instruments.GPI.
recalculate_sat_spot_fluxes
(dataset, skipslices=None, numthreads=1, PSF_cube=None, residuals=False)[source]¶ Recalculate the satellite spots fluxes.
Parameters:  dataset – GPIData object.
 skipslices – a list of datacube slices to skip (supply index numbers e.g. [0,1,2,3]) WARNING! SKIPSLICES features hasn’t been tested with this function.
 numthreads – Number of threads to be used. Default 1 sequential sat spot flux calc. If None, numthreads = mp.cpu_count().
 PSF_cube – 3D array (nl,ny,nx) with the PSF cube to be used in the flux calculation.
 residuals – If True (Default = False) then calculate the residuals of the sat spot fit (gaussian or PSF cube).
Returns: The list of sat spot fluxes. Can be used to redefine dataset.spot_flux.
Return type: spot_fluxes

pyklip.instruments.GPI.
rescale_wvs
(exthdrs, wvs, refwv=None, skipslices=None, bad_sat_spots=None)[source]¶ Hack to try to fix wavelength scaling issue. This will calculate the scaling between channels, and adjust the wavelength solution such that the scaling comes out linear in scaling vs wavelength. Finicky  requires that all images in the dataset have the same number of wavelength channels :param exthdrs: a list of extension headers, from a pyklip.instrument dataset :param wvs: a list of wvs (can repeat. This function will only look at the first cube’s wavelenghts) :param refwv: integer index of the channel to normalize the scaling :type refwv: optional :param skipslices: list of skipped wavelength slices (needs to be consistent with the ones skipped by wv)
Returns: Nlambda*Nexthdrs array of wavelengths that produce a linear plot of wavelength vs scaling Return type: scaled_wvs

pyklip.instruments.GPI.
subtract_satspots
(slice, slice_id, spots_xloc_thisslice, spots_yloc_thisslice, center_thisslice, psfs_func_list)[source]¶ Subtract the satellite spots in a GPI image.
 Inputs:
slice: 2d image slice_id: index of the image from the GPI cube. Used to select the correct PSF from psfs_func_list. spots_xloc_thisslice: list of x position of the sat spots spots_yloc_thisslice: list of y position of the sat spots center_thisslice: (x0,y0) image center psfs_func_list: List of spline fit function for the PSF_cube.
 Can be computed as follow:
numwv,ny_psf,nx_psf = PSF_cube.shape x_psf_grid, y_psf_grid = np.meshgrid(np.arange(nx_psf * 1.)nx_psf//2,np.arange(ny_psf* 1.)ny_psf//2) psfs_func_list = [] from scipy import interpolate for wv_index in range(numwv):
model_psf = PSF_cube[wv_index, :, :] psfs_func_list.append(interpolate.LSQBivariateSpline(x_psf_grid.ravel(),y_psf_grid.ravel(),model_psf.ravel(),
x_psf_grid[0,0:nx_psf1]+0.5,y_psf_grid[0:ny_psf1,0]+0.5))
Output: sat spot subtracted image.
pyklip.instruments.Instrument module¶

class
pyklip.instruments.Instrument.
Data
[source]¶ Bases:
object
Abstract Class with the required fields and methods that need to be implemented

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N input centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

wcs
¶ Array of N wcs astormetry headers for each input image.

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

OWA
¶ (optional) specifies outer working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

output_centers
¶ Array of shape (N,2) for N output centers. Also coresponds to FM centers (does not need to be implemented)

output_wcs
¶ Array of N wcs astrometry headers for each output image (does not need to be implemneted)

creator
¶ (optional) string for creator of the data (used to identify pipelines that call pyklip)

klipparams
¶ (optional) a string that saves the most recent KLIP parameters

flipx
¶ (optional) False by default. Determines whether a relfection about the x axis is necessary to rotate image Northup East left

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

calibrate_output
(img, spectral=False)[source] Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending on if the spectral flag is set.
Assumes the broadband flux calibration is just multiplication by a single scalar number whereas spectral datacubes may have a separate calibration value for each wavelength
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
Returns: calibrated image of the same shape
Return type: calib_img

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

input
Input Data. Shape of (N, y, x)

numwvs
¶

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

readdata
(filepaths)[source] Reads in the data from the files in the filelist and writes them to fields

static
savedata
(self, filepath, data, klipparams=None, filetype='', zaxis=None, more_keywords=None)[source] Saves data for this instrument
Parameters:  filepath – filepath to save to
 data – data to save
 klipparams – a string of KLIP parameters. Write it to the ‘PSFPARAM’ keyword
 filtype – type of file (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”). Wrriten to ‘FILETYPE’ keyword
 zaxis – a list of values for the zaxis of the datacub (for KL mode cubes currently)
 more_keywords (dictionary) – a dictionary {key: value, key:value} of header keywords and values which will written into the primary header

spectral_collapse
(collapse_channels=1, align_frames=True, aligned_center=None, numthreads=None, additional_params=None)[source]¶ Collapses the dataset spectrally, bining the data into the desired number of output wavelengths. This bins each cube individually; it does not bin the data tempoarally. If number of wavelengths / output channels is not a whole number, some output channels will have more frames that went into the collapse
Parameters:  collapse_channels (int) – number of output channels to evenlyish collapse the dataset into. Default is 1 (broadband)
 align_frames (bool) – if True, aligns each channel before collapse so that they are centered properly
 aligned_center – Array of shape (2) [x_cent, y_cent] for the centering the images to a given value
 numthreads (bool,int) – number of threads to parallelize align and scale. If None, use default which is all of them
 additional_params (list of str) – other dataset parameters to collapse. Assume each variable has first dimension of Nframes

wcs
Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”


class
pyklip.instruments.Instrument.
GenericData
(input_data, centers, parangs=None, wvs=None, IWA=0, filenames=None, flipx=False)[source]¶ Bases:
pyklip.instruments.Instrument.Data
Basic class to interface with a basic direct imaging dataset
Parameters:  input_data – either a 1D list of filenames to read in, or a 3D cube of all data (N, y, x)
 centers – array of shape (N,2) for N centers in the format [x_cent, y_cent]
 parangs – Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]
 wvs – Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”
 IWA – a floating point scalar (not array). Specifies to inner working angle in pixels
 filenames – Array of size N for the actual filepath of the file that corresponds to the data
 flipx (boo) – if True, the input images are righthanded (East clockwise of North) and need to be flipped for NorthupEastleft

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

wcs
¶ Array of N wcs astormetry headers for each image.

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

calibrate_output
(img, spectral=False)[source]¶ Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending on if the spectral flag is set.
Assumes the broadband flux calibration is just multiplication by a single scalar number whereas spectral datacubes may have a separate calibration value for each wavelength
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
Returns: calibrated image of the same shape
Return type: calib_img

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

input
Input Data. Shape of (N, y, x)

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

readdata
(filepaths)[source]¶ Reads in the data from the files in the filelist and writes them to fields.

savedata
(filepath, data, klipparams=None, filetype='', zaxis=None, more_keywords=None)[source]¶ Saves data for this instrument
Parameters:  filepath – filepath to save to
 data – data to save
 klipparams – a string of KLIP parameters. Write it to the ‘PSFPARAM’ keyword
 filtype – type of file (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”). Wrriten to ‘FILETYPE’ keyword
 zaxis – a list of values for the zaxis of the datacub (for KL mode cubes currently)
 more_keywords (dictionary) – a dictionary {key: value, key:value} of header keywords and values which will written into the primary header

wcs
Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”
pyklip.instruments.MagAO module¶

class
pyklip.instruments.MagAO.
MagAOData
(filepaths=None, highpass=False)[source]¶ Bases:
object
A sequence of P1640 Data. Each P1640Data object has the following fields and functions :param filepaths: list of filepaths to files :param skipslices: a list of datacube slices to skip (supply index numbers e.g. [0,1,2,3]) :param corefilepaths: a list of filepaths to core (i.e. unocculted) files, for contrast calc :param spot_directory: (None) path to the directory where the spot positions are stored. Defaults to P1640.ini val

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

wcs
¶ Array of N wcs astormetry headers for each image.

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

spot_flux
¶ Array of N of average satellite spot flux for each frame

contrast_scaling
¶ Flux calibration factors (multiply by image to “calibrate” flux)

flux_units
¶ units of output data [DN, contrast]

prihdrs
¶ not used for P1640, set to None

exthdrs
¶ Array of N P1640 headers (these are written by the P1640 cube extraction pipeline)

savedata
()[source]¶ save a specified data in the P1640 datacube format (in the 1st extension header)

IWA

OWA
¶

PAs

band
= 'Ys'¶

bands
= ['HA', 'CONT', "z'", "r'", "i'", 'Ys']¶

calibrate_output
(img, spectral=False, units='contrast')[source]  Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending
on if the spectral flag is set.
Assumes the broadband flux calibration is just multiplication by a single scalar number whereas spectral datacubes may have a separate calibration value for each wavelength
 Args:
 img: unclaibrated image.
 If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
spectral: if True, this is a spectral datacube. Otherwise, it is a broadband image. units: currently only support “contrast” w.r.t central star
 Return:
 img: calibrated image of the same shape (this is the same object as the input!!!)

centers

centralwave
= {'CONT': 0.6428, 'HA': 0.6564, 'Ys': 0.98584704, "i'": 0.76691462, "r'": 0.62645846, "z'": 0.90969793}¶

config
= <configparser.ConfigParser object>¶

configfile
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments/MagAO.ini'¶

filenums

flipx
¶

flux_zeropt
= {'CONT': 0.0, 'HA': 0.0, 'Ys': 0.0, "i'": 0.0, "r'": 0.0, "z'": 0.0}¶

ghstpeak_ratio
= {"cont'": 0.0042, "i'": 0.001998, "line'": 0.0042, "z'": 0.00122}¶

ifs_rotation
= 0.497¶

input

lenslet_scale
= 0.00795¶

observatory_latitude
= 29.0146¶

output

package_directory
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments'¶

readdata
(filepaths, highpass=False)[source] Method to open and read a list of MagAO data highpass: if True, run a Gaussian high pass filter (default size is sigma=imgsize/10)
can also be a number specifying FWHM of box in pixel units

savedata
(filepath, data, klipparams=None, filetype=None, zaxis=None, center=None, astr_hdr=None, fakePlparams=None, more_keywords=None)[source] Save data in a GPIlike fashion. Aka, data and header are in the first extension header
Inputs: filepath: path to file to output data: 2D or 3D data to save klipparams: a string of klip parameters filetype: filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”) zaxis: a list of values for the zaxis of the datacub (for KL mode cubes currently) astr_hdr: wcs astrometry header (None for NIRC2) center: center of the image to be saved in the header as the keywords PSFCENTX and PSFCENTY in pixels. The first pixel has coordinates (0,0) fakePlparams: fake planet params

wcs

wvs

pyklip.instruments.NIRC2 module¶

class
pyklip.instruments.NIRC2.
NIRC2Data
(filepaths=None, highpass=False, find_star='auto', meas_star_flux=False, guess_star=None)[source]¶ Bases:
pyklip.instruments.Instrument.Data
A sequence of Keck NIRC2 ADI Data. Each NIRC2Data object has the following fields and functions
Parameters:  filepaths – list of filepaths to files
 highpass – if True, run a Gaussian high pass filter (default size is sigma=imgsize/10) can also be a number specifying FWHM of box in pixel units
 find_star – (default) ‘auto’ will first try to get the star center coordinates from the FITS header PSFCENTX & PSFCENTY keywords, and if that fails it will do a Radon transform to locate the star via the diffraction spikes (and store the star center in the header for future use). True will force the Radon transform; False will skip the Radon transform even if no center is found in the header.
 guess_star – (default) None. Otherwise an [x, y] coordinate guess for where the star is for find_star. The code will automatially guess the star location based on the FPM otherwise.

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

wcs
¶ Array of N wcs astormetry headers for each image.

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

creator
¶ string for creator of the data (used to identify pipelines that call pyklip)

klipparams
¶ a string that saves the most recent KLIP parameters

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

band
= 'Ms'¶

bands
= ['J', 'H', 'K', 'Ks', 'Kp', 'Lp', 'Ms']¶

calibrate_data
(units='contrast')[source]¶ Calibrates the flux of the output of PSF subtracted data.
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
 units – currently only support “contrast” w.r.t central star
Returns: calibrated image of the same shape (this is the same object as the input!!!)
Return type: img

calibrate_output
(img, spectral=False, units='contrast')[source] Calibrates the flux of the output of PSF subtracted data.
Assumes the broadband flux calibration is just multiplication by a single scalar number whereas spectral datacubes may have a separate calibration value for each wavelength
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
 units – currently only support “contrast” w.r.t central star
Returns: calibrated image of the same shape (this is the same object as the input!!!)
Return type: img

cam
= 'wide'¶

cameras
= ['narrow_pre150413', 'narrow_post150413', 'medium', 'wide']¶

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

centralwave
= {'H': 1.633, 'J': 1.248, 'K': 2.196, 'Kp': 2.124, 'Ks': 2.146, 'Lp': 3.776, 'Ms': 4.67}¶

config
= <configparser.ConfigParser object>¶

configfile
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments/NIRC2.ini'¶

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

flux_zeropt
= {'H': 1.2e06, 'J': 3.039e06, 'K': 4.087e07, 'Kp': 4.67e07, 'Ks': 4.461e07, 'Lp': 5.338e08, 'Ms': 2.285e08}¶

fpm
= 'corona2000'¶

fpm_diam
= {'corona100': 0.1, 'corona1000': 1.0, 'corona150': 0.15, 'corona1500': 1.5, 'corona200': 0.2, 'corona2000': 2.0, 'corona300': 0.3, 'corona400': 0.4, 'corona600': 0.6, 'corona800': 0.8}¶

fpm_yx
= {'medium_corona100': (None, None), 'medium_corona1000': (None, None), 'medium_corona150': (None, None), 'medium_corona1500': (None, None), 'medium_corona200': (None, None), 'medium_corona2000': (None, None), 'medium_corona300': (None, None), 'medium_corona400': (None, None), 'medium_corona600': (None, None), 'medium_corona800': (None, None), 'narrow_corona100': (None, None), 'narrow_corona1000': (None, None), 'narrow_corona150': (None, None), 'narrow_corona1500': (None, None), 'narrow_corona200': (465, 511), 'narrow_corona2000': (None, None), 'narrow_corona300': (None, None), 'narrow_corona400': (413, 508), 'narrow_corona600': (426, 507), 'narrow_corona800': (None, None), 'wide_corona100': (None, None), 'wide_corona1000': (None, None), 'wide_corona150': (None, None), 'wide_corona1500': (None, None), 'wide_corona200': (None, None), 'wide_corona2000': (None, None), 'wide_corona300': (None, None), 'wide_corona400': (None, None), 'wide_corona600': (None, None), 'wide_corona800': (None, None)}¶

fpms
= ['corona100', 'corona150', 'corona200', 'corona300', 'corona400', 'corona600', 'corona800', 'corona1000', 'corona1500', 'corona2000']¶

input
Input Data. Shape of (N, y, x)

lenslet_scales
= {'medium': 0.019829, 'narrow_post150413': 0.009971, 'narrow_pre150413': 0.009952, 'wide': 0.039686}¶

observatory_latitude
= 19.82525¶

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

package_directory
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments'¶

pupil
= 'open'¶

pupil_diam
= {'incircle': 8.792, 'largehex': 0.0, 'open': 9.96, 'smallhex': 0.0}¶

pupils
= ['incircle', 'largehex', 'smallhex', 'open']¶

readdata
(filepaths, highpass=False, find_star='auto', meas_star_flux=False, guess_star=None)[source] Method to open and read a list of NIRC2 data
Parameters:  filespaths – a list of filepaths
 highpass – if True, run a Gaussian high pass filter (default size is sigma=imgsize/10) can also be a number specifying FWHM of box in pixel units
 find_star – (default) ‘auto’ will first try to get the star center coordinates from the FITS header PSFCENTX & PSFCENTY keywords, and if that fails it will do a Radon transform to locate the star via the diffraction spikes (and store the star center in the header for future use). True will force the Radon transform; False will skip the Radon transform even if no center is found in the header.
 guess_star: (default) None. Otherwise an [x, y] coordinate guess for where the star is for find_star.
 The code will automatially guess the star location based on the FPM otherwise.
Returns: Technically none. It saves things to fields of the NIRC2Data object. See object doc string

savedata
(filepath, data, klipparams=None, filetype=None, zaxis=None, center=None, astr_hdr=None, fakePlparams=None, more_keywords=None)[source] Save data in a GPIlike fashion. Aka, data and header are in the first extension header
 Note: In principle, the function only works inside klip_dataset(). In order to use it outside of klip_dataset,
 you need to define the following attribute:
 dataset.output_centers = dataset.centers
 Inputs:
filepath: path to file to output data: 2D or 3D data to save klipparams: a string of klip parameters filetype: filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”) zaxis: a list of values for the zaxis of the datacub (for KL mode cubes currently) astr_hdr: wcs astrometry header (None for NIRC2) center: center of the image to be saved in the header as the keywords PSFCENTX and PSFCENTY in pixels.
The first pixel has coordinates (0,0)fakePlparams: fake planet params more_keywords (dictionary) : a dictionary {key: value, key:value} of header keywords and values which will
written into the primary header

wcs
Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”

pyklip.instruments.NIRC2.
calc_starflux
(cube, center)[source]¶ Fits a 2D Gaussian to an image to calculate the peak pixel value of the central star. The code assumes an unobscurated PSF.
Parameters:  cube – 2D image array. Shape is (256,256)
 center – star center in image in (x,y)
Returns: Best fit amplitude of the 2D Gaussian.
Return type: Amplitude

pyklip.instruments.NIRC2.
get_pa
(hdulist, obsdate=None, rotmode=None, mean_PA=True, write_hdr=True)[source]¶ Given a FITS dataheader unit list (HDUList), returns the NIRC2 PA in [radians]. PA is angle of detector relative to sky; ROTMODE is rotator tracking mode; PARANG is parallactic angle astrometric; INSTANGL is instrument angle; ROTPOSN is rotator physical position. Additional PA offset of 0.252 or 0.262 deg is applied for NIRC2 narrow cam depending on observation date. NOTE that the PA sign is flipped at the very end before output to conform to pyKLIP’s rotation convention.
 Inputs:
hdulist: a FITS HDUList (NOT a single HDU). obsdate: date of observation; will try to get from prihdr if not provided. rotmode: ‘vertical angle’ for ADI mode with PA rotating on detector, or
‘position angle’ for mode with PA orientation fixed on detector. mean_pa: if True (default), return the mean PA during the exposure.
 If False, return the PA at the start of the exposure only. Only applies to vertical angle mode.
write_hdr: if True (default), writes keys to file header and saves them.

pyklip.instruments.NIRC2.
get_star
(hdulist, ctr, obsdate, hp_size=0, im_smooth=0, sp_width=0, spike_angles=None, r_mask='all', rad=100, rad_out=inf, radon_wdw=400, smooth=1, PAadj=0.0, write_hdr=True, pool=None, silent=True)[source]¶ Runs Radon transform starfinding algorithm on image and (by default) saves the results in the original FITS header.
 Inputs:
hdulist: a FITS HDUList (NOT a single HDU). ctr: (y,x) coordinate pair estimate for star position for image [pix]. obsdate: date of observation; will try to get from prihdr if not provided. hp_size: size of highpass filter box (via Fourier transform) in [pix]. im_smooth: sigma of smoothing Gaussian function in [pix]. sp_width: width of diffraction spike mask in [pix]; default is 0 (no masking). spike_angles: list of discrete angles from the assumed star positions along
which the radon transform will sum intensity to search for the true star position (it picks the maximum sum). These should match the angles of the strongest diffraction spikes [deg].r_mask: ‘all’ to mask out circle around ctr coords; anything else to do no radial masking. rad: r_mask==’all’ will mask out all r <= rad [pix]. rad_out: r_mask==’all’ will mask out all r > rad_out [pix]. radon_window: half width of the radon sampling region; size_window = image.shape[0]/2 is suggested.
m & M: The sampling region will be (M*size_window, m*size_window)U(m*size_window, M*size_window). smooth: smooth the radon cost function; for one pixel, replace it by the
 average of its +/ smooth neighbours; default = 2.
PAadj: optional angle by which to rotate diffraction spike angles in [radians]. write_hdr: (default) True will write the Radon transform star center to the original
FITS header in PSFCENTX & PSFCENTY keywords.pool: multiprocessing pool for highpass filtering and other parallel uses. silent: (default) True to suppress additional output to stdout.
 Outputs:
 Returns [X,Y] list of Radon transform star center. Default is to also write the star coordinates to PSFCENTX & PSFCENTY in original FITS header.

pyklip.instruments.NIRC2.
make_spikemask
(data, hdr, ctr, spike_angles, yy, xx, width=31)[source]¶ Construct diffraction spike mask from FITS header information.
data: 2D ndarray image to be masked (just to get size of array). hdr: FITS header for image constructing mask for. ctr: (y,x) coordinates for center of diffraction spike pattern
(usually the estimated star location).spike_angles: position angles for diffraction spikes in image [radians]. yy: mgrid or indices 2D array of pixel ycoordinates. xx: mgrid or indices 2D array of pixel xcoordinates. width: int or float width of spike mask in [pixels]; 0 for no mask.
pyklip.instruments.P1640 module¶

class
pyklip.instruments.P1640.
P1640Data
(filepaths=None, skipslices=None, corefilepaths=None, spot_directory=None, highpass=True, numthreads=1, PSF_cube=None, verbose=False)[source]¶ Bases:
pyklip.instruments.Instrument.Data
Note: update object string when output is decided A sequence of P1640 Data. Each P1640Data object has the following fields and functions
Parameters:  filepaths – list of filepaths to occulted files
 skipslices – a list of datacube slices to skip (supply index numbers e.g. [0,1,2,3])
 corefilepaths – a list of filepaths to core (i.e. unocculted) files, for contrast calc
 spot_directory – (None) path to the directory where the spot positions are stored. Defaults to P1640.ini val
 verbose – [False] if True, print more stuff

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

wcs
¶ Array of N wcs astormetry headers for each image.

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

spot_flux
¶ Array of N of average satellite spot flux for each frame

contrast_scaling
¶ Flux calibration factors (multiply by image to “calibrate” flux)

flux_units
¶ units of output data [DN, contrast]

prihdrs
¶ not used for P1640, set to None

exthdrs
¶ Array of N P1640 headers (these are written by the P1640 cube extraction pipeline)

savedata
()[source]¶ save a specified data in the P1640 datacube format (in the 1st extension header)

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

band
= 'H'¶

bands
= ['H']¶

calibrate_output
(units='contrast')[source] Calibrates the flux of the output of PSF subtracted data.
Assumes self.output exists and has shape (b,N,y,x) for N is the number of images and b is number of KL modes used.
Parameters: units – currently only support “contrast” w.r.t central star Returns: stores calibrated data in self.output

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

centralwave
= {'H': 1.64}¶

config
= <configparser.ConfigParser object>¶

configfile
¶

corefilenames
¶

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

flux_zeropt
= 1.9238e07¶

fpm_diam
= {'H': 19.379844961240313}¶

generate_psfs
(boxrad=7, gauss=True, mirror_adjust=1.0)[source]¶ Generates PSF for each frame of input data. Only works on spectral mode data. :param boxrad: the halflength of the size of the extracted PSF (in pixels) :param # spotyx: Ncube x Nchan x 4 x 2 array of spot (row, col) positions :param gauss [False]: if True, use a gaussian PSF :param mirror_adjust [1.0]: multiply mirror diameter by this factor to adjust for Lyot stop
Returns: saves PSFs to self.psfs as an array of size(N,psfy,psfx) where psfy=psfx=2*boxrad + 1

get_radial_psf
(save=None)[source]¶ Return a pure radial PSF by averaging the original psf. The new PSF is invariant by rotation. A call to generate_psf_cube() is required prior to calling this function. The center pixel index is always (nx/2,nx/2) assuming integer division.
Parameters: save – Optionally automatically save the radial psf cube as a fits file with filename: save+”original_radial_PSF_cube.fits” Returns: a (37,nx,nx) cube with the radial psf. Return type: rad_psf_cube

get_scaling_and_centering
(filepaths, spot_directory=None, skipslices=None)[source]¶ Method to get star position and wavelength scaling factors :param filepaths: a list of filepaths to datacubes :param spot_path: (None) directory where the spot files are stored. Defaults to P1640.ini value :param skipslices: a list of wavelength slices to skip for each datacube (supply index numbers e.g. [0,1,2,3])
Returns: Nothing. Sets centers and scale_factors fields of the P1640Data object

ifs_rotation
= 18.0¶

input
Input Data. Shape of (N, y, x)

lenslet_scale
= 0.01935¶

nchannels_all
= 32¶

nchannels_used
= None¶

observatory_latitude
= 33.355833¶

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

package_directory
= '/home/docs/checkouts/readthedocs.org/user_builds/pyklip/envs/latest/lib/python3.7/sitepackages/pyklip2.4.1py3.7.egg/pyklip/instruments'¶

readdata
(filepaths, skipslices=None, corefilepaths=None, highpass=True, numthreads=1, PSF_cube=None, verbose=False)[source] Method to open and read a list of P1640 data. Handles everything that can be done by reading directly from the P1640 header or cubes, no calculations. Scaling and Centering handled elsewhere
Parameters:  filepaths – a list of filepaths
 skipslices – a list of wavelenegth slices to skip for each datacube (supply index numbers e.g. [0,1,2,3])
 highpass – if True, run a Gaussian high pass filter (default size is sigma=imgsize/10) can also be a number specifying FWHM of box in pixel units
 numthreads – Number of threads to be used. Default 1 sequential sat spot flux calc. If None, numthreads = mp.cpu_count().
 PSF_cube – 3D array (nl,ny,nx) with the PSF cube to be used in the flux calculation.
Returns: Technically none. It saves things to fields of the P1640Data object. See object doc string

savedata
(filepath, data, klipparams=None, filetype='PSF Subtracted Spectral Cube', zaxis=None, center=None, astr_hdr=None, fakePlparams=None, more_keywords=None, user_prihdr=None, user_exthdr=None, extra_exthdr_keywords=None, extra_prihdr_keywords=None)[source] Save data in a fits file in a GPIlike fashion. Aka, data and header are in the extension HDU. For now, the Primary HDU contains the KLIP parameters, scaling, and centering. This may later change storing the data in the Primary HDU, all the headers from the input files in the extension.
 Note: In principle, the function only works inside klip_dataset(). In order to use it outside of klip_dataset,
 you need to define the follwing attributes:
 dataset.output_wcs = np.array([w.deepcopy() if w is not None else None for w in dataset.wcs]) dataset.output_centers = dataset.centers
 Inputs:
filepath: path to file to output data: 2D or 3D data to save klipparams: a string of klip parameters filetype: filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”) zaxis: a list of values for the zaxis of the datacube (for KL mode cubes currently) astr_hdr: wcs astrometry header center: center of the image to be saved in the header as the keywords PSFCENTX and PSFCENTY in pixels.
The first pixel has coordinates (0,0)fakePlparams: fake planet params

scale_factors
¶

spot_directory
¶

spot_locations
¶

spot_ratio
= {}¶

wcs
Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”

pyklip.instruments.P1640.
generate_psf
(frame, locations, boxrad=5, medianboxsize=30)[source]¶ Generates a P1640 PSF for the frame based on the satellite spots
Parameters:  frame – 2d frame of data
 location – array of (N,2) containing [x,y] coordinates of all N satellite spots
 boxrad – half length of box to use to pull out PSF
 medianboxsize – size in pixels of box for median filter
Returns: 2d frame of size (2*boxrad+1, 2*boxrad+1) with average PSF of satellite spots
Return type: genpsf

pyklip.instruments.P1640.
get_p1640_spot_filepaths
(config, data_filepath)[source]¶ Look to see if the spot positions have already been written to file. If the files exist, return their paths; otherwise return None. Input:
config: a ConfigParser object with the file path information data_filepath: the name of the P1640 data file whose spots you want to locateReturns: a path to the files (None if they don’t exist) Return type: filepath

pyklip.instruments.P1640.
rescale_wvs
(exthdrs, wvs, refwv=18, skipslices=None)[source]¶ Hack to try to fix wavelength scaling issue. This will calculate the scaling between channels, and adjust the wavelength solution such that the scaling comes out linear in scaling vs wavelength. Finicky  requires that all images in the dataset have the same number of wavelength channels :param exthdrs: a list of extension headers, from a pyklip.instrument dataset :param wvs: a list of wvs (can repeat. This function will only look at the first cube’s wavelenghts) :param refwv: integer index of the channel to normalize the scaling :type refwv: optional :param skipslices: list of skipped wavelength slices (needs to be consistent with the ones skipped by wv)
Returns: Nlambda*Nexthdrs array of wavelengths that produce a linear plot of wavelength vs scaling Return type: scaled_wvs

pyklip.instruments.P1640.
write_p1640_spots_to_file
(config, data_filepath, spot_positions, spot_directory=None, overwrite=True)[source]¶ EDIT: NOW THIS IS JUST A WRAPPER FOR THE P1640spots.write_spots_to_file() METHOD
Write the spot (row, col) positions to 4 files (1 per spot) in the directory specified in the config file. Input:
config: a ConfigParser object with the file path information data_filepath: source file, spot files will have same prefix spot_positions: a Nspot x Nchan x 2 array of (row, col) spot positions spot_directory: (None) Directory to save the spot files overwrite: True > overwrite (default), False > don’t overwrite existing files Output:
 None
pyklip.instruments.SPHERE module¶

class
pyklip.instruments.SPHERE.
Ifs
(data_cube, psf_cube, info_fits, wavelength_info, keepslices=None, psf_cube_size=21, nan_mask_boxsize=9, IWA=0.15, object_name=None, disable_minimum_filter=False, zeros2nans=False, subtract_psf_background=False)[source]¶ Bases:
pyklip.instruments.Instrument.Data
A sequence of SPHERE IFS Data.
Parameters:  data_cube – FITS file with a 4Dcube (Nfiles, Nwvs, Ny, Nx) with all IFS coronagraphic data Also read spectral cubes and 2D images that have been saved using savedata().
 psf_cube – FITS file with a 3D (Nwvs, Ny, Nx) PSF cube
 info_fits – FITS file with a table in the 1st ext hdr with parallactic angle info
 wavelenegth_info – FITS file with a 1D array (Nwvs) of the wavelength sol’n of a cube
 psf_cube_size – size of the psf cube to save (length along 1 dimension)
 nan_mask_boxsize – size of box centered around any pixel <= 0 to mask as NaNs
 IWA – inner working angle of the data in arcsecs

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

ifs_rdp
¶ Reduction algorithm used to obtain the input data.

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

psfs
¶ Spectral cube of size (Nwv, psfy, psfx) where psf_cube_size defines the size of psfy, psfx.

psf_center
¶ [x, y] location of the center of the PSF for a frame in self.psfs

flipx
¶ True by default. Determines whether a relfection about the x axis is necessary to rotate image Northup East left

nfiles
¶ number of datacubes

nwvs
¶ number of wavelengths

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

calibrate_output
(img, spectral=False, units='contrast')[source]¶ Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending on if the spectral flag is set.
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
 units – currently only support “contrast” w.r.t central star
Returns: calibrated image of the same shape (this is the same object as the input!!!)
Return type: img

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

ifs_rdp

input
Input Data. Shape of (N, y, x)

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

platescale
= 0.007462¶

readdata
(filepaths)[source]¶ Reads in the data from the files in the filelist and writes them to fields

savedata
(filepath, data, klipparams=None, filetype='', zaxis=None, more_keywords=None)[source]¶ Save SPHERE Data.
 Note: In principle, the function only works inside klip_dataset(). In order to use it outside of klip_dataset,
 you need to define the follwing attributes:
 dataset.output_centers = dataset.centers
Parameters:  filepath – path to file to output
 data – 2D or 3D data to save
 klipparams – a string of klip parameters
 filetype – filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”)
 zaxis – a list of values for the zaxis of the datacub (for KL mode cubes currently)
 more_keywords (dictionary) – a dictionary {key: value, key:value} of header keywords and values which will written into the primary header

wcs
¶ Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”

class
pyklip.instruments.SPHERE.
Irdis
(data_cube, psf_cube, info_fits, wavelength_str, psf_cube_size=21, IWA=0.08, OWA=None, keepslices=None)[source]¶ Bases:
pyklip.instruments.Instrument.Data
A sequence of SPHERE IRDIS Data.
Parameters:  data_cube – FITS file with a 4Dcube (Nfiles, Nwvs, Ny, Nx) with all IFS coronagraphic data Also read spectral cubes and 2D images that have been saved using savedata().
 psf_cube – FITS file with a 3D (Nwvs, Ny, Nx) PSF cube If None, psf_cube = data_cube.replace(“cube_coro”,”cube_psf”)
 info_fits – FITS file with a table in the 1st ext hdr with parallactic angle info If None, info_fits = data_cube.replace(“cube_coro”,”info”)
 wavelength_str – string to specifiy the band (e.g. “H2H3”, “K1K2”)
 psf_cube_size – size of the psf cube to save (length along 1 dimension)
 IWA – inner working angle of the data in arcsecs
 OWA – outer working angle of the data in arcsecs

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

OWA
¶ a floating point scalar (not array). Specifies to out working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

psfs
¶ Spectral cube of size (2, psfy, psfx) where psf_cube_size defines the size of psfy, psfx.

psf_center
¶ [x, y] location of the center of the PSF for a frame in self.psfs

flipx
¶ True by default. Determines whether a relfection about the x axis is necessary to rotate image Northup East left

nfiles
¶ number of datacubes

prihdrs
¶ SPHERE headers if reduced by Sphere data center

nwvs
¶ number of wavelengths (i.e. 2 for dual band imaging)

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

OWA

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

calibrate_output
(img, spectral=False, units='contrast')[source]¶ Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending on if the spectral flag is set.
Parameters:  img – unclaibrated image. If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
 spectral – if True, this is a spectral datacube. Otherwise, it is a broadband image.
 units – currently only support “contrast” w.r.t central star
Returns: calibrated image of the same shape (this is the same object as the input!!!)
Return type: img

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

input
Input Data. Shape of (N, y, x)

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

platescale
= 0.012255¶

readdata
(filepaths)[source]¶ Reads in the data from the files in the filelist and writes them to fields

savedata
(filepath, data, klipparams=None, filetype='', zaxis=None, more_keywords=None)[source]¶ Save SPHERE Data.
 Note: In principle, the function only works inside klip_dataset(). In order to use it outside of klip_dataset,
 you need to define the follwing attribute:
 dataset.output_centers = dataset.centers
Parameters:  filepath – path to file to output
 data – 2D or 3D data to save
 klipparams – a string of klip parameters
 filetype – filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”)
 zaxis – a list of values for the zaxis of the datacub (for KL mode cubes currently)
 more_keywords (dictionary) – a dictionary {key: value, key:value} of header keywords and values which will written into the primary header

wavelengths
= {'B_H': (1.625, 1.625), 'B_J': (1.245, 1.245), 'B_Ks': (2.182, 2.182), 'B_Y': (1.043, 1.043), 'H2H3': (1.593, 1.667), 'H3H4': (1.667, 1.733), 'J2J3': (1.19, 1.273), 'K1K2': (2.11, 2.251), 'Y2Y3': (1.022, 1.076)}¶

wcs
¶ Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”
pyklip.instruments.osiris module¶

class
pyklip.instruments.osiris.
Ifs
(data_cube_list, telluric_cube, guess_center=None, recalculate_center_cadi=False, centers=None, psf_cube_size=21, coaddslices=None, nan_mask_boxsize=0, median_filter_boxsize=0, badpix2nan=False, ignore_PAs=True)[source]¶ Bases:
pyklip.instruments.Instrument.Data
A spectral cube of Osiris IFS Data.
Parameters:  data_cube – FITS file list with 3Dcubes (Nwvs, Ny, Nx) with an osiris IFS data
 telluric_cube – single telluric reference FITS file with a 3Dcube (Nwvs, Ny, Nx) with an osiris IFS data.
 psf_cube_size – size of the psf cube to save (length along 1 dimension)
 coaddslices – if not None, combine (addition) slices together to reduce the size of the spectral cube. coaddslices should be an integer corresponding to the number of slices to be combined.

input
¶ Array of shape (N,y,x) for N images of shape (y,x)

centers
¶ Array of shape (N,2) for N centers in the format [x_cent, y_cent]

filenums
¶ Array of size N for the numerical index to map data to file that was passed in

filenames
¶ Array of size N for the actual filepath of the file that corresponds to the data

PAs
¶ Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

wvs
¶ Array of N wavelengths of the images (used for SDI) [in microns]. For polarization data, defaults to “None”

IWA
¶ a floating point scalar (not array). Specifies to inner working angle in pixels

output
¶ Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

psfs
¶ Spectral cube of size (Nwv, psfy, psfx) where psf_cube_size defines the size of psfy, psfx.

psf_center
¶ [x, y] location of the center of the PSF for a frame in self.psfs

flipx
¶ True by default. Determines whether a relfection about the x axis is necessary to rotate image Northup East left

nfiles
¶ number of datacubes

nwvs
¶ number of wavelengths

IWA
a floating point scalar (not array). Specifies to inner working angle in pixels

PAs
Array of N for the parallactic angle rotation of the target (used for ADI) [in degrees]

calibrate_output
(img, spectral=False, units='contrast')[source]¶  Calibrates the flux of an output image. Can either be a broadband image or a spectral cube depending
on if the spectral flag is set.
Assumes the broadband flux calibration is just multiplication by a single scalar number whereas spectral datacubes may have a separate calibration value for each wavelength
 Args:
 img: unclaibrated image.
 If spectral is not set, this can either be a 2D or 3D broadband image where the last two dimensions are [y,x] If specetral is True, this is a 3D spectral cube with shape [wv,y,x]
spectral: if True, this is a spectral datacube. Otherwise, it is a broadband image. units: currently only support “contrast” w.r.t central star
 Return:
 img: calibrated image of the same shape (this is the same object as the input!!!)

centers
Image centers. Shape of (N, 2) where the 2nd dimension is [x,y] pixel coordinate (in that order)

filenames
Array of size N for the actual filepath of the file that corresponds to the data

filenums
Array of size N for the numerical index to map data to file that was passed in

input
Input Data. Shape of (N, y, x)

output
Array of shape (b, len(files), len(uniq_wvs), y, x) where b is the number of different KL basis cutoffs

readdata
(filepaths)[source]¶ Reads in the data from the files in the filelist and writes them to fields

savedata
(filepath, data, center=None, klipparams=None, filetype='', zaxis=None, more_keywords=None)[source]¶ Save OSIRIS Data.
 Note: In principle, the function only works inside klip_dataset(). In order to use it outside of klip_dataset,
 you need to define the following attribute:
 dataset.output_centers = dataset.centers
Parameters:  filepath – path to file to output
 data – 2D or 3D data to save
 center – center of the image to be saved in the header as the keywords PSFCENTX and PSFCENTY in pixels. The first pixel has coordinates (0,0)
 klipparams – a string of klip parameters
 filetype – filetype of the object (e.g. “KL Mode Cube”, “PSF Subtracted Spectral Cube”)
 zaxis – a list of values for the zaxis of the datacub (for KL mode cubes currently)
 more_keywords (dictionary) – a dictionary {key: value, key:value} of header keywords and values which will written into the primary header

wcs
¶ Array of N wcs astormetry headers for each image.

wvs
Array of N wavelengths (used for SDI) [in microns]. For polarization data, defaults to “None”
Module contents¶
pyklip.kpp package¶
Subpackages¶

pyklip.kpp.detection.detection.
point_source_detection
(image, center, threshold, pix2as=None, mask_radius=4, maskout_edge=None, maskout_inner_edge=None, IWA=None, OWA=None)[source]¶ Find the brightest blobs in the image/cube.
Parameters:  image – Image from which to get the SNR map
 center – center of the image (y_cen, x_cen)
 threshold – Threshold under which blob should be ignore.
 pix2as – Platescale (arcsec per pixel).
 mask_radius – Radius of the mask used for masking point sources or the surroundings of the current pixel out of the data. Default value is 4 pixels.
 maskout_edge – mask a maskout_edge (int) pixels around the outer edge of the FOV containing NaNs.
 maskout_inner_edge – mask a maskout_inner_edge (int) pixels border around the center nan disk.
 IWA – inner working angle in pixels.
 OWA – outer working angle in pixels.
Returns: Detection table..
Table containing the list of the local maxima with their info Description by column: [“index”,”value”,”PA”,”Sep (pix)”,”Sep (as)”,”x”,”y”,”row”,”col”] 1/ index of the candidate 2/ Value of the maximum 3/ Position angle in degree from North in [0,360] 4/ Separation in pixel 5/ Separation in arcsec 6/ x position in pixel 7/ y position in pixel 8/ row index 9/ column index

pyklip.kpp.metrics.crossCorr.
calculate_cc
(image, PSF, spectrum=None, nans2zero=False)[source]¶ Perform a cross correlation on the current loaded file.
Parameters:  image – image to get the cross correlation from.
 PSF – Template for the cross correlation
 spectrum – If not None, combine the 3D cube into a 2D image using a weighted mean with spectrum.
 nans2zero – If True, temporarily replace all nans values with zeros for the cross correlation
Return: cross correlated image.

pyklip.kpp.metrics.matchedfilter.
calculate_matchedfilter
(row_indices, col_indices, image, PSF, stamp_PSF_sky_mask, stamp_PSF_aper_mask, mute=True)[source]¶ Calculate the matched filter, cross correlation and flux map on a given image or datacube for the pixels targeted by row_indices and col_indices. These lists of indices can basically be given from the numpy.where function following the example:
import numpy as np row_indices,col_indices = np.where(np.finite(np.mean(cube,axis=0)))By truncating the given lists in small pieces it is then easy to parallelized.
Parameters:  row_indices – Row indices list of the pixels where to calculate the metric in cube. Indices should be given from a 2d image.
 col_indices – Column indices list of the pixels where to calculate the metric in cube. Indices should be given from a 2d image.
 image – 2D or 3D image from which one wants the metric map. PSF_cube should be norm2 normalized. PSF_cube /= np.sqrt(np.sum(PSF_cube**2))
 PSF – 2D or 3D PSF template used for calculated the metric. If nl,ny_PSF,nx_PSF = PSF_cube.shape, nl is the number of wavelength samples, ny_PSF and nx_PSF are the spatial dimensions of the PSF_cube.
 stamp_PSF_sky_mask – 2d mask of size (ny_PSF,nx_PSF) used to mask the central part of a stamp slice. It is used as a type of a high pass filter. Before calculating the metric value of a stamp cube around a given pixel the average value of the surroundings of each slice of that stamp cube will be removed. The pixel used for calculating the average are the one equal to one in the mask.
 stamp_PSF_aper_mask – 3d mask for the aperture.
 mute – If True prevent printed log outputs.
Return: Vector of length row_indices.size with the value of the metric for the corresponding pixels.

pyklip.kpp.metrics.matchedfilter.
calculate_matchedfilter_star
(params)[source]¶ Convert f([1,2]) to f(1,2) call. It allows one to call calculate_shape3D_metric() with a tuple of parameters.

pyklip.kpp.metrics.matchedfilter.
run_matchedfilter
(image, PSF, N_threads=None, maskedge=True)[source]¶ Perform a matched filter on the current loaded file.
Parameters:  image – image for which to get the matched filter.
 PSF – Template for the matched filter. It should include any kind of spectrum you which to use of the data is 3d.
 maskedge – If True (default), mask the edges of the image to prevent partial projection of the PSF. If False, does not mask the edges.
Return: Processed images (matched filter,cross correlation,estimated flux).

pyklip.kpp.stat.statPerPix_utils.
get_image_stat_map_perPixMasking
(image, image_without_planet=None, mask_radius=7, IOWA=None, N=None, centroid=None, N_threads=None, Dr=2, Dth=None, type='SNR')[source]¶ Calculate the SNR, the standard deviation or the probability (tail distribution) of a given image on a per pixel basis, which means that for each pixel the standard deviation is calculated after masking its surroundings.
Parameters:  image – The image for which one wants the statistic.
 image_without_planet – Same as image but where real signal has been masked out. The
 image_without_planet – Same as image but where any real signal has been masked out. The code will use “image_without_planet” to calculate the standard deviation or the PDF. This can be a negatively derotated image in the context of ADI.
 mask_radius – (default=7pix) Radius of the mask used around the current pixel.
 IOWA – (IWA,OWA) inner working angle, outer working angle. It defines boundary to the zones in which the statistic is calculated. If None, kpp.utils.GPIimage.get_IOWA() is used.
 N – Defines the width of the ring by the number of pixels it has to include. The width of the annuli will therefore vary with sepration.
 centroid – (x_cen,y_cen) Define the center of the image. Default is x_cen = (nx1)//2 ; y_cen = (ny1)//2
 N_threads – Number of threads to be used. If None run sequentially.
 Dr – (default=2pix) Width of the annulus (in pixel).
 Dth – Define the angular size of a sector in degree (will apply for either Dr or N)
 type – Indicate the type of statistic to be calculated. If “SNR” (default) simple stddev calculation and returns SNR. If “stddev” returns the pure standard deviation map. If “proba” triggers proba calculation with pdf fitting.
Returns: The statistic map for image.

pyklip.kpp.stat.statPerPix_utils.
get_image_stat_map_perPixMasking_threadTask
(row_indices, col_indices, image, image_without_planet, x_grid, y_grid, N, mask_radius, Dr=2, Dth=None, type='SNR')[source]¶ Helper function for get_image_stat_map_perPixMasking(). Calculate the SNR on the subset of pixels defined by row_indices,col_indices.
Calculate the SNR, the standard deviation or the probability (tail distribution) of a subset of an image on a per pixel basis, which means that for each pixel the standard deviation is calculated after masking its surroundings.
This function is used to ease parallelization.
Parameters:  row_indices – The row indices of images for which we want the statistic.
 col_indices – The column indices of images for which we want the statistic.
 image – The image or cubes for which one wants the statistic.
 image_without_planet – Same as image but where real signal has been masked out. The code will actually use map to calculate the standard deviation or the density function.
 mask_radius – Radius of the mask used around the current pixel when use_mask_per_pixel = True.
 IOWA – (IWA,OWA) inner working angle, outer working angle. It defines boundary to the zones in which the statistic is calculated.
 N – Defines the width of the ring by the number of pixels it has to include. The width of the annuli will therefore vary with sepration.
 centroid – Define the cente rof the image. Default is x_cen = (nx1)//2 ; y_cen = (ny1)//2
 mute – Won’t print any logs.
 N_threads – Number of threads to be used. If None run sequentially.
 Dr – (default=2pix) Defines the width of the ring (in pixel). N is then ignored.
 Dth – Define the angular size of a sector in degree (will apply for either Dr or N)
 type – Indicate the type of statistic to be calculated. If “SNR” (default) simple stddev calculation and returns SNR. If “stddev” returns the pure standard deviation map. If “proba” triggers proba calculation with pdf fitting.
Return: The statistic map for image.

pyklip.kpp.stat.stat_utils.
get_cdf_model
(data, interupt_plot=False, pure_gauss=False)[source]¶ Calculate a model CDF for some data.
/!This function is for some reason still a work in progress. JB could never decide what the best option was. But it should work even if the code is a mess.
Parameters:  data – arrays of samples from a random variable
 interupt_plot – Plot the histogram and model fit. It
 pure_gauss – Assume gaussian statistic. Do not fit exponential tails.
 Return: (cdf_model,new_sampling,im_histo, center_bins) with:
 cdf_model: The cdf model = np.cumsum(pdf_model) pdf_model: The pdf model sampling: sampling of pdf/cdf_model im_histo: histogram from original data center_bins: bin centers for im_histo

pyklip.kpp.stat.stat_utils.
get_cube_stddev
(cube, IOWA, N=2000, centroid=None, r_step=None, Dr=None)[source]¶

pyklip.kpp.stat.stat_utils.
get_image_PDF
(image, IOWA=None, N=2000, centroid=None, r_step=None, Dr=None, image_wide=None)[source]¶ Calculate the PDF of a given image using annuli.
Parameters:  image – The image for which one wants the statistic.
 IOWA – (IWA,OWA) inner working angle, outer working angle. It defines boundary to the zones in which the statistic is calculated. If None, kpp.utils.GPIimage.get_IOWA() is used.
 N – Defines the width of the ring by the number of pixels it has to include. The width of the annuli will therefore vary with sepration. Default is N=3000.
 centroid – (x_cen,y_cen) Define the center of the image. Default is x_cen = (nx1)//2 ; y_cen = (ny1)//2
 r_step – Distance between two consecutive annuli mean separation (in pixel).
 Dr – Width of the annulus (in pixel).
 image_wide – Don’t divide the image in annuli or sectors when computing the statistic. Use the entire image directly. Not available if “pixel based: is defined,
Returns: List of PDF values for each annulus. The sampling of each PDF can be found in sampling_list. cdf_list: CDF values for each annulus. The sampling of each CDF can be found in sampling_list. sampling_list: Sampling for the PDF and the CDF. annulus_radii_list: List of ((r_min+r_max)/2.,r_min,r_max) with r_min,r_max the boundaries of an annulus.
Return type: pdf_list

pyklip.kpp.stat.stat_utils.
get_image_stat
(image, type, IOWA=None, N=None, centroid=None, r_step=2, Dr=2, image_wide=None)[source]¶ Calculate the standard deviation or the mean of a given image using annuli.
Parameters:  image – The image or cubes for which one wants the statistic.
 type – “stddev” or “mean” or “sum”
 IOWA – (IWA,OWA) inner working angle, outer working angle. It defines boundary to the zones in which the statistic is calculated. If None, kpp.utils.GPIimage.get_IOWA() is used.
 N – Defines the width of the ring by the number of pixels it has to include. The width of the annuli will therefore vary with sepration. Default is N=3000.
 centroid – (x_cen,y_cen) Define the center of the image. Default is x_cen = (nx1)//2 ; y_cen = (ny1)//2
 r_step – Distance between two consecutive annuli mean separation. Not available if “pixel based” is defined,
 Dr – If not None defines the width of the ring as Dr. N is then ignored if Dth is defined.
 image_wide – Don’t divide the image in annuli or sectors when computing the statistic. Use the entire image directly. Not available if “pixel based: is defined,
Returns: standard deviation values at the center of each annulus_radii_list: List of ((r_min+r_max)/2.,r_min,r_max) with r_min,r_max the boundaries of an annulus.
Return type: stddev_list

pyklip.kpp.stat.stat_utils.
get_image_stat_map
(image, image_without_planet=None, IOWA=None, N=None, centroid=None, r_step=2, Dr=2, type='SNR', image_wide=None)[source]¶ Calculate the SNR, the standard deviation or the probability (tail distribution) of a given image using concentric annuli.
Parameters:  image – The image for which one wants the statistic.
 image_without_planet – Same as image but where any real signal has been masked out. The code will use “image_without_planet” to calculate the standard deviation or the PDF. This can be a negatively derotated image in the context of ADI.
 IOWA – (IWA,OWA) inner working angle, outer working angle. It defines boundary to the zones in which the statistic is calculated. If None, kpp.utils.GPIimage.get_IOWA() is used.
 N – Defines the width of the ring by fixing the number of pixels of the annulus. The width of the annuli will therefore vary with sepration.
 centroid – (x_cen,y_cen) Define the center of the image. Default is x_cen = (nx1)//2 ; y_cen = (ny1)//2
 r_step – (default=2pix) Distance between two consecutive annuli mean separation (in pixel).
 Dr – (default=2pix) Width of the annulus (in pixel).
 type – Indicate the type of statistic to be calculated. If “SNR” (default) simple stddev calculation and returns SNR. If “stddev” returns the pure standard deviation map. If “mean” returns a map from the radial mean. If “sum” returns a map from the radial sum. If “proba” triggers proba calculation with pdf fitting.
 image_wide – Don’t divide the image in annuli or sectors when computing the statistic. Use the entire image directly.
Returns: The statistic map for image.

pyklip.kpp.stat.stat_utils.
get_image_stddev
(image, IOWA=None, N=None, centroid=None, r_step=2, Dr=2, image_wide=None)[source]¶ Calculate the standard deviation of a given image using annuli.
Parameters:  image – The image for which one wants the statistic.
 IOWA – (IWA,OWA) inner working angle, outer working angle. It defines boundary to the zones in which the statistic is calculated. If None, kpp.utils.GPIimage.get_IOWA() is used.
 N – Defines the width of the ring by the number of pixels it has to include. The width of the annuli will therefore vary with sepration. Default is N=3000.
 centroid – (x_cen,y_cen) Define the center of the image. Default is x_cen = (nx1)//2 ; y_cen = (ny1)//2
 r_step – (default=2pix) Distance between two consecutive annuli mean separation (in pixel).
 Dr – (default=2pix) Width of the annulus (in pixel).
 image_wide – Don’t divide the image in annuli or sectors when computing the statistic. Use the entire image directly. Not available if “pixel based: is defined,
Returns: standard deviation values at the center of each annulus_radii_list: List of ((r_min+r_max)/2.,r_min,r_max) with r_min,r_max the boundaries of an annulus.
Return type: stddev_list

pyklip.kpp.stat.stat_utils.
get_pdf_model
(data, interupt_plot=False, pure_gauss=False)[source]¶ Calculate a model PDF for some data.
/!This function is for some reason still a work in progress. JB could never decide what the best option was. But it should work even if the code is a mess.
Parameters:  data – arrays of samples from a random variable
 interupt_plot – Plot the histogram and model fit. It
 pure_gauss – Assume gaussian statistic. Do not fit exponential tails.
 Return: (pdf_model,new_sampling,im_histo, center_bins) with:
 pdf_model: The pdf model new_sampling: sampling of pdf_model im_histo: histogram from original data center_bins: bin centers for im_histo

class
pyklip.kpp.utils.multiproc.
NoDaemonPool
(processes=None, initializer=None, initargs=(), maxtasksperchild=None, context=None)[source]¶ Bases:
multiprocessing.pool.Pool

Process
¶ alias of
NoDaemonProcess


pyklip.kpp.utils.oi.
get_pos_known_objects
(fakeinfohdr, object_name, pix2as, center=None, MJDOBS=None, OI_list_folder=None, xy=False, pa_sep=False, ignore_fakes=False, fakes_only=False, include_speckles=False, IWA=None, OWA=None)[source]¶ Return the position of real and/or simulated planets in an image based on its headers.
Parameters:  fakeinfohdr – fits file header containing the injected planets related keywords.
 object_name – Name of the star being observed.
 pix2as – platescale.
 center – Center of the image (not needed if pa_sep = True).
 MJDOBS – Julian date of the observation. (needed when OI_list_folder is not None)
 OI_list_folder – List of Object of Interest (OI) that should be masked from any standard deviation calculation. See the online documentation for instructions on how to define it.
 xy – Boolean. Returns the planets coordinate with x,y coordinates in pixels
 pa_sep – Boolean. Returns the planets coordinates as position angle (in arcsec), separation (in pix)
 ignore_fakes – Don’t return fake planets.
 fakes_only – Returns only fake planets.
 include_speckles – Include speckles from the list of object of interest (OI_list_folder)
 IWA – Inner working angle (in pixels).
 OWA – Outer working angle (in pixels).
 Return: (sep_vec,pa_vec) or (x_vec,y_vec) or (row_vec,col_vec) (default)
 Objects coordinates (real/fakes/others). For e.g., sep_vec,pa_vec are vectors and (sep_vec[0],pa_vec[0]) is the coordinate of the first object and so on…

pyklip.kpp.utils.oi.
make_GOI_list
(outputDir, GOI_list_csv, GPI_TID_csv)[source]¶ Generate the GOI files from the GOI table and the TID table (queried from the database).
outputDir: Output directory in which to save the GOI files. GOI_list_csv: Table with the list of GOIs (including separation, PA…). delimiter=’,’ GPI_TID_csv: Table giving the TID code for a given object name. delimiter=’,’ :return: One .csv file per target for which at list one GOI exists. delimiter=’;’
The filename follows: [object]_GOI.csv. For e.g. c_Eri_GOI.csv.

pyklip.kpp.utils.oi.
mask_known_objects
(cube, fakeinfohdr, object_name, pix2as, center, MJDOBS=None, OI_list_folder=None, ignore_fakes=False, fakes_only=False, include_speckles=False, IWA=None, OWA=None, mask_radius=7)[source]¶ Mask point sources in cube with an NaN aperture.
Parameters:  cube – Image or cube to be masked.
 fakeinfohdr – fits file header containing the injected planets related keywords.
 object_name – Name of the star being observed.
 pix2as – platescale.
 center – Center of the image.
 MJDOBS – Julian date of the observation. (needed when OI_list_folder is not None)
 OI_list_folder – List of Object of Interest (OI) that should be masked from any standard deviation calculation. See the online documentation for instructions on how to define it.
 xy – Boolean. Returns the planets coordinate with x,y coordinates in pixels
 pa_sep – Boolean. Returns the planets coordinates as position angle (in arcsec), separation (in pix)
 ignore_fakes – Don’t return fake planets.
 fakes_only – Returns only fake planets.
 include_speckles – Include speckles from the list of object of interest (OI_list_folder)
 IWA – Inner working angle (in pixels).
 OWA – Outer working angle (in pixels).
 mask_radius – Radius of the mask in pixels. (default = 7 pixels)
Return: Masked cube.
Module contents¶
Submodules¶
pyklip.covars module¶

pyklip.covars.
delta
(x, y, sigmas, *args)[source]¶ Generates a diagonal covariance matrix
C_ij = sigma_i sigma_j delta_{ij}
Parameters:  x (np.array) – 1D array of x coordinates
 y (np.array) – 1D array of y coordinates
 sigmas (np.array) – 1D array of errors on each pixel

pyklip.covars.
matern32
(x, y, sigmas, corr_len)[source]¶ Generates a Matern (nu=3/2) covariance matrix that assumes x/y has the same correlation length
C_ij = sigma_i sigma_j (1 + sqrt(3) r_ij / l) exp(sqrt(3) r_ij / l)
Parameters:  x (np.array) – 1D array of x coordinates
 y (np.array) – 1D array of y coordinates
 sigmas (np.array) – 1D array of errors on each pixel
 corr_len (float) – correlation length of the Matern function
Returns: 2D covariance matrix parameterized by the Matern function
Return type: cov (np.array)

pyklip.covars.
sq_exp
(x, y, sigmas, corr_len)[source]¶ Generates square exponential covariance matrix that assumes x/y has the same correlation length
C_ij = sigma_i sigma_j exp(r_ij^2/[2 l^2])
Parameters:  x (np.array) – 1D array of x coordinates
 y (np.array) – 1D array of y coordinates
 sigmas (np.array) – 1D array of errors on each pixel
 corr_len (float) – correlation length (i.e. standard deviation of Gaussian)
 mode (string) – either “numpy”, “cython”, or None, specifying the implementation of the kernel.
Returns: 2D covariance matrix parameterized by the Matern function
Return type: cov (np.array)
pyklip.empca module¶

pyklip.empca.
np_calc_chisq
(data, b, w, coef)[source]¶ Calculate chi squared
Parameters:  im – nim x npix, singleprecision numpy.ndarray. Data to be fit by the basis images
 b – nvec x npts, double precision numpy.ndarray. The nvec basis images.
 w – nim x npts, singleprecision numpy.ndarray. Weights (inverse variances) of the data.
 coef – nvec x npts, double precision numpy.ndarray. The coefficients of the basis image fits.
Returns: chisq, the total chi squared summed over all points and all images

pyklip.empca.
set_pixel_weights
(imflat, rflat, ivar=None, mode='standard', inner_sup=17, outer_sup=66)[source]¶ Parameters:  imflat – array of flattend images, shape (N, number of section indices)
 rflat – radial component of the polar coordinates flattened to 1D, length = number of section indices
 mode –
 ‘standard’: assume poission statistics to calculate variance as sqrt(photon count)
 use inverse sqrt(variance) as pixel weights and multiply by a radial weighting
 inner_sup – radius within which to supress weights
 outer_sup – radius beyond which to supress weights
Returns: pixel weights for empca

pyklip.empca.
weighted_empca
(data, weights=None, niter=25, nvec=5, randseed=1, maxcpus=1, silent=True)[source]¶ Perform iterative lowrank matrix approximation of data using weights.
Generated model vectors are not orthonormal and are not rotated/ranked by ability to model the data, but as a set they are good at describing the data.
Parameters:  data – images to model
 weights – weights for every pixel
 niter – maximum number of iterations to perform
 nvec – number of vectors to solve (rank of the approximation)
 randseed – rand num generator seed; if None, don’t reinitialize
 maxcpus – maximum cpus to use for parallel programming
 silent – bool, whether to show chi_squared for each iteration
Returns: returns the best lowrank approximation to the data in a weighted leastsquares sense (dot product of coefficients and basis vectors).
pyklip.fakes module¶

pyklip.fakes.
LSQ_gauss2d
(planet_image, x_grid, y_grid, a, x_cen, y_cen, sig)[source]¶ Calculate the squared norm of the residuals of the model with the data. Helper function for least square fit. The model is a 2d symmetric gaussian.
Parameters:  planet_image – stamp image (y,x) of the satellite spot.
 x_grid – x samples grid as given by meshgrid.
 y_grid – y samples grid as given by meshgrid.
 a – amplitude of the 2d gaussian
 x_cen – x center of the gaussian
 y_cen – y center of the gaussian
 sig – standard deviation of the gaussian
Returns: Squared norm of the residuals

pyklip.fakes.
PSFcubefit
(frame, xguess, yguess, searchrad=10, psfs_func_list=None, wave_index=None, residuals=False, rmbackground=True, add_background2residual=False)[source]¶ Estimate satellite spot amplitude (peak value) by fitting a symmetric 2d gaussian. Fit parameters: x,y position, amplitude, standard deviation (same in x and y direction)
Parameters:  frame – the data  Array of size (y,x)
 xguess – x location to fit the 2d guassian to.
 yguess – y location to fit the 2d guassian to.
 searchrad – 1/2 the length of the box used for the fit
 psfs_func_list – List of spline fit function for the PSF_cube.
 wave_index – Index of the current wavelength. In [0,36] for GPI. Only used when psfs_func_list is not None.
 residuals – If True (Default = False) then calculate the residuals of the sat spot fit (gaussian or PSF cube).
 rmbackground – If true (default), remove any background slope to the data stamp.
 add_background2residual – If True (default is false) and if rmbackground was true, it adds the background that was removed to the returned residuals.
Returns:  scalar, Estimation of the peak flux of the satellite spot.
ie Amplitude of the fitted gaussian.
Return type: returned_flux

pyklip.fakes.
airyfit2d
(frame, xguess, yguess, searchrad=5, guessfwhm=3, guesspeak=1)[source]¶ Fits a 2d airy function to the data at point (xguess, yguess)
Parameters:  frame – the data  Array of size (y,x)
 xguess,yguess – location to fit the 2d guassian to (should be pretty accurate)
 searchrad – 1/2 the length of the box used for the fit
 guessfwhm – approximate fwhm to fit to
Returns: the peakflux of the Airy function fwhm: diameter between the first minima along one axis xfit: x position yfit: y position
Return type: peakflux

pyklip.fakes.
convert_pa_to_image_polar
(pa, astr_hdr)[source]¶ Given a position angle (angle to North through East), calculate what polar angle theta (angle from +X CCW towards +Y) it corresponds to
Parameters:  pa – position angle in degrees
 astr_hdr – wcs astrometry header (astropy.wcs)
Returns: polar angle in degrees
Return type: theta

pyklip.fakes.
convert_polar_to_image_pa
(theta, astr_hdr)[source]¶ Reversed engineer from covert_pa_to_image_polar by JB. Actually JB doesn’t quite understand how it works…
Parameters:  theta – parallactic angle in degrees
 astr_hdr – wcs astrometry header (astropy.wcs)
Returns: polar angle in degrees
Return type: theta

pyklip.fakes.
gauss2d
(x0, y0, peak, sigma)[source]¶ 2d symmetric guassian function for guassfit2d
Parameters:  x0,y0 – center of gaussian
 peak – peak amplitude of guassian
 sigma – stddev in both x and y directions

pyklip.fakes.
gaussfit2d
(frame, xguess, yguess, searchrad=5, guessfwhm=3, guesspeak=1, refinefit=True)[source]¶ Fits a 2d gaussian to the data at point (xguess, yguess)
Parameters:  frame – the data  Array of size (y,x)
 xguess,yguess – location to fit the 2d guassian to (should be pretty accurate)
 searchrad – 1/2 the length of the box used for the fit
 guessfwhm – approximate fwhm to fit to
 guesspeak – approximate flux
 refinefit – whether to refine the fit of the position of the guess
Returns: the peakflux of the gaussian fwhm: fwhm of the PFS in pixels xfit: x position (only chagned if refinefit is True) yfit: y position (only chagned if refinefit is True)
Return type: peakflux

pyklip.fakes.
gaussfit2dLSQ
(frame, xguess, yguess, searchrad=5, fit_centroid=False, residuals=False)[source]¶ Estimate satellite spot amplitude (peak value) by fitting a symmetric 2d gaussian. Fit parameters: x,y position, amplitude, standard deviation (same in x and y direction)
Parameters:  frame – the data  Array of size (y,x)
 xguess – x location to fit the 2d guassian to.
 yguess – y location to fit the 2d guassian to.
 searchrad – 1/2 the length of the box used for the fit
 fit_centroid – If False (default), disable the centroid fit and only fit the amplitude and the standard deviation
 residuals – If True (Default = False) then calculate the residuals of the sat spot fit (gaussian or PSF cube).
Returns:  scalar, estimation of the peak flux of the satellite spot.
ie Amplitude of the fitted gaussian.
Return type: returned_flux

pyklip.fakes.
generate_dataset_with_fakes
(dataset, fake_position_dict, fake_flux_dict, spectrum=None, PSF_cube=None, PSF_cube_wvs=None, star_type=None, mute=False, SpT_file_csv=None, real_planets_pos=None, sep_skip_real_pl=None, pa_skip_real_pl=None, dn_per_contrast=None, star_spec=None)[source]¶ Generate spectral datacubes with fake planets. It will do a copy of the cubes read in GPIData after having injected fake planets in them. This new set of cubes can then be reduced in the same manner as the campaign data.
Doesn’t work with remove slice: assumes that the dataset is made of a list of similar datacubes or images.
Parameters:  dataset – An object of type GPIData. The fakes are injected directly into dataset so you should make a copy of dataset prior to running this function. In order for the function to query simbad for the spectral type of the star, the attribute object_name needs to be defined in dataset.
 fake_position_dict –
 Dictionary defining the way the fake planets are positionned
 fake_position_dict[“mode”]=”sector”: Put a planet in each klip sector. Can actually generate several
 datasets in which the planets will be shifted in separation and position angle with respect to one another. It can be usefull for fake based contrast curve calculation. Several parameters needs to be defined.  fake_position_dict[“annuli”]: Number of annulis in the image  fake_position_dict[“subsections”]: Number of angular sections in the image  fake_position_dict[“sep_shift”]: separation shift from the center of the sectors  fake_position_dict[“pa_shift”]: position angle shift from the center of the sectors
 fake_position_dict[“mode”]=”custom”: Put planets at given (separation, position angle).
 The following parameter needs to be defined
 fake_position_dict[“pa_sep_list”]: List of tuple [(r1,pa1),(r2,pa2),…] with each tuple givingthe separation and position angle of each planet to be injected.
 fake_position_dict[“mode”]=”ROC”: Generate fake for ROC curves calculation. Use hardcoded parameters.
 fake_flux_dict:
 Dictionary defining the way in which the flux of the fake is defined.
 fake_flux_dict[“mode”]=”contrast”: Defines the contrast value of the fakes.
 fake_flux_dict[“contrast”]: Contrast of the fake planets
 fake_flux_dict[“mode”]=”SNR”: Defines the brightness of the fakes relatively to the satellite spots.
 fake_flux_dict[“SNR”]: SNR wished for the fake planets.
 fake_flux_dict[“sep_arr”]: Separation sampling of the contrast curve in pixels.
 fake_flux_dict[“contrast_arr”]: 5 sigma contrast curve (planet to star ratio).
 PSF_cube – the psf of the image. A numpy array with shape (wv, y, x)
 PSF_cube_wvs – the wavelegnths that correspond to the input psfs
 spectrum –
spectrum name (string) or array  “host_star_spec”: The spectrum from the star or the satellite spots is directly used.
It is derived from the inverse of the calibrate_output() output. ”constant”: Use a constant spectrum np.ones(nl).
 other strings: name of the spectrum file in #pykliproot#/spectra/*/ with pykliproot the
 directory in which pyklip is installed. It that case it should be a spectrum from Mark Marley or one following the same convention. Spectrum will be corrected for transmission.
 ndarray: 1D array with a user defined spectrum. Spectrum will be corrected for transmission.
 star_type – Spectral type of the current star. If None, Simbad is queried.
 mute – If True prevent printed log outputs.
 suffix – Suffix to be added at the end of the filenames.
 SpT_file_csv – Filename of the table (.csv) contaning the spectral type of the stars.
 real_planets_pos – list of position of real point sources in the dataset that should be avoided when injecting fakes. [(sep1,pa1),(sep2,pa2),…] with the separation in pixels and the position angle in degrees.
 sep_skip_real_pl – Limit in seperation of how close a fake can be injected of a known GOI.
 pa_skip_real_pl – Limit in position angle of how close a fake can be injected of a known GOI.
 dn_per_contrast – array of the same size as spectrum giving the conversion between the peak flux of a planet in data number and its contrast.
 star_spec – 1D array stellar spectrum sampling dataset.wvs. Otherwise uses a pickles spectrum in which the temperature in interpolated from the spectral type.

pyklip.fakes.
inject_disk
(frames, centers, inputfluxes, astr_hdrs, pa, fwhm=3.5)[source]¶ Injects a fake disk into a dataset
Parameters:  frames – array of (N,y,x) for N is the total number of frames
 centers – array of size (N,2) of [x,y] coordiantes of the image center
 intputfluxes –
array of size N of the peak flux of the fake disk in each frame OR array of 2D models (North up East left) to inject into the data.
(Disk is assumed to be centered at center of image)  astr_hdrs – array of size N of the WCS headers
 pa – position angles angle (in degrees) of disk plane
 fwhm – if injecting a Gaussian disk (i.e inputfluxes is an array of floats), fwhm of Gaussian
Returns: saves result in input “frames” variable

pyklip.fakes.
inject_planet
(frames, centers, inputflux, astr_hdrs, radius, pa, fwhm=3.5, thetas=None, stampsize=None, field_dependent_correction=None)[source]¶ Injects a fake planet into a dataset either using a Gaussian PSF or an input PSF
Parameters:  frames – array of (N,y,x) for N is the total number of frames
 centers – array of size (N,2) of [x,y] coordiantes of the image center
 inputflux – EITHER array of size N of the peak flux of the fake planet in each frame (will inject a Gaussian PSF) OR array of size (N,psfy,psfx) of template PSFs. The brightnesses should be scaled and the PSFs should be centered at the center of each of the template images
 astr_hdrs – array of size N of the WCS headers
 radius – separation of the planet from the star
 pa – position angle (in degrees) of planet
 fwhm – fwhm (in pixels) of gaussian
 thetas – ignore PA, supply own thetas (CCW angle from +x axis toward +y) array of size N
 stampsize – in pixels, the width of the square stamp to inject the image into. Defaults to 3*fwhm if None
 field_dependent_correction – a function of the form ``output_stamp = correction(input_stamp, input_dx, input_dy) where, input_stamp is a 2D image of shape (y_stamp, x_stamp). input_dx and input_dy have the same shape and represent the offset of each pixel from the star (in pixel units). The function returns an output_stamp of the same shape, but corrected for any field dependent throughputs or distortions.
Returns: saves result in input “frames” variable

pyklip.fakes.
retrieve_planet
(frames, centers, astr_hdrs, sep, pa, searchrad=7, guessfwhm=3.0, guesspeak=1, refinefit=True, thetas=None)[source]¶ Retrives the planet properties from a series of frames given a separation and PA
Parameters:  frames – frames of data to retrieve planet. Can be a single 2D image ([y,x]) for a series/cube ([N,y,x])
 centers – coordiantes of the image center. Can be [2]element lst or an array that matches array of frames [N,2]
 astr_hdrs – astr_hdrs, can be a single one or an array of N of them
 sep – radial distance in pixels
 PA – parallactic angle in degrees
 searchrad – 1/2 the length of the box used for the fit
 guessfwhm – approximate fwhm to fit to
 guesspeak – approximate flux
 refinefit – whether or not to refine the positioning of the planet
 thetas – ignore PA, supply own thetas (CCW angle from +x axis toward +y) single number or array of size N
Returns: (peakflux, x, y, fwhm). A single tuple if one frame passed in. Otherwise an array of tuples
Return type: measured

pyklip.fakes.
retrieve_planet_flux
(frames, centers, astr_hdrs, sep, pa, searchrad=7, guessfwhm=3.0, guesspeak=1, refinefit=False, thetas=None)[source]¶ Retrives the planet flux from a series of frames given a separation and PA
Parameters:  frames – frames of data to retrieve planet. Can be a single 2D image ([y,x]) for a series/cube ([N,y,x])
 centers – coordiantes of the image center. Can be [2]element lst or an array that matches array of frames [N,2]
 astr_hdrs – astr_hdrs, can be a single one or an array of N of them
 sep – radial distance in pixels
 PA – parallactic angle in degrees
 searchrad – 1/2 the length of the box used for the fit
 guessfwhm – approximate fwhm to fit to
 guesspeak – approximate flux
 refinefit – whether or not to refine the positioning of the planet
 thetas – ignore PA, supply own thetas (CCW angle from +x axis toward +y) single number or array of size N
Returns:  either a single peak flux or an array depending on whether a single frame or multiple frames
where passed in
Return type: peakflux
pyklip.fitpsf module¶

class
pyklip.fitpsf.
FMAstrometry
(guess_sep, guess_pa, fitboxsize, method='mcmc')[source]¶ Bases:
pyklip.fitpsf.FitPSF
Specifically for fitting astrometry of a directly imaged companion relative to its star. Extension of
pyklip.fitpsf.FitPSF
.Parameters:  guess_sep – the guessed separation (pixels)
 guess_pa – the guessed position angle (degrees)
 fitboxsize – fitting box side length (pixels)
 method (str) – either ‘mcmc’ or ‘maxl’ depending on framework you want. Defaults to ‘mcmc’.

guess_sep
¶ (initialization) guess separation for planet [pixels]
Type: float

guess_pa
¶ (initialization) guess PA for planet [degrees]
Type: float

guess_RA_offset
¶ (initialization) guess RA offset [pixels]
Type: float

guess_Dec_offset
¶ (initialization) guess Dec offset [pixels]
Type: float

raw_RA_offset
¶ (result) the raw result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

raw_Dec_offset
¶ (result) the raw result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

raw_flux
¶ (result) factor to scale the FM to match the flux of the data
Type: pyklip.fitpsf.ParamRange

covar_params
¶ (result) hyperparameters for the Gaussian process
Type: list of pyklip.fitpsf.ParamRange

raw_sep
¶ (result) the inferred raw result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

raw_PA
¶ (result) the inferred raw result from the MCMC fit for the planet’s location [degrees]
Type: pyklip.fitpsf.ParamRange

RA_offset
¶ (result) the RA offset of the planet that includes all astrometric errors [pixels or mas]
Type: pyklip.fitpsf.ParamRange

Dec_offset
¶ (result) the Dec offset of the planet that includes all astrometric errors [pixels or mas]
Type: pyklip.fitpsf.ParamRange

sep
¶ (result) the separation of the planet that includes all astrometric errors [pixels or mas]
Type: pyklip.fitpsf.ParamRange

PA
¶ (result) the PA of the planet that includes all astrometric errors [degrees]
Type: pyklip.fitpsf.ParamRange

fm_stamp
¶ (fitting) The 2D stamp of the forward model (centered at the nearest pixel to the guessed location)
Type: np.array

data_stamp
¶ (fitting) The 2D stamp of the data (centered at the nearest pixel to the guessed location)
Type: np.array

noise_map
¶ (fitting) The 2D stamp of the noise for each pixel the data computed assuming azimuthally similar noise
Type: np.array

padding
¶ amount of pixels on one side to pad the data/forward model stamp
Type: int

sampler
¶ an instance of the emcee EnsambleSampler. Only for Bayesian fit. See emcee docs for more details.
Type: emcee.EnsembleSampler

fit_astrometry
(nwalkers=100, nburn=200, nsteps=800, save_chain=True, chain_output='bkachain.pkl', numthreads=None)[source]¶ Fits the PSF of the planet in either a frequentist or Bayesian way depending on initialization.
Parameters:  nwalkers – number of walkers (mcmconly)
 nburn – numbe of samples of burnin for each walker (mcmconly)
 nsteps – number of samples each walker takes (mcmconly)
 save_chain – if True, save the output in a pickled file (mcmconly)
 chain_output – filename to output the chain to (mcmconly)
 numthreads – number of threads to use (mcmconly)
Returns:

generate_data_stamp
(data, data_center, data_wcs=None, noise_map=None, dr=4, exclusion_radius=10)[source]¶ Generate a stamp of the data_stamp ~centered on planet and also corresponding noise map
Parameters:  data – the final collapsed data_stamp (2D)
 data_center – location of star in the data_stamp.
 data_wcs – sky angles WCS object. To rotate the image properly [NOT YET IMPLMETNED] if None, data_stamp is already rotated North up East left
 noise_map – if not None, noise map for each pixel in the data_stamp (2D). if None, one will be generated assuming azimuthal noise using an annulus widthh of dr
 dr – width of annulus in pixels from which the noise map will be generated
 exclusion_radius – radius around the guess planet location which doens’t get factored into noise estimate
Returns:

generate_fm_stamp
(fm_image, fm_center, fm_wcs=None, extract=True, padding=5)[source]¶ Generates a stamp of the forward model and stores it in self.fm_stamp :param fm_image: full imgae containing the fm_stamp :param fm_center: [x,y] center of image (assuing fm_stamp is located at sep/PA) corresponding to guess_sep and guess_pa :param fm_wcs: if not None, specifies the sky angles in the image. If None, assume image is North up East left :param extract: if True, need to extract the forward model from the image. Otherwise, assume the fm_stamp is already
centered in the frame (fm_image.shape // 2)Parameters: padding – number of pixels on each side in addition to the fitboxsize to extract to pad the fm_stamp (should be >= 1) Returns:

propogate_errs
(star_center_err=None, platescale=None, platescale_err=None, pa_offset=None, pa_uncertainty=None)[source]¶ Propogate astrometric error. Stores results in its own fields
Parameters:  star_center_err (float) – uncertainity of the star location (pixels)
 platescale (float) – mas/pix conversion to angular coordinates
 platescale_err (float) – mas/pix error on the platescale
 pa_offset (float) – Offset, in the same direction as position angle, to set North up (degrees)
 pa_uncertainity (float) – Error on position angle/true North calibration (Degrees)

class
pyklip.fitpsf.
FitPSF
(fitboxsize, method='mcmc')[source]¶ Bases:
object
Base class to perform astrometry on direct imaging data_stamp using GP regression. Can utilize a Bayesian framework with MCMC or a frequentist framework with least squares.
Parameters:  fitboxsize – fitting box side length (pixels)
 method (str) – either ‘mcmc’ or ‘maxl’ depending on framework you want. Defaults to ‘mcmc’.
 fmt (str) – either ‘seppa’ or ‘xy’ depending on how you want to input the guess coordiantes

guess_x
¶ (initialization) guess x position [pixels]
Type: float

guess_y
¶ (initialization) guess y positon [pixels]
Type: float

guess_flux
¶ guess scale factor between model and data
Type: float

fit_x
¶ (result) the result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

fit_y
¶ (result) the result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

fit_flux
¶ (result) factor to scale the FM to match the flux of the data
Type: pyklip.fitpsf.ParamRange

covar_params
¶ (result) hyperparameters for the Gaussian processa
Type: list of pyklip.fitpsf.ParamRange

fm_stamp
¶ (fitting) The 2D stamp of the forward model (centered at the nearest pixel to the guessed location)
Type: np.array

data_stamp
¶ (fitting) The stamp of the data (centered at the nearest pixel to the guessed location) (2D unless there were NaNs in which 1D)
Type: np.array

noise_map
¶ (fitting) The stamp of the noise for each pixel the data computed assuming azimuthally similar noise (same dim as data_stamp)
Type: np.array

padding
¶ amount of pixels on one side to pad the data/forward model stamp
Type: int

sampler
¶ an instance of the emcee EnsambleSampler. Only for Bayesian fit. See emcee docs for more details.
Type: emcee.EnsembleSampler

best_fit_and_residuals
(fig=None)[source]¶ Generate a plot of the best fit FM compared with the data_stamp and also the residuals :param fig: if not None, a matplotlib Figure object :type fig: matplotlib.Figure
Returns: the Figure object. If input fig is None, function will make a new one Return type: fig (matplotlib.Figure)

fit_psf
(nwalkers=100, nburn=200, nsteps=800, save_chain=True, chain_output='bkachain.pkl', numthreads=None)[source]¶ Fits the PSF to the data in either a frequentist or Bayesian way depending on initialization.
Parameters:  nwalkers – number of walkers (mcmconly)
 nburn – numbe of samples of burnin for each walker (mcmconly)
 nsteps – number of samples each walker takes (mcmconly)
 save_chain – if True, save the output in a pickled file (mcmconly)
 chain_output – filename to output the chain to (mcmconly)
 numthreads – number of threads to use (mcmconly)
Returns:

generate_data_stamp
(data, guess_loc, noise_map, radial_noise_center=None, dr=4, exclusion_radius=10)[source]¶ Generate a stamp of the data_stamp ~centered on planet and also corresponding noise map :param data: the final collapsed data_stamp (2D) :param guess_loc: guess location of where to fit the model in the data :param noise_map: if not None, noise map for each pixel (either same shape as input data, or shape of data stamp)
if None, one will be generated assuming azimuthal noise using an annulus widthh of dr. radial_noise_center MUST be defined.Parameters:  radial_noise_center – if we assume the noise is azimuthally symmetric and changes radially, this is the [x,y] center for it
 dr – width of annulus in pixels from which the noise map will be generated
 exclusion_radius – radius around the guess planet location which doens’t get factored into the radial noise estimate
Returns:

generate_fm_stamp
(fm_image, fm_pos=None, fm_wcs=None, extract=True, padding=5)[source]¶ Generates a stamp of the forward model and stores it in self.fm_stamp :param fm_image: full image containing the fm_stamp :param fm_pos: [x,y] location of the forwrd model in the fm_image :param fm_wcs: if not None, specifies the sky angles in the image. If None, assume image is North up East left :param extract: if True, need to extract the forward model from the image. Otherwise, assume the fm_stamp is already
centered in the frame (fm_image.shape // 2)Parameters: padding – number of pixels on each side in addition to the fitboxsize to extract to pad the fm_stamp (should be >= 1) Returns:

guess_flux

make_corner_plot
(fig=None)[source]¶ Generate a corner plot of the posteriors from the MCMC :param fig: if not None, a matplotlib Figure object
Returns: the Figure object. If input fig is None, function will make a new one Return type: fig

set_bounds
(dx, dy, df, covar_param_bounds, read_noise_bounds=None)[source]¶ Set bounds on Bayesian priors. All paramters can be a 2 element tuple/list/array that specifies the lower and upper bounds x_min < x < x_max. Or a single value whose interpretation is specified below If you are passing in both lower and upper bounds, both should be in linear scale! :param dx: Distance from initial guess position in pixels. For a single value, this specifies the largest distance
form the initial guess (i.e. x_guess  dx < x < x_guess + dx)Parameters:  dy – Same as dx except with y
 df – Flux range. If single value, specifies how many orders of 10 the flux factor can span in one direction (i.e. log_10(guess_flux)  df < log_10(guess_flux) < log_10(guess_flux) + df
 covar_param_bounds – Params for covariance matrix. Like df, single value specifies how many orders of magnitude parameter can span. Otherwise, should be a list of 2elem touples
 read_noise_bounds – Param for read noise term. If single value, specifies how close to 0 it can go based on powers of 10 (i.e. log_10(read_noise_bound) < read_noise < 1 )
Returns:

set_kernel
(covar, covar_param_guesses, covar_param_labels, include_readnoise=False, read_noise_fraction=0.01)[source]¶ Set the Gaussian process kernel used in our fit
Parameters:  covar – Covariance kernel for GP regression. If string, can be “matern32” or “sqexp” or “diag” Can also be a function: cov = cov_function(x_indices, y_indices, sigmas, cov_params)
 covar_param_guesses – a list of guesses on the hyperparmeteres (size of N_hyperparams). This can be an empty list for ‘diag’.
 covar_param_labels – a list of strings labelling each covariance parameter
 include_readnoise – if True, part of the noise is a purely diagonal term (i.e. read/photon noise)
 read_noise_fraction – fraction of the total measured noise is read noise (between 0 and 1)
Returns:

class
pyklip.fitpsf.
ParamRange
(bestfit, err_range)[source]¶ Bases:
object
Stores the best fit value and uncertainities for a parameter in a neat fasion
Parameters:  bestfit (float) – the bestfit value
 err_range – either a float or a 2element tuple (+val1, val2) and gives the 1sigma range

bestfit
¶ the bestfit value
Type: float

error
¶ the average 1sigma error
Type: float

error_2sided
¶ [+error1, error2] 2element array with asymmetric errors
Type: np.array

class
pyklip.fitpsf.
PlanetEvidence
(guess_sep, guess_pa, fitboxsize, sampling_outputdir, l_only=False, fm_basename='Planet', null_basename='Null')[source]¶ Bases:
pyklip.fitpsf.FMAstrometry
Specifically for nested sampling of the parameter space of a directly imaged companion relative to its star. Extension of
pyklip.fitpsf.FitPSF
.Parameters:  guess_sep – the guessed separation (pixels)
 guess_pa – the guessed position angle (degrees)
 fitboxsize – fitting box side length (pixels)
 fm_basename – Prefix of the foward model sampling files multinest saves in /chains/
 null_basename – Prefix of the null hypothesis model sampling files multinest saves in /chains/

guess_sep
¶ (initialization) guess separation for planet [pixels]
Type: float

guess_pa
¶ (initialization) guess PA for planet [degrees]
Type: float

guess_RA_offset
¶ (initialization) guess RA offset [pixels]
Type: float

guess_Dec_offset
¶ (initialization) guess Dec offset [pixels]
Type: float

raw_RA_offset
¶ (result) the raw result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

raw_Dec_offset
¶ (result) the raw result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

raw_flux
¶ (result) factor to scale the FM to match the flux of the data
Type: pyklip.fitpsf.ParamRange

covar_params
¶ (result) hyperparameters for the Gaussian process
Type: list of pyklip.fitpsf.ParamRange

raw_sep
¶ (result) the inferred raw result from the MCMC fit for the planet’s location [pixels]
Type: pyklip.fitpsf.ParamRange

raw_PA
¶ (result) the inferred raw result from the MCMC fit for the planet’s location [degrees]
Type: pyklip.fitpsf.ParamRange

RA_offset
¶ (result) the RA offset of the planet that includes all astrometric errors [pixels or mas]
Type: pyklip.fitpsf.ParamRange

Dec_offset
¶ (result) the Dec offset of the planet that includes all astrometric errors [pixels or mas]
Type: pyklip.fitpsf.ParamRange

sep
¶ (result) the separation of the planet that includes all astrometric errors [pixels or mas]
Type: pyklip.fitpsf.ParamRange

PA
¶ (result) the PA of the planet that includes all astrometric errors [degrees]
Type: pyklip.fitpsf.ParamRange

fm_stamp
¶ (fitting) The 2D stamp of the forward model (centered at the nearest pixel to the guessed location)
Type: np.array

data_stamp
¶ (fitting) The 2D stamp of the data (centered at the nearest pixel to the guessed location)
Type: np.array

noise_map
¶ (fitting) The 2D stamp of the noise for each pixel the data computed assuming azimuthally similar noise
Type: np.array

padding
¶ amount of pixels on one side to pad the data/forward model stamp
Type: int

sampler
¶ function that runs the pymultinest sampling for both hypotheses
Type: pymultinest.run

pyklip.fitpsf.
lnlike
(fitparams, fma, cov_func, readnoise=False, negate=False)[source]¶ Likelihood function :param fitparams: array of params (size N). First three are [dRA,dDec,f]. Additional parameters are GP hyperparams
dRA,dDec: RA,Dec offsets from star. Also coordianaes in self.data_{RA,Dec}_offset f: flux scale factor to normalizae the flux of the data_stamp to the modelParameters:  fma (FMAstrometry) – a FMAstrometry object that has been fully set up to run
 cov_func (function) – function that given an input [x,y] coordinate array returns the covariance matrix e.g. cov = cov_function(x_indices, y_indices, sigmas, cov_params)
 readnoise (bool) – If True, the last fitparam fits for diagonal noise
 negate (bool) – if True, negatives the probability (used for minimization algos)
Returns: log of likelihood function (minus a constant factor)
Return type: likeli

pyklip.fitpsf.
lnprior
(fitparams, bounds, readnoise=False, negate=False)[source]¶ Bayesian prior
Parameters:  fitparams – array of params (size N)
 bounds – array of (N,2) with corresponding lower and upper bound of params bounds[i,0] <= fitparams[i] < bounds[i,1]
 readnoise (bool) – If True, the last fitparam fits for diagonal noise
 negate (bool) – if True, negatives the probability (used for minimization algos)
Returns: 0 if inside bound ranges, inf if outside
Return type: prior

pyklip.fitpsf.
lnprob
(fitparams, fma, bounds, cov_func, readnoise=False, negate=False)[source]¶ Function to compute the relative posterior probabiltiy. Product of likelihood and prior :param fitparams: array of params (size N). First three are [dRA,dDec,f]. Additional parameters are GP hyperparams
dRA,dDec: RA,Dec offsets from star. Also coordianaes in self.data_{RA,Dec}_offset f: flux scale factor to normalizae the flux of the data_stamp to the modelParameters:  fma – a FMAstrometry object that has been fully set up to run
 bounds – array of (N,2) with corresponding lower and upper bound of params bounds[i,0] <= fitparams[i] < bounds[i,1]
 cov_func – function that given an input [x,y] coordinate array returns the covariance matrix e.g. cov = cov_function(x_indices, y_indices, sigmas, cov_params)
 readnoise (bool) – If True, the last fitparam fits for diagonal noise
 negate (bool) – if True, negatives the probability (used for minimization algos)
Returns:

pyklip.fitpsf.
quick_psf_fit
(data, psf, x_guess, y_guess, fitboxsize)[source]¶ A wrapper for a quick maximum likelihood fit to a PSF to the data.
Parameters:  data (np.array) – 2D data frame
 psf (np.array) – 2D PSF template. This should be smaller than the size of data and larger than the fitboxsize
 x_guess (float) – approximate x position of the location you are fitting the psf to
 y_guess (float) – approximate y position of the location you are fitting the psf to
 fitboxsize (int) – fitting region is a square. This is the lenght of one side of the square
Returns: x_fit, y_fit, flux_fit x_fit (float): x position y_fit (float): y position flux_fit (float): multiplicative scale factor for the psf to match the data
pyklip.fm module¶

pyklip.fm.
calculate_fm
(delta_KL_nospec, original_KL, numbasis, sci, model_sci, inputflux=None)[source]¶ Calculate what the PSF looks up postKLIP using knowledge of the input PSF, assumed spectrum of the science target, and the partially calculated KL modes (Delta Z_k^lambda in Laurent’s paper). If inputflux is None, the spectral dependence has already been folded into delta_KL_nospec (treat it as delta_KL).
Note: if inputflux is None and delta_KL_nospec has three dimensions (ie delta_KL_nospec was calculated using pertrurb_nospec() or perturb_nospec_modelsBased()) then only klipped_oversub and klipped_selfsub are returned. Besides they will have an extra first spectral dimension.
Parameters:  delta_KL_nospec – perturbed KL modes but without the spectral info. delta_KL = spectrum x delta_Kl_nospec. Shape is (numKL, wv, pix). If inputflux is None, delta_KL_nospec = delta_KL
 orignal_KL – unpertrubed KL modes (array of size [numbasis, numpix])
 numbasis – array of KL mode cutoffs If numbasis is [None] the number of KL modes to be used is automatically picked based on the eigenvalues.
 sci – array of size p representing the science data
 model_sci – array of size p corresponding to the PSF of the science frame
 input_spectrum – array of size wv with the assumed spectrum of the model
If delta_KL_nospec does NOT include a spectral dimension or if inputflux is not None: :returns:
 array of shape (b,p) showing the forward modelled PSF
 Skipped if inputflux = None, and delta_KL_nospec has 3 dimensions.
klipped_oversub: array of shape (b, p) showing the effect of oversubtraction as a function of KL modes klipped_selfsub: array of shape (b, p) showing the effect of selfsubtraction as a function of KL modes Note: psf_FM = model_sci  klipped_oversub  klipped_selfsub to get the FM psf as a function of K Lmodes
(shape of b,p)Return type: fm_psf If inputflux = None and if delta_KL_nospec include a spectral dimension: :returns: Sum(<SKL>KL) with klipped_oversub.shape = (size(numbasis),Npix)
klipped_selfsub: Sum(<NDKL>KL) + Sum(<NKL>DKL) with klipped_selfsub.shape = (size(numbasis),N_lambda or N_ref,N_pix)Return type: klipped_oversub

pyklip.fm.
calculate_fm_singleNumbasis
(delta_KL_nospec, original_KL, numbasis, sci, model_sci, inputflux=None)[source]¶ Same function as calculate_fm() but faster when numbasis has only one element. It doesn’t do the mutliplication with the triangular matrix.
Calculate what the PSF looks up postKLIP using knowledge of the input PSF, assumed spectrum of the science target, and the partially calculated KL modes (Delta Z_k^lambda in Laurent’s paper). If inputflux is None, the spectral dependence has already been folded into delta_KL_nospec (treat it as delta_KL).
Note: if inputflux is None and delta_KL_nospec has three dimensions (ie delta_KL_nospec was calculated using pertrurb_nospec() or perturb_nospec_modelsBased()) then only klipped_oversub and klipped_selfsub are returned. Besides they will have an extra first spectral dimension.
Parameters:  delta_KL_nospec – perturbed KL modes but without the spectral info. delta_KL = spectrum x delta_Kl_nospec. Shape is (numKL, wv, pix). If inputflux is None, delta_KL_nospec = delta_KL
 orignal_KL – unpertrubed KL modes (array of size [numbasis, numpix])
 numbasis – array of (ONE ELEMENT ONLY) KL mode cutoffs If numbasis is [None] the number of KL modes to be used is automatically picked based on the eigenvalues.
 sci – array of size p representing the science data
 model_sci – array of size p corresponding to the PSF of the science frame
 input_spectrum – array of size wv with the assumed spectrum of the model
If delta_KL_nospec does NOT include a spectral dimension or if inputflux is not None: :returns:
 array of shape (b,p) showing the forward modelled PSF
 Skipped if inputflux = None, and delta_KL_nospec has 3 dimensions.
klipped_oversub: array of shape (b, p) showing the effect of oversubtraction as a function of KL modes klipped_selfsub: array of shape (b, p) showing the effect of selfsubtraction as a function of KL modes Note: psf_FM = model_sci  klipped_oversub  klipped_selfsub to get the FM psf as a function of K Lmodes
(shape of b,p)Return type: fm_psf If inputflux = None and if delta_KL_nospec include a spectral dimension: :returns: Sum(<SKL>KL) with klipped_oversub.shape = (size(numbasis),Npix)
klipped_selfsub: Sum(<NDKL>KL) + Sum(<NKL>DKL) with klipped_selfsub.shape = (size(numbasis),N_lambda or N_ref,N_pix)Return type: klipped_oversub

pyklip.fm.
calculate_validity
(covar_perturb, models_ref, numbasis, evals_orig, covar_orig, evecs_orig, KL_orig, delta_KL)[source]¶  Calculate the validity of the perturbation based on the eigenvalues or the 2nd order term compared
 to the 0th order term of the covariance matrix expansion
Parameters:  evals_perturb – linear expansion of the perturbed covariance matrix (C_AS). Shape of N x N
 models_ref – N x p array of the N models corresponding to reference images. Each model should contain spectral information
 numbasis – array of KL mode cutoffs
 evevs_orig – size of [N, maxKL]
Returns: perturbed KL modes. Shape is (numKL, wv, pix)
Return type: delta_KL_nospec

pyklip.fm.
find_id_nearest
(array, value)[source]¶ Find index of the closest value in input array to input value :param array: 1D array :param value: scalar value
Returns: Index of the nearest value in array

pyklip.fm.
klip_dataset
(dataset, fm_class, mode='ADI+SDI', outputdir='.', fileprefix='pyklipfm', annuli=5, subsections=4, OWA=None, N_pix_sector=None, movement=None, flux_overlap=0.1, PSF_FWHM=3.5, minrot=0, padding=0, numbasis=None, maxnumbasis=None, numthreads=None, corr_smooth=1, calibrate_flux=False, aligned_center=None, psf_library=None, spectrum=None, highpass=False, annuli_spacing='constant', save_klipped=True, mute_progression=False, time_collapse='mean')[source]¶ Run KLIPFM on a dataset object
Parameters:  dataset – an instance of Instrument.Data (see instruments/ subfolder)
 fm_class – class that implements the the forward modelling functionality
 mode – one of [‘ADI’, ‘SDI’, ‘ADI+SDI’] for ADI, SDI, or ADI+SDI
 anuuli – Annuli to use for KLIP. Can be a number, or a list of 2element tuples (a, b) specifying the pixel bondaries (a <= r < b) for each annulus
 subsections – Sections to break each annuli into. Can be a number [integer], or a list of 2element tuples (a, b) specifying the positon angle boundaries (a <= PA < b) for each section [radians]
 OWA – if defined, the outer working angle for pyklip. Otherwise, it will pick it as the cloest distance to a nan in the first frame
 N_pix_sector –
Rough number of pixels in a sector. Overwriting subsections and making it sepration dependent. The number of subsections is defined such that the number of pixel is just higher than N_pix_sector. I.e. subsections = floor(pi*(r_max^2r_min^2)/N_pix_sector) Warning: There is a bug if N_pix_sector is too big for the first annulus. The annulus is defined from
0 to 2pi which create a bug later on. It is probably in the way pa_start and pa_end are defined in fm_from_eigen(). (I am taking about matched filter by the way)  movement – minimum amount of movement (in pixels) of an astrophysical source to consider using that image for a refernece PSF
 flux_overlap – Maximum fraction of flux overlap between a slice and any reference frames included in the covariance matrix. Flux_overlap should be used instead of “movement” when a template spectrum is used. However if movement is not None then the old code is used and flux_overlap is ignored. The overlap is estimated for 1D gaussians with FWHM defined by PSF_FWHM. So note that the overlap is not exactly the overlap of two real 2D PSF for a given instrument but it will behave similarly.
 PSF_FWHM – FWHM of the PSF used to calculate the overlap (cf flux_overlap). Default is FWHM = 3.5 corresponding to sigma ~ 1.5.
 minrot – minimum PA rotation (in degrees) to be considered for use as a reference PSF (good for disks)
 padding – for each sector, how many extra pixels of padding should we have around the sides.
 numbasis – number of KL basis vectors to use (can be a scalar or list like). Length of b If numbasis is [None] the number of KL modes to be used is automatically picked based on the eigenvalues.
 maxnumbasis – Number of KL modes to be calculated from whcih numbasis modes will be taken.
 corr_smooth (float) – size of sigma of Gaussian smoothing kernel (in pixels) when computing most correlated PSFs. If 0, no smoothing
 numthreads – number of threads to use. If none, defaults to using all the cores of the cpu
 calibrate_flux – if true, flux calibrates the regular KLIP subtracted data. DOES NOT CALIBRATE THE FM
 aligned_center – array of 2 elements [x,y] that all the KLIP subtracted images will be centered on for image registration
 spectrum –
 (only applicable for SDI) if not None, optimizes the choice of the reference PSFs based on the
 spectrum shape.
 an array: of length N with the flux of the template spectrum at each wavelength.
 a string: Currently only supports “methane” between 1 and 10 microns.
Uses minmove to determine the separation from the center of the segment to determine contamination and the size of the PSF (TODO: make PSF size another quantity) (e.g. minmove=3, checks how much contamination is within 3 pixels of the hypothetical source) if smaller than 10%, (hard coded quantity), then use it for reference PSF
 highpass – if True, run a Gaussian high pass filter (default size is sigma=imgsize/10) can also be a number specifying FWHM of box in pixel units
 annuli_spacing – how to distribute the annuli radially. Currently three options. Constant (equally spaced), log (logarithmical expansion with r), and linear (linearly expansion with r)
 save_klipped – if True, will save the regular klipped image. If false, it wil not and sub_imgs will return None
 mute_progression – Mute the printing of the progression percentage. Indeed sometimes the overwriting feature doesn’t work and one ends up with thousands of printed lines. Therefore muting it can be a good idea.
 time_collapse – how to collapse the data in time. Currently support: “mean”, “weightedmean”

pyklip.fm.
klip_math
(sci, refs, numbasis, covar_psfs=None, model_sci=None, models_ref=None, spec_included=False, spec_from_model=False)[source]¶ linear algebra of KLIP with linear perturbation disks and point source
Parameters:  sci – array of length p containing the science data
 refs – N x p array of the N reference images that characterizes the extended source with p pixels
 numbasis – number of KLIP basis vectors to use (can be an int or an array of ints of length b) If numbasis is [None] the number of KL modes to be used is automatically picked based on the eigenvalues.
 covar_psfs – covariance matrix of reference images (for large N, useful). Normalized following numpy normalization in np.cov documentation
 The following arguments must all be passed in, or none of them for klip_math to work (#) –
 models_ref – N x p array of the N models corresponding to reference images. Each model should be normalized to unity (no flux information)
 model_sci – array of size p corresponding to the PSF of the science frame
 Sel_wv – wv x N array of the the corresponding wavelength for each reference PSF
 input_spectrum – array of size wv with the assumed spectrum of the model
Returns:  array of shape (p,b) that is the PSF subtracted data for each of the b KLIP basis
cutoffs. If numbasis was an int, then sub_img_row_selected is just an array of length p
KL_basis: array of KL basis (shape of [numbasis, p]) If models_ref is passed in (not None):
delta_KL_nospec: array of shape (b, wv, p) that is the almost perturbed KL modes just missing spectral info
 Otherwise:
evals: array of eigenvalues (size of max number of KL basis requested aka nummaxKL) evecs: array of corresponding eigenvectors (shape of [p, nummaxKL])
Return type: sub_img_rows_selected

pyklip.fm.
klip_parallelized
(imgs, centers, parangs, wvs, IWA, fm_class, OWA=None, mode='ADI+SDI', annuli=5, subsections=4, movement=None, flux_overlap=0.1, PSF_FWHM=3.5, numbasis=None, maxnumbasis=None, corr_smooth=1, aligned_center=None, numthreads=None, minrot=0, maxrot=360, spectrum=None, psf_library=None, psf_library_good=None, psf_library_corr=None, padding=0, save_klipped=True, flipx=True, N_pix_sector=None, mute_progression=False, annuli_spacing='constant', compute_noise_cube=False)[source]¶ multithreaded KLIP PSF Subtraction
Parameters:  imgs – array of 2D images for ADI. Shape of array (N,y,x)
 centers – N by 2 array of (x,y) coordinates of image centers
 parangs – N length array detailing parallactic angle of each image
 wvs – N length array of the wavelengths
 IWA – inner working angle (in pixels)
 fm_class – class that implements the the forward modelling functionality
 OWA – if defined, the outer working angle for pyklip. Otherwise, it will pick it as the cloest distance to a nan in the first frame
 mode – one of [‘ADI’, ‘SDI’, ‘ADI+SDI’] for ADI, SDI, or ADI+SDI
 anuuli – Annuli to use for KLIP. Can be a number, or a list of 2element tuples (a, b) specifying the pixel bondaries (a <= r < b) for each annulus
 subsections – Sections to break each annuli into. Can be a number [integer], or a list of 2element tuples (a, b) specifying the positon angle boundaries (a <= PA < b) for each section [radians]
 N_pix_sector –
Rough number of pixels in a sector. Overwriting subsections and making it sepration dependent. The number of subsections is defined such that the number of pixel is just higher than N_pix_sector. I.e. subsections = floor(pi*(r_max^2r_min^2)/N_pix_sector) Warning: There is a bug if N_pix_sector is too big for the first annulus. The annulus is defined from
0 to 2pi which create a bug later on. It is probably in the way pa_start and pa_end are defined in fm_from_eigen().  movement – minimum amount of movement (in pixels) of an astrophysical source to consider using that image for a refernece PSF
 flux_overlap – Maximum fraction of flux overlap between a slice and any reference frames included in the covariance matrix. Flux_overlap should be used instead of “movement” when a template spectrum is used. However if movement is not None then the old code is used and flux_overlap is ignored. The overlap is estimated for 1D gaussians with FWHM defined by PSF_FWHM. So note that the overlap is not exactly the overlap of two real 2D PSF for a given instrument but it will behave similarly.
 PSF_FWHM – FWHM of the PSF used to calculate the overlap (cf flux_overlap). Default is FWHM = 3.5 corresponding to sigma ~ 1.5.
 numbasis – number of KL basis vectors to use (can be a scalar or list like). Length of b If numbasis is [None] the number of KL modes to be used is automatically picked based on the eigenvalues.
 maxnumbasis – Number of KL modes to be calculated from whcih numbasis modes will be taken.
 corr_smooth (float) – size of sigma of Gaussian smoothing kernel (in pixels) when computing most correlated PSFs. If 0, no smoothing
 aligned_center – array of 2 elements [x,y] that all the KLIP subtracted images will be centered on for image registration
 numthreads – number of threads to use. If none, defaults to using all the cores of the cpu
 minrot – minimum PA rotation (in degrees) to be considered for use as a reference PSF (good for disks)
 maxrot – maximum PA rotation (in degrees) to be considered for use as a reference PSF (temporal variability)
 spectrum – if not None, a array of length N with the flux of the template spectrum at each wavelength. Uses minmove to determine the separation from the center of the segment to determine contamination and the size of the PSF (TODO: make PSF size another quanitity) (e.g. minmove=3, checks how much containmination is within 3 pixels of the hypothetical source) if smaller than 10%, (hard coded quantity), then use it for reference PSF
 padding – for each sector, how many extra pixels of padding should we have around the sides.
 save_klipped – if True, will save the regular klipped image. If false, it wil not and sub_imgs will return None
 flipx – if True, flips x axis after rotation to get North up East left
 mute_progression – Mute the printing of the progression percentage. Indeed sometimes the overwriting feature doesn’t work and one ends up with thousands of printed lines. Therefore muting it can be a good idea.
 annuli_spacing – how to distribute the annuli radially. Currently three options. Constant (equally spaced), log (logarithmical expansion with r), and linear (linearly expansion with r)
 compute_noise_cube – if True, compute the noise in each pixel assuming azimuthally uniform noise
Returns:  array of [array of 2D images (PSF subtracted)] using different number of KL basis vectors as
specified by numbasis. Shape of (b,N,y,x).
Note: this will be None if save_klipped is False
fmout_np: output of forward modelling. perturbmag: output indicating the magnitude of the linear perturbation to assess validity of KLIP FM aligned_center: (x, y) location indicating the star center for all images and FM after PSF subtraction
Return type: sub_imgs

pyklip.fm.
pertrurb_nospec
(evals, evecs, original_KL, refs, models_ref)[source]¶ Perturb the KL modes using a model of the PSF but with no assumption on the spectrum. Useful for planets.
By no assumption on the spectrum it means that the spectrum has been factored out of Delta_KL following equation (4) of Laurent Pueyo 2016 noted bold “Delta Z_k^lambda (x)”. In order to get the actual perturbed KL modes one needs to multpily it by a spectrum.
This function fits each cube’s spectrum independently. So the effective spectrum size is N_wavelengths * N_cubes.
Parameters:  evals – array of eigenvalues of the reference PSF covariance matrix (array of size numbasis)
 evecs – corresponding eigenvectors (array of size [p, numbasis])
 orignal_KL – unpertrubed KL modes (array of size [numbasis, p])
 Sel_wv – wv x N array of the the corresponding wavelength for each reference PSF
 refs – N x p array of the N reference images that characterizes the extended source with p pixels
 models_ref – N x p array of the N models corresponding to reference images. Each model should be normalized to unity (no flux information)
 model_sci – array of size p corresponding to the PSF of the science frame
Returns:  perturbed KL modes but without the spectral info. delta_KL = spectrum x delta_Kl_nospec.
Shape is (numKL, wv, pix)
Return type: delta_KL_nospec

pyklip.fm.
perturb_nospec_modelsBased
(evals, evecs, original_KL, refs, models_ref_list)[source]¶ Perturb the KL modes using a model of the PSF but with no assumption on the spectrum. Useful for planets.
By no assumption on the spectrum it means that the spectrum has been factored out of Delta_KL following equation (4) of Laurent Pueyo 2016 noted bold “Delta Z_k^lambda (x)”. In order to get the actual perturbed KL modes one needs to multpily it by a spectrum.
Effectively does the same thing as pertrurb_nospec() but in a different way. It injects models with dirac spectrum (all but one vanishing wavelength) and because of the linearity of the problem allow one de get reconstruct the perturbed KL mode for any spectrum. The difference however in the pertrurb_nospec() case is that the spectrum here is the asummed to be the same for all
cubes while pertrurb_nospec() fit each cube independently.Parameters:  evals –
 evecs –
 original_KL –
 refs –
 models_ref –
Returns: delta_KL_nospec

pyklip.fm.
perturb_specIncluded
(evals, evecs, original_KL, refs, models_ref, return_perturb_covar=False)[source]¶ Perturb the KL modes using a model of the PSF but with the spectrum included in the model. Quicker than the others
Parameters:  evals – array of eigenvalues of the reference PSF covariance matrix (array of size numbasis)
 evecs – corresponding eigenvectors (array of size [p, numbasis])
 orignal_KL – unpertrubed KL modes (array of size [numbasis, p])
 refs – N x p array of the N reference images that characterizes the extended source with p pixels
 models_ref – N x p array of the N models corresponding to reference images. Each model should contain spectral informatoin
 model_sci – array of size p corresponding to the PSF of the science frame
Returns: perturbed KL modes. Shape is (numKL, wv, pix)
Return type: delta_KL_nospec
pyklip.klip module¶

pyklip.klip.
align_and_scale
(img, new_center, old_center=None, scale_factor=1, dtype=<class 'float'>)[source]¶ Helper function that realigns and/or scales the image
Parameters:  img – 2D image to perform manipulation on
 new_center – 2 element tuple (xpos, ypos) of new image center
 old_center – 2 element tuple (xpos, ypos) of old image center
 scale_factor –
how much the stretch/contract the image. Will we scaled w.r.t the new_center (done after relaignment). We will adopt the convention
>1: stretch image (shorter to longer wavelengths) <1: contract the image (longer to shorter wvs) This means scale factor should be lambda_0/lambda where lambda_0 is the wavelength you want to scale to
Returns: shifted and/or scaled 2D image
Return type: resampled_img

pyklip.klip.
calc_scaling
(sats, refwv=18)[source]¶ Helper function that calculates the wavelength scaling factor from the satellite spot locations. Uses the movement of spots diagonally across from each other, to calculate the scaling in a (hopefully? tbd.) centeringindependent way. This method is definitely temporary and will be replaced by better scaling strategies as we come up with them. Scaling is calculated as the average of (1/2 * sqrt((x_1x_2)**2+(y_1y_2))), over the two pairs of spots.
Parameters:  sats – [4 x Nlambda x 2] array of x and y positions for the 4 satellite spots
 refwv – reference wavelength for scaling (optional, default = 20)
Returns: Nlambda array of scaling factors
Return type: scaling_factors

pyklip.klip.
collapse_data
(data, pixel_weights=None, axis=1, collapse_method='mean')[source]¶ Function to collapse multidimensional data along axis using collapse_method
Parameters:  data – (multidimension)arrays of 2D images or 3D cubes.
 pixel_weights – ones if collapse method is not weighted collapse
 axis – axis index along which to collapse
 collapse_method – currently support ‘median’, ‘mean’, ‘weightedmean’, ‘trimmedmean’, ‘weightedmedian’
Returns: Collapsed data

pyklip.klip.
define_annuli_bounds
(annuli, IWA, OWA, annuli_spacing='constant')[source]¶ Defines the annuli boundaries radially
Parameters:  annuli – number of annuli
 IWA – inner working angle (pixels)
 OWA – outer working anglue (pixels)
 annuli_spacing – how to distribute the annuli radially. Currently three options. Constant (equally spaced), log (logarithmical expansion with r), and linear (linearly expansion with r)
Returns: array of 2element tuples that specify the beginning and end radius of that annulus
Return type: rad_bounds

pyklip.klip.
estimate_movement
(radius, parang0=None, parangs=None, wavelength0=None, wavelengths=None, mode=None)[source]¶ Estimates the movement of a hypothetical astrophysical source in ADI and/or SDI at the given radius and given reference parallactic angle (parang0) and reference wavelegnth (wavelength0)
Parameters:  radius – the radius from the star of the hypothetical astrophysical source
 parang0 – the parallactic angle of the reference image (in degrees)
 parangs – array of length N of the parallactic angle of all N images (in degrees)
 wavelength0 – the wavelength of the reference image
 wavelengths – array of length N of the wavelengths of all N images
 NOTE – we expect parang0 and parangs to be either both defined or both None. Same with wavelength0 and wavelengths
 mode – one of [‘ADI’, ‘SDI’, ‘ADI+SDI’] for ADI, SDI, or ADI+SDI
Returns:  array of length N of the distance an astrophysical source would have moved from the
reference image
Return type: moves

pyklip.klip.
high_pass_filter
(img, filtersize=10)[source]¶ A FFT implmentation of high pass filter.
Parameters:  img – a 2D image
 filtersize – size in Fourier space of the size of the space. In image space, size=img_size/filtersize
Returns: the filtered image
Return type: filtered

pyklip.klip.
klip_math
(sci, ref_psfs, numbasis, covar_psfs=None, return_basis=False, return_basis_and_eig=False)[source]¶ Helper function for KLIP that does the linear algebra
Parameters:  sci – array of length p containing the science data
 ref_psfs – N x p array of the N reference PSFs that characterizes the PSF of the p pixels
 numbasis – number of KLIP basis vectors to use (can be an int or an array of ints of length b)
 covar_psfs – covariance matrix of reference psfs passed in so you don’t have to calculate it here
 return_basis – If true, return KL basis vectors (used when onesegment==True)
 return_basis_and_eig – If true, return KL basis vectors as well as the eigenvalues and eigenvectors of the covariance matrix. Used for KLIP Forward Modelling of Laurent Pueyo.
Returns:  array of shape (p,b) that is the PSF subtracted data for each of the b KLIP basis
cutoffs. If numbasis was an int, then sub_img_row_selected is just an array of length p
KL_basis: array of shape (max(numbasis),p). Only if return_basis or return_basis_and_eig is True. evals: Eigenvalues of the covariance matrix. The covariance matrix is assumed NOT to be normalized by (p1).
Only if return_basis_and_eig is True.
 evecs: Eigenvectors of the covariance matrix. The covariance matrix is assumed NOT to be normalized by (p1).
Only if return_basis_and_eig is True.
Return type: sub_img_rows_selected

pyklip.klip.
make_polar_coordinates
(x, y, center=[0, 0])[source]¶ Parameters:  x – meshgrid of x coordinates
 y – meshgrid of y coordinates
 center – new location of origin
Returns: polar coordinates centered at the specified origin

pyklip.klip.
meas_contrast
(dat, iwa, owa, resolution, center=None, low_pass_filter=True)[source]¶ Measures the contrast in the image. Image must already be in contrast units and should be corrected for algorithm thoughput.
Parameters:  dat – 2D image  already flux calibrated
 iwa – inner working angle
 owa – outer working angle
 resolution – size of noise resolution element in pixels (for speckle noise ~ FWHM or lambda/D) but it can be 1 pixel if limited by pixeltopixel noise.
 center – location of star (x,y). If None, defaults the image size // 2.
 low_pass_filter – if True, run a low pass filter. Can also be a float which specifices the width of the Gaussian filter (sigma). If False, no Gaussian filter is run
Returns: tuple of separations in pixels and corresponding 5 sigma FPF
Return type: (seps, contrast)

pyklip.klip.
nan_gaussian_filter
(img, sigma, ivar=None)[source]¶ Gaussian lowpass filter that handles nans
Parameters:  img – 2D image
 sigma – float specifiying width of Gaussian
 ivar – inverse variance frame for the image, optional
Returns: 2D image that has been smoothed with a Gaussian
Return type: filtered

pyklip.klip.
nan_map_coordinates_2d
(img, yp, xp, mc_kwargs=None)[source]¶ scipy.ndimage.map_coordinates() that handles nans for 2D transformations. Only works in 2D!
Do NaN detection by defining any pixel in the new coordiante system (xp, yp) as a nan If any one of the neighboring pixels in the original image is a nan (e.g. (xp, yp) = (120.1, 200.1) is nan if either (120, 200), (121, 200), (120, 201), (121, 201) is a nan)
Parameters:  img (np.array) – 2D image that is looking to be transformed
 yp (np.array) – 2D array of ycoordinates that the image is evaluated out
 xp (np.array) – 2D array of xcoordinates that the image is evaluated out
 mc_kwargs (dict) – other parameters to pass into the map_coordinates function.
Returns: 2D transformed image. Each pixel is evaluated at the (yp, xp) specified by xp and yp.
Return type: transformed_img (np.array)

pyklip.klip.
rotate
(img, angle, center, new_center=None, flipx=False, astr_hdr=None)[source]¶ Rotate an image by the given angle about the given center. Optional: can shift the image to a new image center after rotation. Also can reverse x axis for those left
handed astronomy coordinate systemsParameters:  img – a 2D image
 angle – angle CCW to rotate by (degrees)
 center – 2 element list [x,y] that defines the center to rotate the image to respect to
 new_center – 2 element list [x,y] that defines the new image center after rotation
 flipx – reverses x axis after rotation
 astr_hdr – wcs astrometry header for the image
Returns: new 2D image
Return type: resampled_img
pyklip.nmf_imaging module¶
pyklip.parallelized module¶

pyklip.parallelized.
generate_noise_maps
(imgs, aligned_center, dr, IWA=None, OWA=None, numthreads=None, pool=None)[source]¶ Create a noise map for each image. The noise levels are computed using azimuthally averaged noise in the images
Parameters:  imgs – array of shape (N,y,x) containing N images
 aligned_center – [x,y] location of the center. All images should be aligned to common center
 dr (float) – how mnay pixels wide the annulus to compute the noise should be
 IWA (float) – inner working angle (how close to the center of the image to start). If none, this is 0
 OWA (float) – outer working angle (if None, it is the entire image.)
 numthreads – number of threads to be used
 pool – multiprocessing thread pool (optional). To avoid repeatedly creating one when processing a list of images.
Returns: array of shape (N,y,x) containing N noise maps
Return type: noise_maps

pyklip.parallelized.
high_pass_filter_imgs
(imgs, numthreads=None, filtersize=10, pool=None)[source]¶ filters a sequences of images using a FFT
Parameters:  imgs – array of shape (N,y,x) containing N images
 numthreads – number of threads to be used
 filtersize – size in Fourier space of the size of the space. In image space, size=img_size/filtersize
 pool – multiprocessing thread pool (optional). To avoid repeatedly creating one when processing a list of images.
Returns: array of shape (N,y,x) containing the filtered images
Return type: filtered

pyklip.parallelized.
klip_dataset
(dataset, mode='ADI+SDI', outputdir='.', fileprefix='', annuli=5, subsections=4, movement=3, numbasis=None, numthreads=None, minrot=0, calibrate_flux=False, aligned_center=None, annuli_spacing='constant', maxnumbasis=None, corr_smooth=1, spectrum=None, psf_library=None, highpass=False, lite=False, save_aligned=False, restored_aligned=None, dtype=None, algo='klip', time_collapse='mean', wv_collapse='mean', verbose=True)[source]¶ run klip on a dataset class outputted by an implementation of Instrument.Data
Parameters:  dataset – an instance of Instrument.Data (see instruments/ subfolder)
 mode – some combination of ADI, SDI, and RDI (e.g. “ADI+SDI”, “RDI”)
 outputdir – directory to save output files
 fileprefix – filename prefix for saved files
 anuuli – number of annuli to use for KLIP
 subsections – number of sections to break each annuli into
 movement – minimum amount of movement (in pixels) of an astrophysical source to consider using that image for a refernece PSF
 numbasis – number of KL basis vectors to use (can be a scalar or list like). Length of b
 numthreads – number of threads to use. If none, defaults to using all the cores of the cpu
 minrot – minimum PA rotation (in degrees) to be considered for use as a reference PSF (good for disks)
 calibrate_flux – if True calibrate flux of the dataset, otherwise leave it be
 aligned_center – array of 2 elements [x,y] that all the KLIP subtracted images will be centered on for image registration
 annuli_spacing – how to distribute the annuli radially. Currently three options. Constant (equally spaced), log (logarithmical expansion with r), and linear (linearly expansion with r)
 maxnumbasis – if not None, maximum number of KL basis/correlated PSFs to use for KLIP. Otherwise, use max(numbasis)
 corr_smooth (float) – size of sigma of Gaussian smoothing kernel (in pixels) when computing most correlated PSFs. If 0, no smoothing
 spectrum – (only applicable for SDI) if not None, optimizes the choice of the reference PSFs based on the spectrum shape.  an array: of length N with the flux of the template spectrum at each wavelength.  a string: Currently only supports “methane” between 1 and 10 microns. Uses minmove to determine the separation from the center of the segment to determine contamination and the size of the PSF (TODO: make PSF size another quantity) (e.g. minmove=3, checks how much contamination is within 3 pixels of the hypothetical source) if smaller than 10%, (hard coded quantity), then use it for reference PSF
 psf_library – if not None, a rdi.PSFLibrary object with a PSF Library for RDI
 highpass – if True, run a Gaussian high pass filter (default size is sigma=imgsize/10) can also be a number specifying FWHM of box in pixel units
 lite – if True, run a low memory version of the alogirhtm
 save_aligned – Save the aligned and scaled images (as well as various wcs information), True/False
 restore_aligned – The aligned and scaled images from a previous run of klip_dataset (usually restored_aligned = dataset.aligned_and_scaled)
 dtype – data type of the arrays. Should be either ctypes.c_float(default) or ctypes.c_double
 algo (str) – algorithm to use (‘klip’, ‘nmf’, ‘empca’, ‘none’). None will run no PSF subtraction.
 time_collapse – how to collapse the data in time. Currently support: “mean”, “weightedmean”, ‘median’, “weightedmedian”
 wv_collapse – how to collapse the data in wavelength. Currently support: ‘median’, ‘mean’, ‘trimmedmean’
 verbose (bool) – if True, print KLIP processes.
 Returns
Saved files in the output directory Returns: nothing, but saves to dataset.output: (b, N, wv, y, x) 5D cube of KL cutoff modes (b), number of images
(N), wavelengths (wv), and spatial dimensions. Images are derotated. For ADI only, the wv is omitted so only 4D cube

pyklip.parallelized.
klip_parallelized
(imgs, centers, parangs, wvs, filenums, IWA, OWA=None, mode='ADI+SDI', annuli=5, subsections=4, movement=3, numbasis=None, aligned_center=None, numthreads=None, minrot=0, maxrot=360, annuli_spacing='constant', maxnumbasis=None, corr_smooth=1, spectrum=None, psf_library=None, psf_library_good=None, psf_library_corr=None, save_aligned=False, restored_aligned=None, dtype=None, algo='klip', compute_noise_cube=False, verbose=True)[source]¶ Multitprocessed KLIP PSF Subtraction
Parameters:  imgs – array of 2D images for ADI. Shape of array (N,y,x)
 centers – N by 2 array of (x,y) coordinates of image centers
 parangs – N length array detailing parallactic angle of each image
 wvs – N length array of the wavelengths
 filenums (np.array) – array of length N of the filenumber for each image
 IWA – inner working angle (in pixels)
 OWA – outer working angle (in pixels)
 mode – one of [‘ADI’, ‘SDI’, ‘ADI+SDI’] for ADI, SDI, or ADI+SDI
 anuuli – number of annuli to use for KLIP
 subsections – number of sections to break each annuli into
 movement – minimum amount of movement (in pixels) of an astrophysical source to consider using that image for a refernece PSF
 numbasis – number of KL basis vectors to use (can be a scalar or list like). Length of b
 aligned_center – array of 2 elements [x,y] that all the KLIP subtracted images will be centered on for image registration
 numthreads – number of threads to use. If none, defaults to using all the cores of the cpu
 minrot – minimum PA rotation (in degrees) to be considered for use as a reference PSF (good for disks)
 maxrot – maximum PA rotation (in degrees) to be considered for use as a reference PSF (temporal variability)
 annuli_spacing – how to distribute the annuli radially. Currently three options. Constant (equally spaced), log (logarithmical expansion with r), and linear (linearly expansion with r)
 maxnumbasis – if not None, maximum number of KL basis/correlated PSFs to use for KLIP. Otherwise, use max(numbasis)
 corr_smooth (float) – size of sigma of Gaussian smoothing kernel (in pixels) when computing most correlated PSFs. If 0, no smoothing
 spectrum – if not None, a array of length N with the flux of the template spectrum at each wavelength. Uses minmove to determine the separation from the center of the segment to determine contamination and the size of the PSF (TODO: make PSF size another quanitity) (e.g. minmove=3, checks how much containmination is within 3 pixels of the hypothetical source) if smaller than 10%, (hard coded quantity), then use it for reference PSF
 psf_library – array of (N_lib, y, x) with N_lib PSF library PSFs
 psf_library_good – array of size N_lib indicating which N_good are good are selected in the passed in corr matrix
 psf_library_corr – matrix of size N_sci x N_good with correlation between the target franes and the good RDI PSFs
 save_aligned – Save the aligned and scaled images (as well as various wcs information), True/False
 restore_aligned – The aligned and scaled images from a previous run of klip_dataset (usually restored_aligned = dataset.aligned_and_scaled)
 dtype – data type of the arrays. Should be either ctypes.c_float(default) or ctypes.c_double
 algo (str) – algorithm to use (‘klip’, ‘nmf’, ‘empca’)
 compute_noise_cube – if True, compute the noise in each pixel assuming azimuthally uniform noise
Returns:  array of [array of 2D images (PSF subtracted)] using different number of KL basis vectors as
specified by numbasis. Shape of (b,N,y,x).
aligned_center: (x,y) specifying the common center the output images are aligned to
Return type: sub_imgs

pyklip.parallelized.
klip_parallelized_lite
(imgs, centers, parangs, wvs, filenums, IWA, OWA=None, mode='ADI+SDI', annuli=5, subsections=4, movement=3, numbasis=None, aligned_center=None, numthreads=None, minrot=0, maxrot=360, annuli_spacing='constant', maxnumbasis=None, corr_smooth=1, spectrum=None, dtype=None, algo='klip', compute_noise_cube=False, **kwargs)[source]¶ multithreaded KLIP PSF Subtraction, has a smaller memory foot print than the original
Parameters:  imgs – array of 2D images for ADI. Shape of array (N,y,x)
 centers – N by 2 array of (x,y) coordinates of image centers
 parangs – N length array detailing parallactic angle of each image
 wvs – N length array of the wavelengths
 filenums (np.array) – array of length N of the filenumber for each image
 IWA – inner working angle (in pixels)
 OWA – outer working angle (in pixels)
 mode – one of [‘ADI’, ‘SDI’, ‘ADI+SDI’] for ADI, SDI, or ADI+SDI
 anuuli – number of annuli to use for KLIP
 subsections – number of sections to break each annuli into
 movement – minimum amount of movement (in pixels) of an astrophysical source to consider using that image for a refernece PSF
 numbasis – number of KL basis vectors to use (can be a scalar or list like). Length of b
 annuli_spacing – how to distribute the annuli radially. Currently three options. Constant (equally spaced), log (logarithmical expansion with r), and linear (linearly expansion with r)
 maxnumbasis – if not None, maximum number of KL basis/correlated PSFs to use for KLIP. Otherwise, use max(numbasis)
 corr_smooth (float) – size of sigma of Gaussian smoothing kernel (in pixels) when computing most correlated PSFs. If 0, no smoothing
 aligned_center – array of 2 elements [x,y] that all the KLIP subtracted images will be centered on for image registration
 numthreads – number of threads to use. If none, defaults to using all the cores of the cpu
 minrot – minimum PA rotation (in degrees) to be considered for use as a reference PSF (good for disks)
 maxrot – maximum PA rotation (in degrees) to be considered for use as a reference PSF (temporal variability)
 spectrum – if not None, a array of length N with the flux of the template spectrum at each wavelength. Uses minmove to determine the separation from the center of the segment to determine contamination and the size of the PSF (TODO: make PSF size another quanitity) (e.g. minmove=3, checks how much containmination is within 3 pixels of the hypothetical source) if smaller than 10%, (hard coded quantity), then use it for reference PSF
 kwargs – in case you pass it stuff that we don’t use in the lite version
 dtype – data type of the arrays. Should be either ctypes.c_float (default) or ctypes.c_double
 algo (str) – algorithm to use (‘klip’, ‘nmf’, ‘empca’)
 compute_noise_cube – if True, compute the noise in each pixel assuming azimuthally uniform noise
Returns:  array of [array of 2D images (PSF subtracted)] using different number of KL basis vectors as
specified by numbasis. Shape of (b,N,y,x).
Return type: sub_imgs

pyklip.parallelized.
rotate_imgs
(imgs, angles, centers, new_center=None, numthreads=None, flipx=False, hdrs=None, disable_wcs_rotation=False, pool=None)[source]¶ derotate a sequences of images by their respective angles
Parameters:  imgs – array of shape (N,y,x) containing N images
 angles – array of length N with the angle to rotate each frame. Each angle should be CCW in degrees.
 centers – array of shape N,2 with the [x,y] center of each frame
 new_centers – a 2element array with the new center to register each frame. Default is middle of image
 numthreads – number of threads to be used
 flipx – flip the x axis after rotation if desired
 hdrs – array of N wcs astrometry headers
Returns: array of shape (N,y,x) containing the derotated images
Return type: derotated
pyklip.rdi module¶

class
pyklip.rdi.
PSFLibrary
(data, aligned_center, filenames, correlation_matrix=None, wvs=None, compute_correlation=False)[source]¶ Bases:
object
This is an PSF Library to use for reference differential imaging

master_library
¶ aligned library of PSFs. 3D cube of dim = [N, y, x]. Where N is ALL files
Type: np.ndarray

aligned_center
¶ a (x,y) coordinate specifying common center the library is aligned to
Type: arraylike

master_filenames
¶ array of N filenames for each frame in the library. Should match with pyklip.instruments.Data.filenames for crossmatching
Type: np.ndarray

master_correlation
¶ N x N array of correlations between each 2 frames
Type: np.ndarray

master_wvs
¶ N wavelengths for each frame
Type: np.ndarray

nfiles
¶ the number of files in the PSF library
Type: int

dataset
¶ Type: pyklip.instruments.Instrument.Data

correlation
¶ N_data x M array of correlations between each 2 frames where M are the selected frames and N_data is the number of files in the dataset. Along the N_data dimension, files are ordered in the same way as the dataset object
Type: np.ndarray

isgoodpsf
¶ array of N indicating which M PSFs are good for this dataset
Type: np.ndarray

add_new_dataset_to_library
(dataset, collapse=False, verbose=False)[source]¶ Add all the files from a new dataset to the PSF library and add them to the correlation matrix. If a mask was used for the correlation matrix, use it here too.
NOTE: This routine already assumes that the data has been centered.
Parameters: dataset (pyklip.instruments.Instrument.Data) –

prepare_library
(dataset, badfiles=None)[source]¶ Prepare the PSF Library for an RDI reduction of a specific dataset by only taking the part of the library we need.
Parameters:  dataset (pyklip.instruments.Instrument.Data) –
 badfiles (np.ndarray) – a list of filenames corresponding to bad files we want to also exclude
Returns:

save_correlation
(filename, overwrite=False, clobber=None, format='fits')[source]¶ Saves self.correlation to a file specified by filename :param filename: filepath to store the correlation matrix :type filename: str :param overwrite: if true overwrite the previous correlation matrix :type overwrite: bool :param clobber: same as overwrite, but deprecated in astropy. :type clobber: bool :param format: type of file to store the correlation matrix as. Supports numpy?/fits?/pickle? (TBD) :type format: str

pyklip.spectra_management module¶

pyklip.spectra_management.
calibrate_star_spectrum
(template_spectrum, template_wvs, filter_name, magnitude, wvs)[source]¶ Scale the Pickles stellar spectrum of a star to the observed apparent magnitude and returns the stellar spectrum in physical units sampled at the specified wavelengths. Currently only supports 2MASS filters and magnitudes. TODO: implement a way to take magnitude error input and propogate the error to the final spectrum
Parameters:  template_spectrum – 1D array, model spectrum of the star with arbitrary units
 template_wvs – 1D array, wavelengths at which the template_spectrum is smapled, in units of microns
 filter_name – string, 2MASS filter, ‘J’, ‘H’, or ‘Ks’
 magnitude – scalar, observed apparent magnitude of the star
 mag_error – scalar or 1D array with 2 elements, error(s) of the magnitude, not yet implemented
 wvs – 1D array, the wvs at which to sample the scaled spectrum, in units of angstroms
Returns: 1D array, scaled stellar spectrum sampled at wvs
Return type: scaled_spectrum

pyklip.spectra_management.
extract_planet_spectrum
(cube_para, position, PSF_cube_para, method=None, filter=None, mute=True)[source]¶

pyklip.spectra_management.
find_lower_nearest
(array, value)[source]¶ Find the lower nearest element to value in array.
Parameters:  array – Array of value
 value – Value for which one wants the lower value.
Returns: (low_value, id) with low_value the closest lower value and id its index.

pyklip.spectra_management.
find_nearest
(array, value)[source]¶ Find the nearest element to value in array.
Parameters:  array – Array of value
 value – Value for which one wants the closest value.
Returns: (closest_value, id) with closest_value the closest lower value and id its index.

pyklip.spectra_management.
find_upper_nearest
(array, value)[source]¶ Find the upper nearest element to value in array.
Parameters:  array – Array of value
 value – Value for which one wants the upper value.
Returns: (up_value, id) with up_value the closest upper value and id its index.

pyklip.spectra_management.
get_planet_spectrum
(spectrum, wavelength, ori_wvs=None)[source]¶ Get the normalized spectrum of a planet for a GPI spectral band or any wavelengths array. Spectra are extraced from .flx files from Mark Marley et al’s models.
Parameters:  spectrum – Path of the .flx file containing the spectrum.
 wavelength – array of wavelenths in microns (or string with GPI band ‘H’, ‘J’, ‘K1’, ‘K2’, ‘Y’). (When using GPI spectral band wavelength samples are linearly spaced between the first and the last wavelength of the band.)
Returns: is the gpi sampling of the considered band in micrometer. spectrum: is the spectrum of the planet for the given band or wavelength array and normalized to unit mean.
Return type: wavelengths

pyklip.spectra_management.
get_specType
(object_name, SpT_file_csv=None)[source]¶ Return the spectral type for a target based on Simbad or on the table in SpT_file
Parameters:  object_name – Name of the target: ie “c_Eri”
 SpT_file – Filename (.csv) of the table containing the target names and their spectral type. Can be generated from quering Simbad. If None (default), the function directly tries to query Simbad.
Returns: Spectral type

pyklip.spectra_management.
get_star_spectrum
(wvs_or_filter_name=None, star_type=None, temperature=None, mute=None)[source]¶ Get the spectrum of a star with given spectral type interpolating the pickles database. The spectrum is normalized to unit mean. It assumes type V star.
 Inputs:
 wvs_or_filter_name: array of wavelenths in microns (or string with GPI band ‘H’, ‘J’, ‘K1’, ‘K2’, ‘Y’).
 (When using GPI spectral band wavelength samples are linearly spaced between the first and the last wavelength of the band.)
 star_type: ‘A5’,’F4’,… Is ignored if temperature is defined.
 If star_type is longer than 2 characters it is truncated.
temperature: temperature of the star. Overwrite star_type if defined.
 Output:
 (wavelengths, spectrum) where
 wavelengths: Sampling in mum. spectrum: is the spectrum of the star for the given band.