napari LF

Light field imaging plugin for napari

---

Deconvolves a 4D light field image into a full 3D focus stack reconstruction

napari-LF contains an analytic and neural net analysis methods for light field images. To download example light field images, see our repository napari-LF-docs-samples.

LF Analyze¶

LF Analyze, the analytic method, provides three basic processes to Calibrate, Rectify, and Deconvolve light field images:

The Calibrate process generates a calibration file that represents the optical setup that was used to record the light field images. The same calibration file can be used to rectify and deconvolve all light field images that were recorded with the same optical setup, usually the same microscope and light field camera. The Calibrate process requires as input the radiometry frame, dark frame, optical parameters, and volume parameters to generate the calibration file, which is subsequently used to rectify and deconvolve related light field images. The calibration file includes a point spread function (PSF) derived from the optical and volume parameters and is stored in HDF5 file format.

The Rectify process uses the calibration file for an affine transformation to scale and rotate experimental light field images that were recorded with a light field camera whose microlens array was (slightly) rotated with respect to the pixel array of the area detector and whose pixel pitch is not commensurate with the microlens pitch. After rectification, the rectified light field has the same integer number of pixels behind each microlens. When the Deconvolve process is called for an experimental light field image, rectifying the light field image is automatically applied before the iterative deconvolution does begin. However, the rectified light field image is not saved and is not available for viewing. Therefore, by pushing the Rectify button in the middle of the napari-LF widget, only the rectification step is invoked and the rectified light field image is saved to the project directory.

The Deconvolve process uses the PSF and a wave optics model to iteratively deconvolve a light field image into a stack of optical sections.

The Parameter panels, located in the lower half of the napari-LF widget, allows the user to specify settings for the reconstruction process. Once the appropriate parameters are selected, the Calibrate button followed by the Deconvolve button can be pushed to complete the reconstruction.

Neural Net¶

Neural Net provides a method of applying a trained neural net model to deconvolve a light field image. Based on Pytorch Lightning and a provided base class, you can either create your own network, or use the pre-shipped networks (LFMNet, VCDNet, ...).

Quickstart¶

1. Install the napari-LF plugin into your napari environment, as described below under Installation.
2. From the napari Plugins menu, select the napari-LF plugin to install its widget into the napari viewer.

LF Analyze¶

1. Near the top of the widget, select your project folder containing the following images: light field, radiometry, and dark frame.
2. Calibration
   1. In the processing panel, navigate to Calibrate, Required (top tab Calibrate, bottom tab Required), which is the default selection.
   2. Select radiometry and dark frame images from pull down menus.
   3. Write the name of the calibration file you would like to produce, e.g. calibration.lfc.
   4. Enter the appropriate optical parameters according to your microscope and sample material.
   5. Enter the volume parameters you would like for your 3D reconstuction.
   6. Push the Calibrate button.
3. Deconvolution
   1. In the processing panel, navigate to Deconvolve, Required.
   2. Select light field image and calibration file from pull down menus.
   3. Write the name of the output image stack you would like to produce, e.g. output_stack.tif.
   4. Push the Deconvolve button. The 3D focal stack reconstruction will display in the napari viewer and be saved in your original project folder.

Neural Net¶

1. Click on the LF Analyze logo to toggle to the Neural Net mode.
2. Near the top of the widget, select your project folder containing the light field image and the trained neural net. If you do not already have a trained model, you can train a model using this Jupyter notebook.
3. In the processing panel, select your light field image and neural net model.
4. Write the name of the output image stack you would like to produce, e.g. output_stack.tif.
5. Push the Deconvolve button. The 3D focal stack reconstruction will display in the napari viewer and be saved in your original project folder.

Getting Help¶

For details about each parameter, hover over each parameter textbox to read the tooltip description. For additional information about the reconstruction process, see our User Guide.

Installation¶

After you have napari installed, you can one of the methods below to install napari-LF.

Method 1: You can install napari-LF via pip:

pip install napari-LF

Method 2: Use the napari plugin menu.

1. Open napari from the command line:

    napari

2. From the napari menu, select Plugins > Install/uninstall Packages.

3. Either (a) scroll through the list of available plugins to find napari-LF, or (b) drag and drop a downloaded napari-LF directory into the bottom bar.

4. Select Install to install the light field plugin.

Method 3: Install the latest development version from the command line.

pip install git+https://github.com/PolarizedLightFieldMicroscopy/napari-LF.git

Lastly, to access the installed plugin, open napari from the command line:

napari

From the napari menu, select Plugins > Main Menu (napari-LF). Note that you may need to close and reopen napari for the napari-LF to appear.

Installation for developers¶

Create a virtual environment from the command line for napari with the python libraries necessary for the light field plugin:

conda create -y -n napari-lf -c conda-forge python==3.9
conda activate napari-lf

Clone the github repository:

conda install git
git clone https://github.com/PolarizedLightFieldMicroscopy/napari-LF.git
cd napari-LF
pip install -e .

The necessary dependencies should be installed automatically with napari-LF. If for some reason pyopencl does not get installed properly, try installing with conda:

conda install -c conda-forge pyopencl

Contributing¶

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License¶

Distributed under the terms of the BSD-3 license, "napari-LF" is free and open source software

Issues¶

If you encounter any problems, please file an issue along with a detailed description.