Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Explaining the porting plan for Hessian Matrix

This notebook supports PR <>.

Unlike other notebooks in this repository, this one is hand-written by me (Matthew). I used AI only to check and review.

The purpose of this document is to set out the various trade-offs in the decisions I had to make for the PR.

The issues

There are two issues that this PR addresses, and then a variety of consequences.

Here’s the summary:

Background

The Skimage1 API hessian_matrix has two algorithms for calculating the Hessian, at a given Gaussian smoothing sigma, at each pixel. The Hessian HH is the second derivative of the image:

H=(HiiHijHijHjj),Hij=∂2∂xi ∂xj(Gσ∗f)H = \begin{pmatrix} H_{ii} & H_{ij} \\ H_{ij} & H_{jj} \end{pmatrix}, \qquad H_{ij} = \frac{\partial^{2}}{\partial x_i \, \partial x_j}(G_{\sigma} \ast f)

We can get this second derivative in two ways; call these: finite differences (FD); and convolution with second derivative of Gaussian (C2DG).

Quoting the docstring for hessian_matrix:

The distributive property of derivatives and convolutions allows us to restate the derivative of an image, I, smoothed with a Gaussian kernel, G, as the convolution of the image with the derivative of G.

∂∂xi(I∗G)=I∗(∂∂xiG)\frac{\partial }{\partial x_i}(I \ast G) = I \ast \left( \frac{\partial }{\partial x_i} G \right)

In words, the derivative of II convolved with the Gaussian, is equal to II convolved with the derivative of the Gaussian.

The Hessian is the second derivative, and has (for 2D) four matrix entries, as above.

Hessian Review goes into more detail, but here is the summary code from that notebook.

Skimage1 hessian_matrix has parameter use_gaussian_derivatives, with default None, where None currently maps to False, giving a FutureWarning that the default will change to True “in a future version”. This parameter and warning arrived with scikit-image version 0.20.0 (February 2023).

use_gaussian_derivatives=False (the default) corresponds to the FD algorithm above; =True is a version of the C2DG algorithm (but see below).

I don’t go into it here, but it turns out that the scipy.ndimage.gaussian_filter second-order kernels (order=2) break a couple of contracts for the second derivative; please see Hessian Review for detail, and this StackOverflow discussion for further background.

The broken contracts are:

The consequences for these two broken contracts are two-fold:

Now we have the background, let’s go to the issues

Issue 1: bug in calculation of Hessian matrix

Remember, hessian_matrix has two modes:

This issue is about the first use_gaussian_derivatives=True case, used by us in filters: meijering, frangi and sato, and therefore, via frangi, by the hessian filter (see What changes for the callers in the Hessian Review notebook).

I consider the correction to the calculation with use_gaussian_derivatives=True to be a bug-fix. The results of the filters above will change, generally slightly, typically at the image edges, but these changes are correct, and so need no compatibility wrappers to reproduce the old behavior. See the notebook link above for an analysis.

Similarly, anyone using use_gaussian_derivatives=True in the wild will see there results change, but we consider those results correct, and therefore, they will have a bug-fix.

Issue 2: removing finite difference (FD) method

use_gaussian_derivatives=False is the current default for hessian_matrix. But we can show that the alternative, fixed C2DG method is more accurate than FD at all sigmas. So we would prefer to remove the FD algorithm from hessian_matrix. But this involves a change in default output. We need to give users a way to reproduce their previous Skimage1 results using that default.

Who uses the FD default in our codebase?

So we expect no changes to the tests, for the upcoming Skimage2 versions of shape_index or hessian_matrix_det.

Who uses the FD default in the wild?

In a Claude (Opus 5.5) search, around 37 repos used use_gaussian_derivatives=False in calls to hessian_matrix, and around 4500 files call hessian_matrix without specifying use_gaussian_derivatives. The 37 will get errors when porting to Skimage2 (no parameter of that name); the 4500, and the 37, will need some method of reproducing the FD case when migrating.

I asked for another Claude search of Github for uses of hessian_matrix_det either using approximate=False or using it on 3D images, so that the routine falls back to the hessian_matrix default call. It found no instances. I don’t think we need to cover that case, but we might want to clean up the API — see below.

There were more users of shape_index — and the differences in output with the non-FD hessian_matrix were quite substantial. So these users need a migration path.

Plan implemented in the PR

  1. Refactored Skimage2 hessian_matrix to a) drop use_gaussian_derivatives and b) use the correctly sampled Gaussian second-derivative kernel. Test.

  2. Made a migration helper _skimage2.migration.hessian_matrix_finite_difference to reproduce the use_gaussian_derivatives=False behavior in Skimage1. Point people migrating to Skimage2 as this stopgap.