Pareidolia, but underwater: What is under the green paint?

[Jorge_Stolfi]

Like a responsible parent, I have been trying to pass on to my computer my Superior Pareidolia skills. Specifically, the ability to see inked details that were painted over.

The Painter who applied the semi-opaque tempera colors often painted over inked outlines.   Examples are easily seen where these inked strokes were still dark and clear, like (A,B) below.  
   
Besides obscuring those strokes, it seems that the painting also washed away some of the ink, and sometimes deposited it a short distance away, as in (D).  

Thus any ink strokes that were already quite faint and faded, like (E), must have become invisible to the naked after being painted over.  And that is why we need Artificial Superior Pareidolia.

The idea is as follows.  

• Take an image of an area which is suspected of having "invisible" drawings or text under some semi-opaque paint.  
  
• Select a set of pixels A representative of what one wants to detect, like places where there is definitely ink covered by green paint. 
  
• Select one or more additional sets B, C, ... that are to be distinguished from A -- like places where there is green paint with but almost surely without ink underneath.  
  
• Look at the colors of those pixels as points of three-dimensional space, within the unit cube where (0,0,0) is black, (1,1,1) is white, (1,0,0) is red, etc.   Here is an example with three subsets of a page, representative of blank vellum (red), dark text ink (green), and green paint over blank vellum (blue):
             
  
• Approximate each cloud ou points A, B, C, ... by a trivariate Gaussian probability density function (PDF). This can be visualized as a fuzzy ellipsoid with varied dimensions along three axes, with some generic orientation in space.
  
• Take each pixel of the image and use Bayes's formula to estimate the probability that the pixel belongs to each distribution A, B, C, ...  or is an "outlier" that probably does not belong to any of them.
  
• Write one grayscale image for each set, showing the probability of each pixel belonging to that set.
   
  

Ideally we should do this with high-resolution uncompressed multispectral images with frontal illumination and linear encoding.  But we don't have multispectral scans for any of the pages that may have significant details hidden under the paint.  (The herbal pages have green paint, but the ink that can be seen under it is just boring nervures or leaf outlines.   At best, those images could be useful to validate this approach.)  And even those that we do have are taken with oblique lighting that creates light and dark spots at every tiny bump on the vellum surface.

So we must do with the Beinecke 2014 scans, which have frustratingly low resolution (some ink traces being only a couple of pixels across), only the three RGB color coordinates, oblique illumination, non-linear "gamma" encoding, and complex JPEG compression artifacts.  But, sigh, that is life...

[To be continued]

All the best, --stolfi

[Sorry for the big images, but I couldn't figure out how to insert only a thumbnail of the attachment, with the full version opening on a click.  Is that possible?]