CI Photocommunity

Register a free account now!

If you are registered, you get access to the members only section, can participate in the buy & sell second hand forum and last but not least you can reserve your preferred username before someone else takes it.

Extreme RAW

Steaphany

Well-Known Member
:eek: Warning - Extremely Techi Thread :eek:

Arvo and I have been corresponding and delving into the guts of the X3F file format. Arvo is working on a personal software development project and he needed reference X3F files which I provided. Along the way Arvo pointed me to a freeware Windows utility which will read and extract blocks of data and image regions from an X3F file:

The CEZEO
Please, Log in or Register to view URLs content!
is a cool little utility which can get an advanced photographer right to the X3F sensor data. This is as Raw as it gets.

Here is the CEZEO X3F.exe Window:

cX3F.jpg

The Properties and Cam data contain an assortment of data which I still have to review. My focus to date with X3F.exe is it's ability to extract image blocks.

The Small preview is just that, a tiny 221 x 147 pixel version of the image. When extracted by X3F.exe, the file type saved is a Windows Bitmap, .BMP, file.

Here is an JPG conversion of an example Small preview from the X3F file that I've been playing with:

Preview.jpg

*** More to come ***
 

Steaphany

Well-Known Member
The Jpg preview is an interesting image space, it contains a full resolution JPG version of the image. (And reviewers of the SD14 said that saving a RAW and JPG version of images was not possible) The X3F.exe extraction of this section provides a ready to use JPG. I've come to realize that programs like IrfanView, which claim to read X3F files, simply access this data area - ignoring the rest.

Now to the Raw data :z04_sabber: X3F.exe reads and saves the RAW image section as a 48 bit per pixel Tiff format file. If you do not use a Transformation matrix to manipulate and or scale the data, the high order bits will remain zeroed.

To present examples that I've done, I open the Tiff files in Photoshop and minimally process them by adding an Levels adjustment layer where I simply clicked Auto:

pslevels.jpg

and followed this by adding a Curves adjustment layer where I create a single point to shape the transfer function's curve:

pscurves.jpg

If you click to check the option to Apply Matrix to the RAW image, upon clicking Save selected section, you will get a window where you can specify the transformation matrix used to process and manipulate the data:

matrix.jpg

This is the lowest software level possible to perform initial image manipulations, everything prior is electronics and optics.

*** Not done yet ***
 

Steaphany

Well-Known Member
Since it will make comparison easier, here is a Jpg Preview extraction, of course reduced to fit the forum's requirements:

jpg.JPG

Please keep in mind that this image is what the SD14's internal software and processing deems as proper post processing.

The Matrix window of X3F.exe has a button where you can specify a Default Matrix which contains the values of:

Code:
Please, Log in or Register to view codes content!

and here is the resulting Default Matrix processed image:

DefMatrx.jpg

In a paper published in The Proceedings of the SPIE, Volume 5074, "Operation and performance of a color image sensor with layered photodiodes" by authors David L. Gilblom of Alternative Vision Corporation, Sang Keun Yoo of HanVision Co., Ltd., and Peter Venturac of Foveon, Inc., a transformation matrix is provided with the values of:

Code:
Please, Log in or Register to view codes content!

With this being the result:

gilblomMatrix.jpg

Arvo provided me with a transformation matrix which I believe he created by extracting header data transformation matrices from my supplied test images:

Code:
Please, Log in or Register to view codes content!

with this being the result:

Arvo_Matrix.jpg

Much :z02_respekt: to Arvo for coming up with this wonderful matrix. The image quality is right on and only slightly off from my SD14's internal processing.

A note regarding this matrix, it's SD14 specific:

Arvo said:
This is read from camera CAMF data (sensor to XYZ matrix multiplied to WB matrix), multiplied to XYZ to sRGB conversion matrix. Cofficients depend on temperature, but not heavily.

But like I once said, this matrix is not ideal either - for me it gives different results than SPP. So does extraction utility not use "spatial gain table", "bad pixel table" and other parameters, thereby results may have some banding, hot pixels and other artefacts.

It still produces a wonderful image and the colors of Ashlene's coat are more accurate than the SD14's.

I'll be posting in another thread how I used X3F.exe to reprocess my 24 images that I shot last November of the constellation Cassiopeia clearing up many of the problems that I had back when I was first working with them.
 

foveonfan

Well-Known Member
Steaphany, another big thank you. The tiny pinball machine between my ears is hardly up to following this in the first instance, but I assure you I have no intention of leaving it at that.

Sincere regards, JR
 

Arvo

Well-Known Member
In a paper published in The Proceedings of the SPIE, Volume 5074, "Operation and performance of a color image sensor with layered photodiodes" by authors David L. Gilblom of Alternative Vision Corporation, Sang Keun Yoo of HanVision Co., Ltd., and Peter Venturac of Foveon, Inc., a transformation matrix is provided with the values of:

Code:
Please, Log in or Register to view codes content!
One quick remark: this matrix is presented for Foveon to XYZ color space conversion, not to (s)RGB space - this may explain funny colors on image :) Probably this "deafult matrix" is for XYZ too.

You can find XYZ to sRGB conversion matrices on the net (somehow there are different versions of these - color science is so complex that even knowledgeable people disagree with each other sometimes). And you have to adjust results for different illuminants (white balance).

But I can agree that this little utility allows good play with image colors. For example you can try two black and white scenaries:

1. Balanced energetically - set all 9 coefficients equal, this way you get B/W image, not corresponding to human eye sensitivity, but to light energy. Using such approach you should get least noise - may be usable in astrophotography?

2. Balanced almost according to human eye - set coefficients 0 1 0 (or 0 X 0 - increase X to get proper white) in each row, this way only green channel data will be used. Foveon sensor green channel response is similar to human eye sensitvity (but not exactly of course).
 

Steaphany

Well-Known Member
I did calculate a transformation matrix combining the Foveon to XYZ space with a XYZ to sRGB space matrix also listing in the same paper.

The resulting image ended up far darker and with an overall Cyan cast. I just didn't include it in my post since the goal was sharing the capabilities of X3F.exe.
 
Top