Add the ability to read floating point data files

[bzigon]

Hello, I would like to suggest an enhancement to Neural2D. Add the ability to read floating point data files instead of just BMP files. Keep it simple. Assume the files have an extension of .DAT.
The header could look like this.

1. 4 byte integer magic number -- to identify the file
2. 4 byte integer -- the width of the data
3. 4 byte integer -- the height of the data
4. 4 byte integer -- the number of channels (numbered 0, 1 , 2, etc)
5. 1 byte character -- S or D to signify the data is single or double precision data (4 bytes or 8 bytes)
6. a stream of width x height x channels x (4 or 8) bytes

Example. A 640 x 480 matrix of floats with R, G, and B "channels" would look something like

0000 0000 F00D FACE ! 4 byte magic number
0000 0000 0000 0280 ! hex for 640
0000 0000 0000 01E0 ! hex for 480
0000 0000 0000 0003 ! hex for 3
0053 ! ascii for 'S'
.... 1,228,800 bytes for channel 1 in row major format
.... 1,228,800 bytes for channel 2 in row major format
.... 1,228,800 bytes for channel 3 in row major format

Note : 640 x 480 x 4 = 1,228,800

Ideally my inputData.txt file could now look like this ...

images/thumbnails/test-1234.dat -1 -1 -1 -1 1
images/thumbnails/test-1235.dat 1 -1 -1 -1 -1
images/thumbnails/test-1236.dat -1 1 -1 -1 -1

Thanks.

[jeandudey]

In addition to this issue it would be good that the user can handle it's own image format.

How?

Creating a pure virtual base class with predefined methods and members to allow the users creates his own classes that inherits from the base class, why? to allow creating classes like BMPImage that have the same methods from the base class and used in neural2d core to process it.

[bzigon]

I kinda like that idea.

[bzigon]

I like the idea of Neural2D "doing 1 simple thing well". That's why I want to keep this floating point request simple. Now I am thinking through the tool chain.

Let's say I have 1000 BMP files in 24 bit RGB.

I would use ImageMagick to convert the BMP to RGB float using the quantum:format command.

http://www.imagemagick.org/Usage/formats/#rgb

I am not sure this is good, but the RGB file format has no header! Just raw data.

I can write a simple C program to read the RGB files, perform standardization on the data and write them out in DAT format so that they have the header for Neural2D.

Then Neural2D reads the .DAT files and trains on the .DAT data.

After I have the model trained I want to take pictures of stuff and have them classified.

I take a picture in BMP format.
I use ImageMagick to convert BMP to RGB with no header.
I run my C program to convert RGB to DAT.
I hand my DAT file with the weights to Neural2D and it classifies for me.

Hmmmm ....

[davidrmiller]

Thanks @bzigon and @jeandudey for the excellent ideas. I've made a branch that implements some design ideas for your consideration.

This branch introduces the concept of a image reader. The base class for an image reader has only one virtual function called getData(), which extracts the data from an image file and saves the data in a container of floats:

class ImageReader
{
public:
    virtual xySize
    getData(string const &filename, vector<float> &dataContainer, ColorChannel_t channel = NNet::R) = 0;
};

To support a new image file format, just define a new subclass of ImageReader and implement the getData() member function. Most everything else happens automatically.

I moved the BMP image reader into a new source file named imageReaderBMP.cpp. I also added imageReaderDat.cpp which implements an image reader for the floating point binary format suggested by @bzigon (with minor variations). I have not tested that code, so it probably doesn't work right at the moment, but it illustrates how to add a new image reader class.

There were a few design constraints that I tried to follow:

1. The design should should allow new image readers to be implemented very easily.
2. The filename extension should not be used to determine the file format. We should examine the data in the file to see what format it is.

Here's a review of the overall design: Each class Net instance contains a SampleSet object which manages everything related to the input data. The SampleSet object has a list of all the available image reader objects. When the SampleSet object wants to read an image file, it calls each available image reader in turn until one of them successfully reads the file data. If none of them are able to read the file, then we issue an error message about an unsupported file format.

[bzigon]

David

I like your approach. I also like that you are "eating your own dog food"
and derived the ImageReaderBMP class from the ImageReader.

Neural2D is all about teaching. You've made it easy to learn about
NN's and CNN's. These floating point capabilities are going to increase
the value of the tool. So now I'm thinking to myself .. "Does Neural2D need
to be accelerated by the GPU?"

What if I wrote the code that allowed it to be accelerated by the GPU?
I could follow your goal of using it to teach. Keep the GPU code straight
forward too.

What do you think?

Bob

On Fri, Jun 5, 2015 at 6:10 PM, David R. Miller notifications@github.com
wrote:

Thanks @bzigon https://github.com/bzigon and @jeandudey
https://github.com/jeandudey for the excellent ideas. I've made a branch
https://github.com/davidrmiller/neural2d/tree/neural2d-imageReader that
implements some design ideas for your consideration.

This branch introduces the concept of a image reader. The base class for
an image reader has only one virtual function called getData(), which
extracts the data from an image file and saves the data in a container of
floats:

class ImageReader
{
public:
virtual xySize
getData(string const &filename, vector &dataContainer, ColorChannel_t channel = NNet::R) = 0;
};

To support a new image file format, just define a new subclass of
ImageReader and implement the getData() member function. Most everything
else happens automatically.

I moved the BMP image reader into a new source file named
imageReaderBMP.cpp. I also added imageReaderDat.cpp which implements an
image reader for the floating point binary format suggested by @bzigon
https://github.com/bzigon (with minor variations). I have not tested
that code, so it probably doesn't work right at the moment, but it
illustrates how to add a new image reader class.

There were a few design constraints that I tried to follow:

The design should should allow new image readers to be implemented
very easily.
2.

The filename extension should not be used to determine the file
format. We should examine the data in the file to see what format it is.

Here's a review of the overall design: Each class Net instance contains a
SampleSet object which manages everything related to the input data. The
SampleSet object has a list of all the available image reader objects. When
the SampleSet object wants to read an image file, it calls each available
image reader in turn until one of them successfully reads the file data. If
none of them are able to read the file, then we issue an error message
about an unsupported file format.

—
Reply to this email directly or view it on GitHub
#21 (comment)
.

Bob

LinkedIn http://www.linkedin.com/pub/bob-zigon/6/658/85a

[davidrmiller]

@bzigon, I think the GPU is an awesome idea, and there's nobody I'd trust more to do the coding. I propose that you open a new issue for discussing it some more.

[davidrmiller]

The neural2d-imageReader branch is ready for review. It implements an image reader for the .dat binary file format as originally proposed, with minor variation. Tested on Linux, Cygwin, and MSVC.

There are two new .dat test files in the images/ subdirectory that need review to see if the format and byte order are correct. Those two test files contain three channels of 8x8 elements arranged in the same layout as the images/8x8-test.bmp test file, where the 8-bit pixel values are converted to floating point using the formula (n / 128.0 - 1.0). I added two unit tests to src/unitTest.cpp: one test reads the R channel in single precision; the other reads the G channel in double precision.

[davidrmiller]

Merged into master.