FishNet Deep Neural Networks
Deep convolutional neural networks are the state-of-the-art technique for object recognition in images. FishNet is going to be a set of deep neural
FishNet A1
This network is for processing underwater image series (video) and detect human swimmers . It will probably incorporate design elements of recurrent networks. This network does not exist at the moment and is a research topic.
FishNet A2
This network is for analyzing swim technique based on motion data human swimmers as extracted from FishNet A1. Just like A1, it's a research topic at the moment.
FishNet B
This network is for processing image series and recognizing humans in a standing posture on a solid surface. Like FishNet A, it doesn't exist yet but is less of a research topic since similar networks have already been developed and published by others.
FishNet UV
This is a neural network for learning the motion control parameters of underwater vehicles. I will probably use reinforcement learning techniques for training: the UV needs to complete a underwater route while being penalized for list time and significant deviations from the route. Research project.
FishNet Builder
FishNet Builder is a software tool that I will be using to prepare training data for FishNet and to manage the training process.
Features
Features in version 0.1 are
- Extraction of frames of a video into a series of PNG images.
- A Python scripting console for processing images using OpenCV, Numpy, etc.
Usage
In version 0.1 there are two work areas; one for extracting images from videos, and one for experimenting with image processing scripts on images.
Videos Work Area
Cropping
Images Work Area
Image Processing Script
Your image processing script uses the in_image OpenCV matrix object as input and assigns the result to the out_image object. If a string object named debug is found its contents will be shown in the message area to the right.
User Settings
On Ubuntu Linux the user settings are saved to
~/.config/robo.fish/FishNet Builder.conf
where ~ represents your home directory.
Source Code
Availability
The source code is available on GitHub. FishNet Builder uses Qt and a number of cross-platform libraries. However, I'm developing this tool exclusively on Ubuntu Linux. You may need to modify the project settings to build on another flavor of Linux. You will definitely need to modify the settings for building on macOS or Windows.
Dependencies
- Qt Creator 3.5 with Qt 5.5
- libavcodec
- libavutil
- libavformat
- libswscale
- Python 3.6
- OpenCV 3 (core module, image processing module)
- Boost 1.67.0
On Ubuntu Linux 16.04 Xenial you can install most of these libraries with this command in the terminal:
sudo apt install qt5-default qtmultimedia5-dev qtquick1-5-dev \ libavcodec-dev libavutil-dev libavformat-dev libswscale-dev \ qtdeclarative5-dev qtdeclarative5-models-plugin qtdeclarative5-dialogs-plugin
and use Miniconda to install a Python 3.6 environment. Ubuntu 18.04 Bionic comes with Python 3.6 out of the box.
Please read the notes on configuring Boost for the projects featured on this web site.
The Ubuntu repositories have not upgraded OpenCV to version 3 yet. I recommend building OpenCV 3 from source.
License
The source code of FishNet Builder is published under a permissive BSD-style license that is compatible with commercial projects.