Core Module

Revision as of 2019-08-08T21:28:36 by Kai (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)


The Core module is a combination of an RTC with a camera, attached to a physical frame that can be mounted on or inside of a robot. The mounting frame and the electrical interfaces are designed to fit multiple robot designs so they can be reused if necessary. Each frame has multiple spaced sections that serve as connection handles. Typically you would clip the frame on a robot body with matching 16-millimeter hooks.

The design of the Core module is openly available on GitHub.


The Core software serves two purposes. First of all, it provides a library of functions that are used in multiple robots. These functions include low-level functions like

  • abstraction for sensors and actuators
  • camera control
  • communication with the RSA
  • communication with FishNet Builder

and high-level functions like

  • handling the dynamics of the robot body
  • computer vision with the FishNet neural networks
  • path planning
  • mission execution

Secondly, the Core software provides the harness in which the task control software of a specific robot runs. This harness takes care of the real-time, concurrent execution of sub-tasks and their ordering.


There are three hardware versions of the Core module:

  • Core Raspi
  • Core Jetson
  • Core Phone

Core Raspi

This is the initial design of the Core and consists of a Raspberry Pi RTC and a Pi camera attached to the Core frame. The camera is inside an enclosure that is attached to a bridge-shaped piece, which can be clasped on the Core frame. A wired connection and the SPI is used to communicate with the RSA. This version of Core was used in the LT0 device.

Core-mechanical-design-1.png Core-mechanical-design-2.png
Electrical Interfaces
  • The RTC communicates with the RSA in the robot body via the SPI standard.
  • The RTC connects to the camera either over the CSI-2 port
  • Power is supplied to the RTC by the robot body. Typically, the RSA controls the power to the RTC. The camera is powered by the RTC.

Software Considerations

On Raspbian you need to install the following repository packages: Core uses wiringPi to access the SPI pins on the Raspberry Pi in order to communicate with the RSA. To install wiringPi, type

sudo apt install wiringpi

On Raspberry Pi (version 3), Core uses the userland libraries,,,, to access the camera. Download the userland source code and build the libraries as follows:

sudo apt install git cmake
git clone
cd userland

This will automatically install the userland framework and code examples to /opt/vc.

Core Jetson

This version is conceptually similar to Core Raspi. The Raspberry Pi is, however, replaced by a Jetson Nano. The camera is still the same as in Core Raspi and the there is still a wired connection to the RSA that uses the SPI protocol.

Software Considerations
sudo apt install apt-utils ssh openssh-server tmux v4l-utils

Core Phone

In this version, the Core consists of a smartphone, an Apple iPhone Xs, that communicates with the RSA wirelessly via Bluetooth.