Arbalet Mesh (en)


#1

Arbalet Mesh project
A mobile and handy interface

The Arbalet Mash project is in line with others Arbalet Living Lab’s projects, which is being able to provide an introduction in computer science to everyone, this in a artistic and entertaining environment through large format “Pixel Art”
If you love in Bordeaux, maybe you noticed already the work of our association, such as our collaboration with the , or one the A1 building’s facade in the university campus


We are a team of 7 IT students from ENSEIRB-MATMECA engineering’s school whose decided to join Arbalet Living Lab project to help make the implementation simpler and cheaper to increase the accessibility of this learning tool to everyone.

Pixel Art

“Pixel Art” is an art style which reminds the art style of old-school video game, when the screen resolution didn’t allow back then to show more pixels, thus the drawing were often very minimalists and pixels were very visible, like the drawing below :


Drawing found dribble and Wikipédia

Arbalet Mesh principle

In our case, we are displaying images on very large surface, it is necessary for every pixel to have a micro controller, and so, they need to be synchronize between themselves. Before, it was necessary to link every pixel to a central unit which was responsible to send every pixel the information they need to function. But they were some drawbacks, for instance, every micro controller had to be physically linked to the central unit, which restricted severely the area of effect and caused some infrastructure problems, not to mention the long time it took.

Mesh Topology

The Mesh topology, or network communication system in mesh, is an organisation thought so every dot in the network is connected to his neighbours, here, information will be sent from one dot to his closest neighbour, until every component of the network gets the information. Using this topology has two main advantages :

  • If one the component were to be out of order, we can still communicate with the others components thanks to the re-routing principle.
  • We are not forced to link every component to the main unit, which help reducing the amount of wiring necessary for proper functioning (no network wiring necessary).

Our goal

The goal for us is to find an evolutionary and efficient implementation answering the needs of this project, in order to do this, we have to widen our knowledge about the different technologies and modules the more fitted for the project, from a software point of view (programming language), electronic (extension module and micro controller type) and material. We will have to realize a stable and robust software integration to produce in the end 6 connected pixels in real-time, and, hopefully show the final prototype by summer 2019.


#2

Pixels

Pixels will be installed indoor, on window sills. Moreover, they have to be identical and switchable.
To do so, they need to :

  • communicate with each others using Mesh topology,
  • light up in a selected colour which can change,
  • be powered.

Electronic components

1) Communication and color choice

Pixels communicate with each others by using a WifiMesh network. Each of them is composed of a ESP32 card wich ensure Wifi communication.
ESP32 cards having a microprocesseur and enough memory, they are autosufficient - there is no need of Arduino card nor RasberryPi card.
Their goals are to ensure the network communication and to modify the pixel colour when needed.

2) Lights

The LED strip WS2812B, which is used in other Arbalet Projects, is used once again.
Both his power requirement and logical level are at 5V.

3) Alimentation

A power supply unit provides to a Pixel the required energy.
ESP32 cards and WS2812B strips having different operating voltage (respectively 3,3V and 5V), the power supply has to provide each one of them with the right voltage.

Circuit diagram


#3

First iteration

In this first iteration, we first focused on our material, then decided on simple objectives in order to accustom ourselves with it.

Materials used

For this project, we will use :

  • 6 ESP32 card, each one corresponding to a pixel. They are aquipped with development board, being easier to work with.
  • 6 USB cables, 3 of 20cm lenght, and 3 of 1m length.
  • 2 chargers with USB port
  • 6 Alim 5V 2A
  • 6 femelle adaptator to be weld

We then decided on the objectives of this first iteration, focused on the communication within the network.

The network

Let’s first introduce the structure of the Network. There is the server, to which the clients will connect, which supervises the connexion to the Mesh Network. It selects which informations will be send to the Mesh network, about which pixels to light and how.

This server then relay these information to the mesh Network, composed of several nodes, each one being an ESP32 card. A node has different function :

  • Root node : node located on the entrance of the network, link with the server.
  • Intermediary node : node running code and relaying messages to other nodes.
  • Leaf node : terminal node, not having to relay messages.

The network being able to arrange itself in order to relay messages to all nodes as fast as possible, these roles are interchangeable between nodes, which also help to overcome quickly network failure.

Our objectives

We decided two focus on two main features :

  • Communication server - ESP32. First, we will use serial communication through USB cable. Then, we’ll check if it’s possible to use Wi-Fi connexion without creating interferences with the Mesh network.
  • Communication ESP32 - ESP32, base of the Mesh network. We’ll start by testing how to initialize a connexion between these two cards, then we’ll verify the Mesh network property by distancing them enough so they can’t communicate anymore, and adding a third one between them.

To make sure that all data are send and received properly, cards will send each others number, and a card receiving a number will blink a LED as many times as what they received. The cards will be programmed by using the Arduino IDE, setup to upload code directly on the cards.

Our objective with this is to implement a proof of concept of the Mesh network.


#4

Second iteration

Previous iteration results

As decided at the first iteration, the group implemented some proofs of concept in order to better understand the ESP32 cards. The serial interface was testes between a computer and a card, as also the network between a computer and a card and finally between the WiFi interface between multiples cards. The latter wasn’t completely successful, so new tests will be made to understand fully its behavior. More than that, it’s still necessary to do tests between a computer and a card without an access point.

The physical structure

The cards will be placed inside a compartiment connected to a tube. Inside the tube, there will be a ribbon of LEDs with an opening which allows that their light reaches the windows where the project will be installed. The following figure shows the structure’s idea.

The addressing protocol

For the installation to be fast and efficient, there should be an initial addressing protocol. First, the system needs to know the installation dimensions. After that, it should try finding the physical positioning of the cards in some automatic way, using the signals intensity as a metric to measure their relative distances. However, it’s possible that this initial search doesn’t recognizes perfectly the positioning, so it needs to be an interface in order to calibrate the wrongly positioned pixels. The current idea is having each pixel blink one after one in such a way that the calibration responsable can select, through an interface, where exactly the blinking pixel is located. Even so, the group should better think about the different ways to make this calibration easier.

Our objectives for the next iteration

The following objectives should be achieved for the next iteration:

  • Understand better the communication between the different ESP32 cards: looking at the MAC address of each card connected to the network, it’s possible to know from where our network-related problems are coming from
  • Find the standard range for the ESP32 cards: over IRC or microcontrollers’ forums, the group should try to find what is the ESP32 antenna’s range its users usually have
  • Test the communication between different ESP32 cards using the MDF library: a different choice of implementation which hasn’t properly worked at the previous iteration, different from PainlessMesh
  • Test the communication between a computer and a card through WiFi without an access point: the tests made used an access point to make the connection between the two of them, so now a direct communication should be tested
  • Test the Mesh network communication: after validating the previous tests, the last proof of concept is to establish the communication between a card connected (through WiFi) to a computer while also being connected to the other cards
  • Think about the addressing protocol which will be used to install the Arbalet Mesh’s structure

#5

Previous iteration results

A decrease in the bauds rate used allowed the ESP to increase in range. Empirical searches led to a confirmed 20m range for communication between two ESP32. ESP MDF library was successfully used for communication between two ESP32. An adressing protocol was realised.

Addressing protocol & communication

To implement the necessary initial addressing protocol between ESP32 and the server, and to successfully guarantee the sending and retrieving of messages between the electronic cards, we decided to realize 3 different frames :

  1. a beacon frame sent when a node connect into the mesh network, relayed by the neighboring nodes, including a frame number and the MAC address of the node joining the network ;
  2. a install frame, sent by the server during the initialization phase, including a frame number and associating with each pixel an id number ;
  3. a communication frame, including a frame number and an array, each case of the array containing RGB values, the place of the case in the array being directly linked with the id number of the corresponding pixel.

Our next goals

These are our next goals for the current iteration :

  • Analyzing antennas usefulness, especially the necessary PINs on the electronic card, the materials needed in order to have a “home-made” antenna, and the potential necessity to have special programing needed ;
  • Keeping the work going on the communication between ESP-ESP and computer-ESP, and realize a PoC ;
  • Choosing a framework between MDF and PainlessMesh ;
  • Building the previously mentioned frames.

#6

Fourth iteration

Previous iteration result

A formal description of the different frames (Beacon, Install and communication) was provided. The three frames were presented in the following form:

  1. Beacon : {n frame, @MAC}
  2. Install : {n frame, {@MAC1, … , @MACn}}
  3. Color : {n frame, {{r,g,b},…,{r,g,b}}}

An initial addressing protocol was proposed and many scenarios have been dealt with (the addition, the editing and the deletion of a node). The ESP MDF library was chosen for the rest of the project. A research on antennas was done in order to extend the scope of the ESP32 cards. A simultaneous communication between computer-ESP and ESP-ESP was established. The communication between the computer and the first ESP32 card is done through WIFI technology while the other is made by Mesh.

Manual addressing protocol with assistance:

The manual addressing protocol consists of recovering the MAC addresses of the ESP32 cards, then associate with each address its position in the plane. For this purpose, it was suggested to develop a front end part which allows for the administrator to provide the dimensions of the building (the height and the width) on a first page before a second page then appears in the form of a grid. The pixels begin to light up one by one while the administrator provides the position of the pixel turned on using the screen.

Our next goals:

  • Documenting the initial addressing protocol.
  • Documenting the readdressing protocol.
  • Implementing an assisted addressing protocol with a python script.
  • Realizing a configuration of the geometry of the backend by frontend.

#7

Fifth iteration

Back on the previous iteration

At the last iteration, we documented the assisted manual addressing protocol developing the utilisation of BEACON, INSTALL and COLOR frame and the utilisation of frontend.

We updated the document of specifications regarding the maximum size for one frame in an ESP network. The goal is to send a single COLOR frame for not dividing the data.

Demos

Addressing demo

The goal of this demo is to demonstrate that every ESP card stores its physic coordinates according to the structure. The idea is to blink the LED of an ESP card (x * y) times, this demo allows us to check if the coordinates (x, y) stored in the card match with its physic coordinates, except the case in which the card would be in (y, x).

COLOR frames demo

The goal of this demo is to demonstrate that every ESP card receives and processes the COLOR frame. The idea is that every ESP card linked to a LED switch on at the reception of the frame.

State Machine

A state machine is a finite automaton which lists the different states of our ESP cards and their transitions. This automaton allows us to tell the states of cards. Let’s see an example of state modification during the initial state. During this phase, the ESP card is not yet in a Mesh network, when it gets a Mesh network, the card sends a BEACON frame and wait for an INSTALL frame which gives it an ID then its state is now a listening state, a state in which it can run the COLOR frame.

Goals for the next iteration

  • Finishing the assisted manual addressing protocol, with the machine of states

  • Configuring the geometry of backend and frontend (Access to data using the database, user interface, …).

  • Addressing demo for all the structures feasible with 6 pixels

  • COLOR frame demo

  • Updating the document of specifications regarding the different failures that can happen

  • Documenting the re-addressing protocol