Tags: Ideate, Prototype
Foreword: This month was full of ups and downs. Our progress has been slowed and interrupted multiple times due to the preparations for the Oslo Trip.
In the beginning of the month I started working intensely on our wearable; the bracelet. The idea was to give the audience a bracelet which will be used during the play. During the play, the audience will receive 2 options, so they could choose between them by raising or lowering their hand.
I had a meeting with our teachers, Daniel and Harry, in which I asked them what is the best approach of creating this concept. Both of them said that the best option would be to connect the bracelets to a server and within Unreal Engine 5, call the server for data. In short, the entire process would be:
Audience Raises/Lowers the Hand > The Gyroscope & Accelerometer of the Bracelet picks up the signal > The Bracelet sends the data to the server > The server stores and sorts the data > Unreal Engine 5 receives data from the server.
The research started by checking the hardware; what exactly was needed in order to achieve my goal. After checking out hardware stores, forums and talking with the teachers, I have found that the best option would be to use the Arduino Nano 33 IoT.
This small board is perfect in our case. For a bracelet we needed a small board which has the sensors necessary and can communicate with a server without the need of an external adaptor.
The necessary sensors were:
Accelerometer: The rate of change of velocity of the body with respect to time is called acceleration. According to relative theory, depending upon the relative object taken to measure acceleration, there are two types of acceleration. The proper acceleration, which is the physical acceleration of the body relative to inertia or the observer who is at rest relative to the object being measured.
The coordinate acceleration depends upon the choice of coordinate system and choice of observers. This is not equal to proper acceleration. Accelerometer sensor is the electromechanical device used to measure the proper acceleration of the object. (Source)
Gyroscope: Gyroscope sensor is a device that can measure and maintain the orientation and angular velocity of an object. These are more advanced than accelerometers. These can measure the tilt and lateral orientation of the object whereas accelerometer can only measure the linear motion.
Gyroscope sensors are also called as Angular Rate Sensor or Angular Velocity Sensors. These sensors are installed in the applications where the orientation of the object is difficult to sense by humans.
Measured in degrees per second, angular velocity is the change in the rotational angle of the object per unit of time. (Source)
When it comes to connectivity with the server, there are multiple ways of doing it but I decided to connect them via local Wi-Fi using either MQTT or HTTP protocols. I decided to use Wi-Fi because the Arduino Nano 33 IoT does have its own Wi-Fi module. (Source) So, connecting it to the Wi-Fi would be the easiest and convenient choice.
For the server, I have decided to use a Raspberry Pi 2. The main reason of using a Raspberry Pi is its affordability. In this case, the Raspberry Pi 2 was free for us, since we rented it from Saxion for this project, but a retail Raspberry Pi can be between 20 and 80 euros, depending on the specs. This Raspberry Pi 2 is an older model and we have encountered a few difficulties regarding it; since it's old, it is relatively slow, having only 1Gb of Ram but also its WiFi module is outdated, supporting only WiFi 2.4GHz.
The RAM limitation is acceptable, since our project does not require that much RAM. However, the lack of support for WiFi 5GHz was a big issue for us. Most WiFi routers, if not all, only have WiFi 5GHz; Eduroam and ACT-VR-LAB being 2 examples. Because of this, out Raspberry Pi 2 couldn't connect to these networks. To overcome this issue we had to create our own 2.4GHz hotspot, on our own expenses. The Arduino Nano had the same issue, so both of them required a 2.4GHz hotspot.
Coming back to our idea and the need of a bracelet. The idea was to allow the audience to participate in the play by raising or lowering their hand, in order to make choices. In order to create this, we would need to detect hand movement. In order to create software for an Arduino Board, I have to download their own IDE, called Arduino IDE.
There are a few other preparation I need to make before programming the board. For the Arduino to work properly, I had to install a few libraries on it: Arduino_LSM6DS3, Madgwick, WiFiNINA, ArduinoHttpClient. All of these libraries are in Lab 6 of Input Output from Year 1, 2022-2023. I have used that Lab as research since it is about communication using an Arduino.
Now, the other preparation are on the server side, on the Raspberry Pi 2. Here I also need to install a few things. The server already came with the Raspberry Pi OS preinstall, so I just had to update it to the latest version. Then I installed PHP, together with its extensions, and Flask.
After installing the libraries needed it was time to start working on the idea itself.
The first step would be to create the software on the server side.
This Python script creates a simple web server using Flask on a Raspberry Pi, exposing an endpoint ('/accelerometer_data') to receive data from an Arduino. When the server receives data through an HTTP POST request, it prints the accelerometer data to the console. The script is designed to be a basic foundation for handling data transmission from an Arduino to the Raspberry Pi. We could customize the receive_accelerometer_data function to suit our own specific needs, enabling tailored processing or storage of the received information. The server runs on the Raspberry Pi, making it accessible for data exchange between the Arduino and the Raspberry Pi over a local network.
After creating the script for the server, it was time to make the one for the board. I went through a lot of trial and error until I managed to make it work.
This Arduino code interacts with an LSM6DS3 accelerometer sensor and a Wi-Fi module, enabling the detection of a raised hand based on X-axis accelerometer readings. The system connects to a specified network, and it sends relevant data to a Raspberry Pi server when a hand is raised for a predefined duration. The code utilizes the Arduino LSM6DS3 library for accelerometer functionality and the WiFiNINA library for Wi-Fi communication. Key parameters such as Wi-Fi credentials, server IP address, and port are configured at the beginning. The Arduino continuously reads accelerometer data, monitoring the X-axis for values below a specified threshold to identify a raised hand. Upon detection, it records the start time and, if the hand remains raised for the required duration, it sends a message to the Raspberry Pi server via HTTP POST. Additionally, the system recognizes when the hand is lowered, transmitting a corresponding message. The code includes measures such as delays to regulate data transmission frequency.
After this, the Bracelet Prototype was complete. When the user raises their hand, the Arduino will detect a change and send and if the change corresponds with a hand raised, it will send a post to the Raspberry Pi, the server. On the server side, in the console it will display the ArduinoID and Hand Raised or Hand Lowered. ArduinoID is just a way of indexing the bracelet. With this, we can copy paste the entire Arduino Code on multiple different board ad give each of them their own ID. The server can now get data from multiple bracelets. This data can later on be requested by Unreal Engine.
This blogpost covered only the research and prototyping of the Bracelet Idea. This concept was later on removed during the Oslo Trip. I covered all the details about the Oslo Trip in this Blogpost (CLICK).
Comentários