Sensors - retrieve, stream, and save

[CaydenPierce]

We want to collect data from a number of sensors on the ASG and ASP and save them in a database.

Some sensors are sparse in time and can be saved directly to the database (e.g. GPS). Others are time series signals and must be saved to files that are referenced by the database (e.g. video).

Sensors

Audio: pull from microphone, encode audio on ASG, stream to ASP, save locally (how to save - in file chunks?)
Video: get from ASG camera in background (AndroidHiddenCamera), stream to ASP, save in files chunks
Accelerometer, Compass, Gyroscope, Ambient Light Sensor: ASG stream to ASP and save, ASP pull its own and save (head and body senors of user)
GPS: pull from android phone - no point in getting from ASG

Video

• find and implement better, encrypted video streaming from ASG to ASP
• save in file chunks (e.g. 30 second blocks) and save reference to each chunk with metadata in database

The state-of-the-art appears to be WebRTC:

• app streaming WebRTC from Vuzix M400: https://github.com/dalkofjac/smartglassy-demo-foi
• android WebRTC: https://github.com/renyuzhuo/WebRTC-Android-Learn

Keep in mind that Vuzix Blade and other Android smart glasses we are using now are old Android (Vuzix Blade == 5.1) so we may be limited in libraries we can use on the ASG side.

Audio

• stream encrypted raw data (or encoded with AAC or similiar) from ASG to ASP
• save in file chunks (e.g. 30 second blocks) and save reference to each chunk with metadata in database

Other

• get GPS every n seconds on ASP, save to database
• Accelerometer, Magnetometer, Gyroscope - get on ASG, stream to ASP, save to chunked files, and reference in database. Also pull on ASP, save to chunked files, and reference in database