Monday, April 29, 2013

Data logging...

Logging data from "Things", I've run computer operated planes, rockets, cars, helicopters. All of these share a similar problem. How do you log data?  I like to log ALL raw sensor readings unmodified.  This ends up in a steam of multi rate data, ie I see raw IMU readings at 200Hz, Compass readings at 20Hz  GPS at 20Hz, and various things like mode changes and responses at varying rates from 200Hz to to once at power up.

Some things I've learned:


  • I don't trust a file system. If your thing crashes the data you are really interested in is the last few bits of data before impact. I've lost too much data to trust a file system to be robust when the recording media is de-powered and spattered all over the landscape
  • You can't squeeze everything you want to record in a telemetry stream.
  • I keep reinventing this same wheel.
  • In the course of a project the format of what you are recording evolves. Keeping the code for the data display and the data recorder in sync is difficult and often I end up unable to read earlier logs.
My current approach is as follows:
 Record everything to flash directly, it makes no difference if its a SD card or dedicated flash chip I use the same format.  I fully erase the flash (all the sectors on SD) then on power up I scan the flash page or sector by sector looking for the first blank page/sector.  I then start recording new data in that sector....

My data recording has a big circular buffer that records the data in stream format....
All the recorded data is in blocks with the format:
START ID ... (Next START)


Where START is a specific byte and ID is a numeric value representing the kind of data...
If the value START appears in the data is is escaped...The packet ends with  the next start....

I then have a background process that takes this circular buffer and writes it to flash sectors where there is a full sector worth of data.


Now I might record something like my GPS reading...


typedef struct
{
 double lattitude; //Radians
 double longitude;//Radians
 double ht;  //m
 unsigned long msec;
 unsigned short week;
 double ECEF_X;//m
 double ECEF_Y;//m
 double ECEF_Z;//m
 float hspeed;//m/sec
 float head; //radians
 float vv; //m/sec
 unsigned char flag1;
 unsigned char flag2;
 unsigned char nsat;
 WORD seq;
}CombinedTrimGps;


So that would be stored as

START GPS_ID the contents of the structure.....

Now if in the past if I changed the record so I added say a list of  SNR reading to the record I'd have to redo the code that decodes the GPS data from the data logs....

In an ideal world C++ would have introspection and it would be able to put the format of the structure into the data log the first time its used.....  then when the data reader sees the data it immediately knows what format the data is in....

Since C++ does not have introspection... I wrote some code that comes very close...


void LogRecord(CombinedTrimGps & item)
{
  LogStart(LOG_TYPE_BD960 ,"GPS");
  LogElement(lattitude,"lattitude " );
  LogElement(longitude,"longitude " );
  LogElement(ht       ,"ht        " );
  LogElement(msec     ,"msec      " );
  LogElement(week     ,"week      " );
  LogElement(ECEF_X   ,"ECEF_X    " );
  LogElement(ECEF_Y   ,"ECEF_Y    " );
  LogElement(ECEF_Z   ,"ECEF_Z    " );
  LogElement(hspeed   ,"hspeed    " );
  LogElement(head     ,"head      " );
  LogElement(vv       ,"vv        " );
  LogElement(flag1    ,"flag1     " );
  LogElement(flag2    ,"flag2     " );
  LogElement(nsat     ,"nsat      " );
  LogEnd();
}

Using Macros this expands into...



void LogRecord(CombinedTrimGps & item)
{
static bool bIntroed;
BYTE tid=LOG_TYPE_BD960;
if(!bIntroed)
{
bIntroed=true;
StartItemIntro(tid,sizeof(item),"GPS");
ShowElement(tid,&item,&item.lattitude,item.lattitude,"lattitude " ) ;
.
.//All the elements
.
}
LogRawRecord(tid,(const unsigned char *)&item,sizeof(item));
}

I have a different ShowElement  overloaded function for each type I might want to log....


Thus the first time this data type is logged it puts the format into the data log record....
This is something I've wanted since I first did the rocket stuff 6 years ago.

I've now got a self describing log format that is complete and efficient....
I can also use the exact same format for Telemetry....

On my rockets I had an optional parameter that went to the logging function that selected if the data went to the LOG, to telemetry or to both locations....

Last night I got the whole chain to work and the picture below is derived from this chain of events...
IE I drove the AVC car around my driveway -> recorded log -> process log generating records -> put in excel -> Then made a graph of locations....

Now I have to get the data replay tool I wrote a few years ago to read this format and my data logging tools will be complete!







3 Comments:

Blogger heroineworshipper said...

Logging has given way to screen capturing the ground station. As the sensors evolved to optical flow, machine vision, & the IMU has become a $10 jelly bean component, that's been good enough to fix any problem. It can't produce google earth plots, but those have gone out of style.

3:38 PM  
Anonymous Max said...

How do you time stamp the data? Is that just another type of data logged periodically? Or do you just infer time from the data rate? To be able to read stuff in a generic way, I guess one would have to assign _unique_ IDs to every kind of data (so either one for each GPS record component, or one for each different GPS record format), then teach his software to interpret every single one of those types, never re-interpreting or removing older data types.

@heroineworshipper: so how do you retrieve the data stream later from the screen cap? You OCR it...? Watching it replayed is nice but you can't put that on a graph. Besides - didn't he just say "You can't squeeze everything you want to record in a telemetry stream" ?

10:51 PM  
Blogger Paul Breed said...

The current set of data types from the car is:
LOG_TYPE_MSG//text messages
LOG_TYPE_IMU //Imu data
LOG_TYPE_RC_READ //Values read from RC receiver
LOG_TYPE_USER_VAR //Settings
LOG_TYPE_GYRO_ZERO //Gyro zero
LOG_TYPE_ODO //Odometer count
LOG_TYPE_MODE_CHANGE //Current mode manula automatic etc...
LOG_TYPE_HEALTH //battery voltage CPU usage
LOG_TYPE_COMPASS_V2 ///Compass calcs
LOG_TYPE_NAVSTATE //navigation sub system
LOG_TYPE_SUBMODE //Minor operating mode, straight, corner etc....
LOG_TYPE_BD960 //GPS info
LOG_TYPE_LOOP_VAR //PID loop values from steering.

The IMU happens at 200Hz and the GPS frame has gps week and gps msec values giving you current time.

5:53 AM  

Post a Comment

<< Home